Assignment 2: Attention Mechanisms & GPT-2 Implementation

🚀 Assignment Overview

This assignment has two main components:

• Attention.py - Implement attention mechanisms from basic to multi-head (50 points)
• gpt2_model.py - Build GPT-2 components with modern optimizations (50 points)

Complete all TODO sections in the starter code. The implementations should follow the hints and maintain the original function signatures.

• Use the exact file names shown above (case-sensitive)
• Complete all TODO implementations in the starter code
• Maintain original function signatures and class names
• Each file should not exceed 5MB