Skip to content
Noah Boufercha

Location

Leonberg, DE

Deep learning · Python · NLP

Transformer language model from scratch

Architecture diagram of the model: token embedding, transformer blocks, softmax
Period
2026
Stack
  • [Python]
  • [PyTorch]
  • [Transformer]
  • [RoPE]
  • [NLP]
Code
Private repository

A GPT-style language model implemented entirely from scratch, following Stanford CS336: BPE tokenizer, multi-head self-attention with RoPE, transformer blocks, AdamW optimizer and training loop.

Another project

AI pre-selection of applicationsView project