Founding Cohort — 150 seats

microGPT Foundations

The Karpathfinder

Inspired by one of the OGs of deep learning.

An unofficial microGPT course for learning GPTs from first principles.

Andrej Karpathy showed the world that you can understand neural networks by building them from scratch — one line at a time.

This course follows that philosophy. You'll walk through a tiny GPT implementation, understand every block, and build your own variant.

No hand-waving. No black boxes. Just code, math, and clarity.

Join the Founding Cohort Get the Free Lesson

Founding: $695 Core / $1,095 Verified

One-time payment · Lifetime access · 14-day refund policy

gpt.py

1classGPT(nn.Module):

2 def __init__(self, config):)

3 super().__init__()

4 self.tok_emb = nn.Embedding(config.vocab_size, config.n_embd)

5 self.pos_emb = nn.Embedding(config.block_size, config.n_embd)

6 self.blocks = nn.Sequential(

7 *[Block(config) for _ in range(config.n_layer)]

8 )

9 self.ln_f = nn.LayerNorm(config.n_embd)

10 self.head = nn.Linear(config.n_embd, config.vocab_size)

~ 200 lines. That's the whole model.

The Problem

You're still treating GPTs like magic.

Most people use LLMs every day without understanding a single line of how they work.

You've watched the YouTube explanations, but the code still feels opaque.

You can call an API, but you can't reason about what's happening inside.

You know the buzzwords — attention, embeddings, transformers — but not the mechanics.

You hesitate to modify model code because you don't trust your own understanding.

The Mechanism

One tiny codebase. Complete understanding.

Inspired by Karpathy's approach: build it yourself, understand everything.

You'll follow the complete data path through a working GPT implementation:

raw texttoken idsembeddingsattentionlogitslossgradientsoptimizationsampling

By the end, you won't just "kind of get transformers."

You'll be able to open the file, point at any line, and explain exactly what it does and why it's there.

The Trail Map

Follow the path from tokens to logits.

Seven modules. One codebase. Each checkpoint builds on the last until you can explain, modify, and build your own tiny GPT.

MODULE 0

Trailhead

Orientation, prerequisites, and what a GPT is actually trying to do. Set up your environment and understand the landscape before the hike begins.

Learner understands the project scope, has a working environment, and can articulate what next-token prediction means at a high level.

MODULE 1

Token Trail

Documents, BOS, vocabulary, tokenization, and next-token prediction. Understand what the model sees and what it's trying to predict.

Learner can explain what the model is predicting and why, trace from raw text to token IDs, and describe the vocabulary.

MODULE 2

Gradient Gorge

Bigram intuition, loss functions, autograd, backpropagation, and learning dynamics. Where the actual learning happens.

Learner can explain where learning happens, how parameters update, and what the loss function is measuring.

MODULE 3

Attention Pass

Embeddings, positional information, self-attention, residual connections, MLP blocks, and layer normalization. The transformer core.

Learner can trace a complete forward pass and explain every major component of the transformer architecture.

MODULE 4

Optimizer Ridge

Adam optimizer, training loops, logits interpretation, sampling strategies, temperature, and inference. From training to generation.

Learner can configure training, interpret logits, and generate text with different sampling strategies.

MODULE 5

Summit Project

Modify the model, swap the dataset or architecture, and build your own tiny GPT variant. Ship something real.

Learner ships a working variant and can defend every modification they made.

BONUS

Beyond the Map

What scales from tiny GPTs to real-world systems, and what changes in production. The bridge from learning to building.

Learner understands the gap between microGPT and production systems, and knows where to go next.

View full curriculum details

The Transformation

From black box to builder.

Before

Buzzwords without mechanism
Tutorials without intuition
API usage without understanding
Confidence without competence

After

Explain every major block in a transformer
Trace the full forward pass with confidence
Modify architecture and understand the tradeoffs
Build and ship your own tiny GPT variant

What's Included

Everything you need to follow the path.

7-Module Curriculum

From tokens to sampling, every concept built on the last

Gated Member Portal

Track your progress, complete quizzes, build your understanding

Quizzes & Articulation Checks

Prove you understand — don't just watch and nod

Capstone Project

Build your own GPT variant and defend your choices

Certificate of Completion

A verifiable credential that proves real understanding

Private Community

Learn alongside other builders in our Circle community

Lifetime Updates

As the field evolves, so does the course

Free Lesson Preview

Try before you commit — no credit card required

Referral Rewards

Share the path and earn rewards

Built For

Developers

Who want to understand what's actually inside the models they use every day

ML Aspirants

Building first-principles intuition before diving into larger frameworks

AI Builders

Shipping products on top of LLMs who need to understand the engine

Technical Creators

Who teach, write, or explain AI and want to go deeper than surface-level

Not For You If

You want high-level summaries without touching code

You refuse to read Python or work through implementations

You want passive entertainment, not active mastery

You think understanding means watching someone else understand

Inspired by the pioneers of AI education

The Philosophy

Inspired by the way the OGs teach.

Andrej Karpathy changed how people learn neural networks. Instead of starting with theory and working down, he starts with code and works up. Build it, run it, break it, understand it.

The Karpathfinder follows that same philosophy. You won't watch lectures about transformers — you'll build one. A tiny, readable GPT that fits in a single file. Every line explained. Every block justified.

"What I cannot create, I do not understand." — Richard Feynman

Your Guide

Lloyd Clark

[Instructor bio — add your background, credentials, and what drives you to teach this material. Keep it honest and specific. No fake credentials.]

[email protected]

14-Day Clarity Guarantee

If the first two modules don't create genuine clarity about how a tiny GPT works, email us and we'll refund your purchase. No questions, no hoops. We're confident in the path.

Choose Your Path

Pricing

One-time payment. Lifetime access. No subscriptions, no upsells, no surprises.

Free Lesson

Try the first lesson and diagnostic quiz before you commit.

First lesson on tokens & prediction
Diagnostic quiz
No credit card required