Results
Latch generates GPU programs from models and learning rules. We test how fast they run and how much the compiler can handle.
Faster execution
These clocks compare complete updates of the same model and learning recipe. Each run uses one H100; the paired measurements were repeated on two H100s at fresh, mid and late training states.
| Workload | Latch | Conventional | What this demonstrates |
|---|---|---|---|
| TinyMoE tuned B2 × 1,024 | 13.63–13.77 ms | 37.50–38.42 ms | About 2.75× faster updates at this small batch |
| TinyMoE recipe Q, B32 × 1,024 | 68.27–68.61 ms | 114.02–114.77 ms | About 1.67× faster updates at batch 32 |
The fastest conventional control measured here compiles the trunk with TorchInductor and replays a CUDA graph. Its recipe-Q numerical limitation is described below. These are execution ratios, not data-efficiency gains.
Training to the same quality
TinyMoE-200M trains from random initialization on the same fixed text stream. Each comparison below keeps the recipe and batch size fixed between Latch and the conventional control. The target is validation loss 3.30. The full horizon is 655.36 million tokens.
| Program and recipe | Latch | Conventional | Measured scope |
|---|---|---|---|
| Current program, Q B32, seed 1740 | 857 s | 1,432 s | One full-run confirmation: about 40% less update time |
| Earlier compiler (c2), Q B32 | 885–941 s | 1,407–1,432 s | Three seeds; 0.629–0.659× the paired control time |
| Earlier compiler (c2), tuned B2 | 2,523–2,583 s | 7,044–7,340 s | Three seeds; 0.352–0.358× the paired control time |
The confirmation used the same generated program as this release. Its batch-2 full training results remain from the earlier compiler.
Update time excludes compilation, evaluation and checkpointing. In the Q confirmation, elapsed time from the first evaluation panel to the target panel, including evaluation and checkpoints, was 1,117 versus 1,766 seconds.
Numerical scope. Latch passes the tested fresh- and trained-state FP64 checks. The conventional control fails one trained-state gradient check: relative L2 error 0.0914 against a 0.08 limit, traced to TorchInductor rounding placement. It remains the fastest measured control, but is not numerically qualified.
Batch size changes the learning recipe. In the earlier full runs, tuned B2 needed fewer tokens than B32 to reach the target, but about 2.8× as much update time.
Other measured compiler work
These measurements used the earlier 0.9.0rc1 package. Their runnable examples remain included; these timings have not been remeasured for this release.
| Task | Latch vs control | What it demonstrates |
|---|---|---|
| Selective updates, B2 × 1,024 | 10.28 vs 66.45 ms per microbatch; 6.5× | Shared weights update every microbatch, experts every 16 |
| GradLev on GPT-2 small | 53.98 vs 189.53 ms per 1,024-step block; 3.5× | Dependent token-by-token adaptation; both use ordinary PyTorch for the rest of GPT-2 |
| Looped TinyMoE, B32 × 1,024 | 114.82 vs 154.22 ms per update; 1.34× | Repeated layers share weights and accumulate gradients across visits |
| TinyMoE cold build | 544.5 → 283.1 s on 64 vCPUs | Generation and compilation got faster with identical generated output; separate from GPU speed |
Different models and learning rules
The compiler determines operation order, synchronization and memory reuse from the input program. This release includes TinyMoE, selective updates, looped SMELT-style TinyMoE, GradLev and the fast-example families. Examples and installation.
E2-TTT, Granite and experimental distributed training remain research workloads outside this runnable release.
Latch is a research preview with a limited set of supported operations. The goal is to make changing a learning program enough to produce a new GPU implementation. How the compiler works.