Explainers
No explainer here yet. This is the kind of cell that gets one as soon as something in the queue earns it.
Unread — 6
Kimi Linear: An Expressive, Efficient Attention Architecture
Kimi Delta Attention hybrid: a 48B/3B-active model cuts KV cache by up to 75% and hits 6× decoding throughput at 1M context.
arXiv 2510.26692 paper · Oct 2025Hybrid Architectures for Language Models: Systematic Analysis and Design Insights
Head-to-head study of inter-layer versus intra-layer attention+Mamba fusion, ending in concrete recipes for hybrids.
arXiv 2510.04800 paper · Apr 2026Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality
62 runs on WikiText-103 put five cosine-routing variants within 1 perplexity of each other — routing topology barely matters.
arXiv 2604.14419 paper · Apr 2026Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
Recurrent-depth transformers reach systematic generalisation via a three-stage grokking process, but excess recurrence degrades it.
arXiv 2604.07822 paper · Mar 2026ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
Drops the tokenizer: coding-rate segmentation of raw bytes keeps a static graph and beats both BPE and prior byte-level models.
arXiv 2603.03583 code · v0.5.1, Jun 2026flash-linear-attention — efficient implementations for emerging model architectures
Triton kernels for DeltaNet, Gated DeltaNet, RWKV-7, Mamba-2/3, NSA and KDA — the reference stack for these blocks.
GitHub — fla-org/flash-linear-attention