-
One Launch for Any Batch: The Binary Search Inside vllm-metal's Varlen Attention
How vllm-metal's paged attention kernels resolve which sequence owns each token with a per-threadgroup binary search, what that buys continuous batching on Apple Silicon, and what happened when we replaced it with an O(1) lookup map.
-
vllm-metal vs mlx_lm: Contiguous vs Paged Varlen KV Cache
Why we replaced mlx_lm's attention layer with paged varlen attention driven by the vLLM scheduler, and the deployment cliff that motivates it.
-
I turned my Timeular Tracker into a macOS macropad
Flipping an 8-sided dice to switch macOS desktops, built end to end with Claude Opus.
-
Rethinking vllm-metal's Memory Budget for Apple Silicon
Exploring memory allocation strategies for LLM inference engines on Apple Silicon's unified memory architecture.