AX Engine. APPLE SILICON · LOCAL INFERENCE
Past the DRAM ceiling.On your Mac.
01 / Dense / 6-bit 2026-09-21
Measured on one Mac. Shown in context. M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification
Decode tok/s · 256 tokens · greedy · median of 20 runs (4 prompts × 5 reps)
Qwen 3.8 27B · 6-bit · flappy 1.04× vs MTPLX · treat as parity 0 40 80 tok/s
Qwen 3.8 27B · 6-bit · long_code 1.26× vs MTPLX · 451–852 prompt tokens 0 40 80 tok/s
AX / MTPLX: MTP depth 3 · OMLX: depth 1 · mlx-lm: direct AR
02 / MoE / MXFP4 2026-09-21
Measured on one Mac. Shown in context. M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification
Decode tok/s · 256 tokens · greedy · median of 20 runs (4 prompts × 5 reps)
Qwen 3.6 35B-A3B · 6-bit · flappy 1.87× vs MTPLX · MTP decode 0 125 250 tok/s
Qwen 3.8 27B · MXFP4 · flappy 1.20× vs MTPLX · MTP decode 0 125 250 tok/s
AX / MTPLX: MTP depth 3 · mlx-lm: direct AR · OMLX could not load these raw packs.
Prefix cache off · n-gram stacking off · AX Engine 7.5.3 @ c5d22c4a All results & methodology ↗ 03 / Gemma / direct 2026-09-21
Measured on one Mac. Shown in context. M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification
Decode tok/s · 256 tokens · greedy · median of 20 runs (4 prompts × 5 reps)
Gemma 4 12B · community 4-bit 1.13× vs OMLX · direct AR, no MTP 0 80 160 tok/s
Gemma 4 26B-A4B · community 4-bit 1.06× vs mlx-lm · treat as parity 0 80 160 tok/s
Community packs, not AXQ. All measured lanes are direct AR; no Assistant-MTP sidecar.
Prefix cache off · n-gram stacking off · AX Engine 7.5.3 @ c5d22c4a All results & methodology ↗ 04 / Tiel · MXFP4 ↗
TIEL / 35B A3B · MXFP4 MTP Completion (tok/s) M5 Max 128 GiB · python-lru · 2026-09-20 MTPLX 2.11.3 sustained 177.44
0 55 110 165 220 tok/s
Includes time to first token · 6-sample median
AX Engine 7.4.0 · 574d2d19 + fix 5e07befe Source report ↗ Local inference on your terms.
✳ MAC-FIRST INFERENCE RUNTIME
Dense 27B past the DRAM ceiling. Apple Silicon inference with speculative MTP decoding. One AXQ 6-bit pack, native Metal graph, OpenAI-compatible server on 127.0.0.1:31418.
+ ● ● ● ax-engine / performance ↗
Qwen 3.8 27B · 6-bit · flappy Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 40 80 tok/s
1.04× vs MTPLX · treat as parity
AX / MTPLX: MTP depth 3 · OMLX: depth 1 · mlx-lm: direct AR
Source artifacts ↗ Qwen 3.8 27B · 6-bit · long_code Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 40 80 tok/s
1.26× vs MTPLX · 451–852 prompt tokens
AX / MTPLX: MTP depth 3 · OMLX: depth 1 · mlx-lm: direct AR
Source artifacts ↗ Qwen 3.6 35B-A3B · 6-bit · flappy Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 125 250 tok/s
1.87× vs MTPLX · MTP decode
AX / MTPLX: MTP depth 3 · mlx-lm: direct AR · OMLX could not load these raw packs.
Source artifacts ↗ Qwen 3.8 27B · MXFP4 · flappy Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 125 250 tok/s
1.20× vs MTPLX · MTP decode
AX / MTPLX: MTP depth 3 · mlx-lm: direct AR · OMLX could not load these raw packs.
Source artifacts ↗ Gemma 4 12B · community 4-bit Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 80 160 tok/s
1.13× vs OMLX · direct AR, no MTP
Community packs, not AXQ. All measured lanes are direct AR; no Assistant-MTP sidecar.
Source artifacts ↗ Gemma 4 26B-A4B · community 4-bit Decode (tok/s) M5 Max 128 GB · 2026-09-21 · Campaign evidence, not certification 0 80 160 tok/s
1.06× vs mlx-lm · treat as parity
Community packs, not AXQ. All measured lanes are direct AR; no Assistant-MTP sidecar.
Source artifacts ↗ Tiel 35B A3B · MXFP4 MTP Completion (tok/s) M5 Max 128 GiB · python-lru · 2026-09-20 · Development pack, not certified MTPLX 2.11.3 sustained 177.44
0 110 220 tok/s
Includes time to first token · 6-sample median
Full-resident native API · Explicit throughput-MTP. Separate workload from Qwen decode; not default-server speed.
Source artifacts ↗ ← 1 / 7 → Pause charts
See the speed benchmarks↗
⌘ macOSmacOS 26.2+ · Apple Silicon (M2 or newer) 5 supported model families 76.90 tok/s decode · M5 Max 128 GB Apache-2.0 open source AX ENGINE / Models
Supported models. AX Engine supports Qwen 3.5 / 3.6 / 3.8, Gemma 4, GLM, Nemotron, Whisper, and embedding models through a direct-first runtime contract. The qualification pack is Qwen 3.8 27B AXQ 6-bit MTP; other families stay supported but are not the first-run target.
Chat
27B Qwen 3.8 27B AXQ 6-bit MTP — `qwen3.8-27b:axq`
Coding agent
MoE Ornith 1.5 35B-A3B coding agent
Vision chat
VL Qwen3-VL 30B-A3B AXQ
Full model matrix, certifications, and MTP pack notes ↗ On the horizon AX Engine 7.5.x ships 6-bit AXQ MTP for Qwen 3.8 27B (SKU: Mac mini M4 Pro 64 GB) and Tiel/Cyber-Tiel MXFP4 MTP packs (campaign: M5 Max 128 GB).
See the roadmap → ✳ Your inference
on your Mac. Dense 27B, MXFP4 MoE, native MTP, OpenAI-compatible endpoints. Run it on the machine you already own.
Install AX Engine↗ 