50× speedup, guided by one benchmark
μ is a small array language — expressive, dynamically typed, interpreted. Making it fast without sacrificing generality is the core engineering challenge. This note documents the methodology: pick one real program, profile it ruthlessly, and fix only what the numbers justify. No speculative tuning, no benchmark-specific shortcuts — every fix must generalise to any μ program that hits the same pattern.
The program is a Monte Carlo GBM option pricer — a few dozen lines of μ that generate 10,000 price paths and price an Asian call. It is a good probe because it has three measurable phases, a clear C reference implementation to calibrate against, and a Python+NumPy version that sets a realistic target for an interpreted language.
clang -O3 -ffast-math -march=native, --quiet.
The benchmark as a probe
The core of the simulation:
bm: {[n] sqrt(-2.0*log(1.0-?n)) * cos(2.0*pi*?n) }
price_path: {[w] S0 * exp(+\((w*vol_dt)+drift)) }
za: {(bm nsteps)}' !npairs // Phase A: BM draws
pos_paths: price_path' za // Phase B: GBM paths
neg_paths: price_path' {neg x}' za // Phase B: antithetic
paths: pos_paths , neg_paths
finals: {last x}' paths // Phase C: reduce
payoffs: 0 | (finals - 1.05*S0)
price: exp(-0.05*T) * (+/ payoffs) % (2*npairs)
Three phases isolate costs cleanly:
| Phase | Expression | Python+NumPy | μ (current) |
|---|---|---|---|
| A — BM generation | {(bm nsteps)}' !npairs | ~22 ms | ~18 ms |
| B — GBM paths | price_path' za (×2) | ~25 ms | ~8 ms |
| C — payoff | {last x}', mean | ~6 ms | ~1 ms |
| Total | ~53 ms | ~46 ms |
Starting point: three root causes
Before any work, at 1,000 paths: mu takes ~287 ms, C takes ~2.4 ms. A profile immediately shows three causes:
-
Output printing (~64% of wall-clock).
Every assignment
x: valcallsval_fprint→__dtoa. For a 252-float array, 252 formatted writes per line.--quietalone drops 287 ms → 102 ms. -
Linear string scan per env lookup.
env_get("vol_dt")runsstrlen+memcmpagainst every live name in the frame (~16 entries). Called millions of times per run. -
Per-element heap boxing.
+\and element-wise*each allocate a freshvalue_tnode (24 bytes) per output element — 252 allocations per operation per path.
The eleven fixes — 102 ms to 5.5 ms
All measurements at 1,000 paths / 500 pairs, --quiet. The bar tracks
the running total; each fix removes one class of overhead.
Interned environment keys → 51 ms
- What
- All symbol names are deduplicated into a static bump-pointer buffer via
sym_intern().env_set_iname/env_get_inamecompare keys with pointer==instead ofstrlen+memcmp. The implicit argsx,y,zare pre-interned at startup. - Why
- The hot path calls
eval_expron every node, resolving names in a ~16-entry frame with string comparison. At 252 steps × 10k paths that is 2.5 million frame scans per run. - Impact
- 102 ms → 51 ms. The env scan was the single largest cost.
Packed-array arithmetic → 23 ms
- What
- Homogeneous float/long lists use
LIST_STORAGE_FLOAT/LIST_STORAGE_LONGbacked by a flatdouble[]/long[]. All arithmetic verbs,+\, and math keywords detect packed storage and operate with a singlefloat_alloc(n)+ one tight C loop. - Why
- Each previous arithmetic step allocated 252 × 24-byte
value_tnodes and avalue_t*[]items array. The inner loop for+\wwas memory-bound, not compute-bound. - Impact
- 51 ms → 23 ms. Also a prerequisite for the fused kernels in F7–F11.
RNG + each' env reuse → 17 ms
- F3 — RNG
verb_rand(n)previously allocatednindividualvalue_tnodes. Replaced with a direct write intofloat_alloc(n)using xoshiro256+ (period 2²⁵⁶−1, passes PractRand). Eliminates 252kval_alloccalls per path.- F4 —
each'env reuse - The handler previously copied the entire 16 KB
env_tstruct on every iteration. Fix: build the local env once from the captured frame, then per iteration resetlocal.lento the post-seed depth and rebind only the argument with oneenv_set_iname. - Impact
- 23 ms → 17 ms combined.
Scratch-pool reset → 13.5 ms
- What
- The bump-pointer pools are treated as a call stack inside
each'. The output slab is pre-allocated beforeeval_expr; all scratch lands above it. After the call, pool tops reset to just above the output — discarding all intermediates with no memcpy. - Why
- Without this, ~9 intermediate float slabs per
price_pathcall accumulated across all 10k iterations. Working set reached ~19 MB at 1k paths and overflowed at larger counts. - Impact
- 17 ms → 13.5 ms. Working set: 19 MB → 4 MB. Enabled the 10k-path default.
Scalar accumulation → 13.5 ms overall · Phase C: 32→1 ms
- What
- When
each''s body returns a scalar float, results are written directly into a pre-allocatedfloat_alloc(n)slab and returned as a packedFLOAT[]— no per-elementval_alloc. - Why
- At 10k paths,
{last x}' pathswas allocating 10,000value_tnodes and returning aLIST_STORAGE_GENERALarray, forcing downstream+/to unbox every element. Makingfinalsa packedFLOAT[]lets all downstream ops hit the fast path. - Impact
- Negligible at 1k paths, but Phase C at 10k paths: 32 ms → 1 ms. Critical for making the 10k-path default practical.
Function-body inliner → 11 ms
- What
- The optimiser stores inlined bodies back into
EXPR_FUNC_LITERALnodes at definition time. Whenprice_path:{[w] S0*exp(sums(incr w))}is defined,incrandsumsare substituted and scalar captures (vol_dt,drift,S0) are folded toEXPR_CONSTANT. The stored body becomesS0 * exp(+\((w*vol_dt)+drift))with all constants embedded. - Why
- Without this, every call to
price_path(w)resolvedincrandsumsby name and dispatched throughapply_func_oncetwice — two env lookups and two call levels per path. - Impact
- 13.5 ms → 11 ms.
GBM fused kernel → 9 ms
- What
- The optimiser pattern-matches
S0 * exp(+\((w*C1)+C2))in function bodies and replaces it with__gbm_path_fused__: a single C loop (prefix-sum log-price +expf), onefloat_alloc(252), no intermediate arrays. Structurally identical to the C reference. - Why
- Even with packed arrays, the AST-walked loop for
+\+*+expallocates three 252-float slabs per path. The kernel collapses all three into one output slab and lets the CPU pipelinefma+expf. - Impact
- Phase B at 10k paths: ~118 ms → ~19 ms. At 1k paths: 11 ms → 9 ms.
Normalisation pass → 8 ms
- What
- A new
opt_normalizepass runs beforeopt_expr: (1) fold all-constant subtrees toEXPR_CONSTANT(2*acos(-1.0)→6.2832); (2) substitute scalar env names into expression bodies; (3) reassociateCONST*(CONST*expr)→(CONST*CONST)*expr, collapsing2*pi*?ninto one multiplication and saving onefloat_alloc(252)per BM call. - Impact
- 9 ms → 8 ms. Also prerequisite for F10's pattern match.
BM fused kernel → 7 ms
- What
- Pattern-matches
sqrt(-2*log(1-?n)) * cos(C*?n)and replaces it with__bm_fused__: one loop, two xoshiro256+ draws per element, onefloat_alloc(n), zero intermediate arrays. - Why
- Box-Muller in AST form generates two 252-float RNG arrays then a chain of
intermediate slabs for
log,sqrt,cos,*— seven operations, seven allocations per path. - Impact
- 1k paths: 8 ms → 7 ms. Phase A at 10k paths: ~52 ms → ~44 ms.
Wrapper inlining + pre-resolved constants → 5.5 ms
- F11a — Wrapper inlining
- The benchmark calls
bmvia a closure:{(bm nsteps)}'. After F10,bm's stored body wasAPPLY(__bm_fused__, [n, 12.5664]). The inliner checked if the captured frame was safe to inline through — it was not, because__bm_fused__hasbody=NULL(a bare verb alias), andframe_is_inlinable_for_bodytreated anybody==NULLfunction as unsafe.
The fix: distinguish fused kernel sentinels (uppercase verb character defined inopt.h) from arbitrary unknown closures. Fused kernels are compile-time constants — they never close over mutable state. Allowing them as inlinable unblocks the substitution, exposingAPPLY(__bm_fused__, [252, 12.5664])directly in the adverb body. - Impact of F11a
- Phase A at 10k paths: ~44 ms → ~18 ms. Now matches C's ~18.5 ms — the BM loop in μ and C are structurally identical.
- F11b — Pre-resolved fused constants
- All
make_*_fused_callbuilders previously emittedEXPR_NAME("__gbm_path_fused__")as the function node. At runtime,eval_exprresolves everyEXPR_NAMEvia a linear env scan — on every iteration of everyeach'loop.
The fix: emitEXPR_CONSTANT(val_func(NULL, VERB_*_FUSED, ...))instead.eval_expronEXPR_CONSTANTreturns the embedded value with zero env lookup. A new helpermake_fused_fn_node(char verb)centralises the pattern. - Impact of F11b
- Phase B at 10k paths: ~19 ms → ~8 ms.
Current state: head-to-head at 10,000 paths
| Implementation | Total | Phase A | Phase B | vs Python |
|---|---|---|---|---|
| mu — unoptimised | ~1,000 ms | — | — | ~19× |
| mu — after F1–F9 | ~78 ms | ~52 ms | ~19 ms | ~1.5× |
| mu — after F10 | ~70 ms | ~44 ms | ~19 ms | ~1.3× |
| mu — after F11 (current) | ~46 ms | ~18 ms | ~8 ms | ~0.87× |
| Python + NumPy | ~53 ms | ~22 ms | ~25 ms | 1× |
C (-O3 -ffast-math -march=native) | ~24 ms | ~18.5 ms | ~3.6 ms | ~0.45× |
Current μ at ~46 ms is faster than Python+NumPy (~53 ms) on this workload.
Phase A is well ahead (~18 ms vs ~22 ms). Phase B is where the gains are largest:
μ's fused GBM kernel (~8 ms) beats NumPy's vectorised path construction (~25 ms)
because NumPy must allocate and broadcast full n_paths × n_steps matrices,
while μ processes each path independently with a single tight C loop.
The remaining overhead vs C is Phase B's per-path float_alloc and
function dispatch — architecture-level cost, not algorithmic.
What was not done, and why
Antithetic-path fusion.
The benchmark runs price_path on both za and
za_neg. A benchmark-specific optimisation could fuse both passes,
halving Phase B's iteration count. Rejected: it does not generalise to any other
μ program. The general fix (F11b) eliminated the per-call overhead instead.
SIMD / vectorised BM. NEON intrinsics could give 2–4× throughput on the BM kernel. Not done: Phase A is already at C parity. Complexity without proportional gain.
What comes next
μ is now faster than Python+NumPy overall. The interesting remaining question is whether the per-phase advantage can be widened. Phase B (~8 ms) is the main target — NumPy takes ~25 ms here, so there is headroom, but also a structural cost:
-
Per-path allocation and dispatch. Each of the 10,000 calls
to
price_path'entersapply_func_onceand allocates afloat[252]output slab. Batching all paths into a single 2D kernel (n_paths × n_steps) would eliminate this but requires a different adverb evaluation strategy. -
The
negpass.{neg x}' zacallseval_exprper element — O(n_paths) AST dispatch. Recognising{neg x}'as a map-negate kernel is a straightforward extension of the existing fusion patterns.
The {neg x}' map-negate pass is the simplest win: recognising
it as a built-in map kernel would eliminate O(n_paths) eval_expr
dispatch with a trivial pattern addition. Beyond that, closing further vs C
requires the 2D batch kernel — a larger architectural change with diminishing
returns given the Python comparison is already won.
General lessons
- Profile before fixing. F1–F3 alone covered ~80% of the total reduction. Guessing would have produced far worse leverage.
- One bottleneck hides the next. String comparison → boxing → rand allocation → env copy → pool pressure — the waterfall shape in the fix table is not coincidence.
- Every fix must generalise. Interned env, packed arrays, and fused kernels all benefit any μ program that hits those patterns.
- Fused kernels are a force multiplier. F8 + F10 together dropped Phase B from ~118 ms to ~19 ms, not by being clever, but by writing structurally identical code to the C reference.
- Guard conditions in the optimiser need care. F11a's
conservative safety check in
frame_is_inlinable_for_bodysilently blocked a correct, high-value transformation. The fix was two lines; the diagnosis required understanding the full opt pipeline.
Timings: Apple M-series, clang -O3 -ffast-math -march=native,
./mu --quiet --time. μ commit: main, May 2026.
Source: experiments/montecarlo/.