OCaml Workshop 2026: paper (PDF) · talk slides. This post is the less filtered, longer-form version.

TL;DR: I built simdjson-oxcaml using OxCaml’s performance features. It is 5× faster than existing OCaml JSON parsers. I can make it 10× faster when user-defined unboxed variants land.

❤️ Unboxed types and SIMD are brilliant. More, please!

🤔 Ecosystem maturity and support for non-value layouts are still rough.

💀 Templates feel awkward. Layout polymorphism when?

And as for a GC’d language with a clean escape hatch when you need more performance… I’m not so sure now.

OxCaml¹ is Jane Street’s performance-oriented version of OCaml, with features inspired in part by Rust.² OxCaml had been on my radar, but Anil Madhavapeddy’s post about a zero-allocation HTTP server was what made me take a closer look. Fortunately, I also had a particular problem in mind: make JSON parsing go brr by implementing a simdjson-style parser in native OxCaml.

I didn’t want to copy simdjson blindly. Its borrowed DOM is a natural fit for its reusable tape, but I wanted an API that felt natural in OCaml: an ordinary, owned Json.t variant that callers could easily pass around and pattern-match on. That output contract introduced its own constraints and design choices, particularly around allocation and the garbage collector.

So simdjson-oxcaml³ exposes two APIs. The main one returns that owned Json.t; a lower-level borrowed tape isolates the cost of materializing the tree and makes comparison with C++ simdjson fairer. On simdjson’s benchmark corpus, the owned parser reaches about 730 MB/s—roughly five times the throughput of existing OCaml parsers—and the tape reaches 1.25 GB/s. That’s the fast part. The not too fast part is that even the borrowed tape remains 2.7 times slower than the original C++ implementation.

SIMD vectors, unboxed types, and bit-manipulation intrinsics let me express the algorithm directly in OxCaml. Making it fast still required profiling, benchmarking, and reading disassembly. CPU-level techniques transferred from C++. Allocation strategies had to fit OCaml’s garbage collector. Some combinations of the new features also needed extensive annotations or library workarounds. The result was not C++ in OCaml syntax; it was a faster OCaml program with OCaml-shaped tradeoffs.

A simdjson primer

Simdjson separates byte classification from grammar parsing (the figure below).⁴ Its scanner produces a structural index. A top-down parser follows that index and writes the parsed document to a flat tape. The library’s DOM API is a navigation layer over the tape rather than a separately allocated tree.

Scanner and structural index

The scanner processes 64-byte blocks. It tracks quotes and escapes, validates UTF-8, and classifies punctuation, whitespace, and the first byte of each number or literal. It uses bit masks to track strings and vector table lookups to classify bytes.

The output is a structural index: byte positions at which parsing decisions can occur. In the figure below, it marks punctuation, the opening key quote, and the starts of 12 and true. It does not parse either atom; it lets the parser skip irrelevant positions.

C++ simdjson expands those masks into a flat array of 32-bit byte positions. The parser can then jump directly from one relevant position to the next.

Parser and tape

The second stage is a top-down parser over the structural stream. It recurses into containers and dispatches quotes and atoms to string, number, or literal routines. SIMD remains useful in string parsing. Grammar and numbers are mostly scalar.

The tape stores parsed entries in document order. An opening container points to the first entry after its scope, and its closing entry points back to the opening entry (the figure below). Decoded strings live in a side byte buffer. These links let a DOM handle skip a complete value or iterate over a container without allocating a tree node for each value.

Simdjson’s DOM values are lightweight handles into the tape. The parser owns and reuses that storage, so parsing another document with the same parser invalidates the previous document and any values obtained from it (simdjson DOM API).⁵

simdjson::dom::parser parser;
auto doc = parser.parse(input).value();
auto name = doc["name"];

auto next = parser.parse(next_input).value();
// doc and name are now invalid

Simdjson benchmarks one warmed parser that reuses these buffers.

The simdjson-oxcaml parser

APIs and result

I applied this architecture in OxCaml, but made the main result an owned algebraic data type:

module Json : sig
  type t =
    | Null
    | Bool of bool
    | Int of int
    | Float of float#
    | String of string
    | Object of assoc array
    | Array of t array
  and assoc = { key : string; value : t }
end

The returned Json.t is independent of the parser: code can pattern-match on it and parse another document without invalidating it. A second API exposes the borrowed tape. The two paths share the scanner and parser, so their difference measures tree materialization.

The chart below gives the main result. The reusable and one-shot DOM rows return the same Json.t. Only the former keeps grown scratch buffers between calls. The benchmarking section at the end describes the setup.

The owned parser is five times faster than the existing OCaml rows; the tape is faster again but remains well behind C++ simdjson. The features behind the improvement are standard in systems languages. A five-fold gain from adding them suggests that regular OCaml can leave meaningful performance untapped when code cannot express the representations or operations it needs.

What OxCaml made possible

Four OxCaml features let me implement the complete parser in OCaml, including the SIMD-heavy scanner and string decoder.

Unboxed types

Unboxed types carry data outside the ordinary OCaml value layout. In the OCaml value representation,⁶ a standalone float normally points to a heap block; float# carries the payload directly (the figure below). Unboxed products and records hold scanner state and results without intermediate blocks. Built-in variants such as or_null similarly avoid an option block.

SIMD support

OxCaml exposes built-in SIMD vector types. Loading 16 bytes produces one int8x16#; comparing it with a vector containing 16 quote bytes performs the same comparison in every lane (the figure below). Four such loads cover one 64-byte scanner block.

Bit-manipulation intrinsics

Population count and count-trailing-zeros intrinsics count structural positions and extract the next set bit from a mask. When lowered correctly, each operation becomes a single CPU instruction: popcnt or tzcnt.

Stack allocation and allocation checks

stack_ allocates a local value on the stack instead of the GC-managed heap. In the example below, both records are allocated, but the stack-allocated record is reclaimed when the function returns.

type point = { x : int; y : int }

let make_points () =
  let heap = { x = 1; y = 2 } in
  let local = stack_ { x = 3; y = 4 } in
  print_int (local.x + local.y);
  heap

[@zero_alloc] verifies that a function does not allocate on the OCaml heap. [@zero_alloc opt] performs the same check after optimization. I use the latter where inlining and scalar replacement can remove an intermediate aggregate. Later, I show why eliminating an aggregate mattered more than moving it to the stack.

Making the parser fast

The first faithful scanner implementation ran at 400 MB/s on twitter.json; adding parsing on top would only make it slower. The current tape parser reaches 1.4 GB/s on the same input. Benchmarks, profiles, and generated code showed that the scanner loop allocated, small helpers remained calls, and some intrinsics became software helpers.

The work fell into three themes:

1. CPU-level techniques transferred directly from C++;

2. allocation and representation had to fit a generational collector; and

3. some combinations of new features needed compiler annotations or library workarounds.

The final scanner produces packed structural-index masks at 6 GB/s. A baseline that only traverses the input with the same SIMD loads and computes a checksum reaches 22 GB/s (the figure below); this is the approximate throughput ceiling for the scanner on this machine. Classification, UTF-8 validation, quote tracking, and mask construction consume most of that headroom.

CPU-level techniques transferred from C++

The usual CPU-level techniques worked: unroll common cases, outline rare ones, fuse passes, and interleave independent work. I kept changes only when benchmarks and profiles showed an improvement. The following examples show how these techniques appeared in the scanner and parser.

Interleaving scalar and vector work

Writing a structural mask also runs popcnt and updates the index count. The original loop did this after classifying block N. The revised loop loads N, writes and counts the mask for N-1, then classifies N (the figure below). This exposes scalar work while vector loads are in flight.

Backend stalls fell from 25% to 22%, even though the new loop completed (“retired”) slightly more instructions. Exposing independent work mattered more than minimizing the count.

Fusing string scanning and copying

While parsing a JSON string, the parser must find its closing quote, decode any escapes, and store the decoded bytes. For the tape, those bytes go into parser-owned storage. C++ simdjson’s copy_and_find loop⁷ copies each SIMD block while looking for the closing quote or a backslash. My first implementation had preserved the SIMD search but lost the fusion: it found the end of an ordinary span, then passed that span to Stdlib.Buffer.add_substring to read and copy it a second time.

On twitter, decoded strings average 15–20 bytes, and add_substring’s checks and separate runtime blit accounted for 8% of sampled cycles. A reusable Bytes.t exposed the write position and let each 16-byte iteration:

1. load the block from the JSON input;

2. store all 16 bytes at the current output position;

3. compute the special-byte mask and find its first set bit; and

4. advance the output position by only the bytes before that bit.

The store is speculative. Bytes after a quote or backslash are written but not committed and are overwritten later. A quote ends the string; a backslash enters the scalar escape decoder (the figure below).

The chart below shows gains of 17–28% on string-heavy fixtures. The owned path still uses String.sub for ordinary strings because an owned OCaml string requires a final allocation and copy.

The decimal-integer loop

The standard OCaml number-conversion functions were far too slow for this parser, so it uses a dedicated integer parser and the Clinger and Lemire fast paths for floating-point conversion (Lemire 2021).⁸ Decimal integers start with an eight-digit SWAR (“SIMD within a register”) fast path. A scalar loop handles the remaining digits. The comparison below shows 19 instructions per digit in OxCaml against eight in C++. Both compute v = 10v + d. OxCaml also handles tagged positions and digits, an explicit bound, and a runtime poll—the safe-point check that lets the runtime interrupt long-running native code.

.loop:
 imul   $0xa,%rdx,%rdx   # value *= 10
 movzbl %cl,%ecx         # digit
 add    $0x1,%rax        # next byte
 add    %rcx,%rdx        # value += digit
 movzbl (%rax),%r12d     # load byte
 lea    -0x30(%r12),%ecx # byte - '0'
 cmp    $0x9,%cl         # digit?
 jbe    .loop

.loop:
 mov    %rsi,%rax
 sar    $1,%rax              # untag pos
 movzbq 0(%rbp,%rax),%rax    # load byte
 lea    -0x5f(%rax,%rax),%r8 # tagged digit
 cmp    $1,%r8
 jl     .done
 cmp    $0x13,%r8
 jg     .done                # digit range
 add    $2,%rsi              # tagged pos++
 cmp    %r13,%rsi
 setl   %al
 movzbq %al,%rax             # bound value
 sar    $1,%r8               # untag digit
 imul   $0xa,%rdi,%rdi
 add    %r8,%rdi             # value = 10v+d
 cmp    (%r14),%r15          # runtime poll
 jbe    .poll
 test   %rax,%rax            # test bound
 jne    .loop

The OxCaml loop clearly does more work, but it does not necessarily take 19/8 as long. Modern out-of-order CPUs can execute independent instructions in parallel. For example, some representation and bounds-checking work can overlap the integer multiplication. Instruction count exposes the extra work, while benchmarks determine its actual cost. Unrolling the first scalar digit shows the difference: it increased the number of retired instructions but removed a common back edge and runtime poll, producing the largest gain in the corresponding table.

Outlining rare integer and float paths also reduced register pressure and code size. Routing every float through substring allocation and float_of_string_opt, by contrast, reduced numbers.json to 170 MB/s. Native decimal conversion is essential.

Memory behaviour did not transfer

OCaml allocates into a minor heap and promotes survivors. Old-to-young pointers require a write barrier, and references left in reusable buffers keep objects reachable for longer. The structural index, the parser’s internal buffers for accumulating array and object elements, and the result tree therefore need different allocation strategies.

Removing allocation from the scanner loop

The first scanner expanded each mask into positions and appended them to Dynarray. Each live slot is an Elem block. Clearing replaces it with Empty, so refilling allocates a block per index even when capacity is kept.

Packed masks remove per-index appends: input length determines an exact int64# array, with one word per block (the figure below).

The fold still allocated two boxed records per 64 bytes:

Simd.Int8x64.String.fold_blocks input
  ~init:{ state = initial_state; block_idx = 0 }
  ~f:(fun { state; block_idx } block ->
    let result = scan_block state block in
    Structure.set_mask structure block_idx
      result.structural_mask;
    { state = result.state; block_idx = block_idx + 1 })

Unboxed products kept both records’ fields in loop state. Together with packed output, this moved the scanner to 2 GB/s.

Representing the structural index

The scanner naturally produces masks. The parser consumes positions. I compared packed masks with C++ simdjson’s flat 32-bit indices (the figure below). Dynarray remains in the benchmark only as the clearly slower baseline. Packed won overall, although flat was faster when its worst-case allocation was already warm.

After warmup, packed and flat allocate no reported words. Flat is about 10% faster when consuming every position, but reserves four bytes per input byte against one bit for packed. On canada.json, that is 9 MB versus 280 KB.

End to end, flat gains 4% with a reusable parser but loses 25% one-shot because each call initializes its worst-case reservation (the figure below). Packed uses 32 times less capacity and works well in both modes.

C++ simdjson benchmarks a warmed parser, making flat preallocation look almost free. The one-shot result shows why usage mode matters.

The cost of owning Json.t

After warmup, the 1,250 MB/s tape reports no minor or major allocation: its words and string buffer contain no OCaml pointers. Materializing Json.t drops throughput to 730 MB/s. On canada.json, an earlier profile attributed 40% of cycles to allocation and GC runtime.

Numeric leaves

Numbers expose the largest obvious opportunity. On numbers.json, the owned parser reaches 560 MB/s against 890 MB/s for the tape. float# avoids a separate float box, but Json.Float and Json.Int still allocate one variant block per leaf. Those blocks remain reachable through the result tree and may be promoted by the GC.

User-defined unboxed variants could flatten numeric alternatives while preserving the pattern-matchable API. OxCaml currently provides only built-ins such as or_null.⁹ This remains an unmeasured but likely large opportunity.

Accumulating arrays and objects

Containers add a different source of allocation and GC work. Container length is unknown until the closing bracket. The parser therefore marks a shared scratch buffer, appends elements, copies the completed slice into the newly allocated array or object in the resulting Json.t, and restores the mark (the figure below).

A reusable parser amortizes growth, but its scratch arrays eventually become old. Appending a young Json.t then triggers the write barrier and keeps the value reachable until its slot is cleared. Each element is also written to scratch and to the final result. This works well for long containers but poorly for small ones.

The parser therefore keeps up to four elements in local variables and constructs the result array or object directly. Larger containers spill to scratch. Despite the extra branching in the code, this was still a win because it avoided scratch writes, write barriers, and stale buffer references that could keep completed values alive. This improved canada and citm_catalog by 14%. For spills, Array.sub allocates and copies the result in one operation. A high-water mark also lets the parser clear stale references once per parse rather than once per container, improving representative reusable DOM rows by 8%.

We rejected several alternatives after benchmarking them. Pre-sizing 4,096 pointer slots made twitter.json one third slower by placing the buffer directly in the major heap. Fresh or growing per-container buffers created more minor-heap traffic. A 32M-word minor heap changed optimized canada.json throughput by only 1%. Pointer placement and reachability mattered more than collector tuning.

Stack allocation removed heap traffic, not the work

The number parser returns short-lived records for digit runs and number parts. Adding stack_ removed their accidental heap allocation, but not record construction or calls.

With inline annotations, scalar replacement removes the records. Without them, the records allocate and throughput falls. Restoring only stack_ gives zero heap words but remains slower because the aggregate work survives (the corresponding table).

The fast case is not “the same allocation, on the stack.” It is no allocation and no aggregate work at all. Heap allocation puts a ceiling on throughput. Inlining and scalar replacement are what remove the execution cost.

OxCaml does not provide a simple escape hatch from the GC. Stack allocation and unboxed types provide more control over allocation and representation, but the collector still shapes the design. Fast code must account for generations, write barriers, promotion, reachability, inlining, and scalar replacement. OxCaml makes that code easier to express, but using its low-level features still requires understanding and sometimes working around the runtime.

Some feature combinations were still rough

The core features worked, but combinations of new layouts, SIMD values, and templates exposed optimizer and library gaps.

Inlining needed extensive manual guidance

The scanner ultimately needed twenty-six forced-inline annotations. Together they roughly doubled its throughput. One surprising miss was a tiny local helper inside Chars.process. The listing highlights make_group_mask. Both call sites are immediately below it.

let process block =
  let low_class = lookup ~table:low_lut block in
  let high_class =
    block |> high_nibbles |> lookup ~table:high_lut
  in
  let klass = low_class land high_class in
  let make_group_mask group =
    equal (klass land group) (zero ())
    |> Int64_u.lnot
  in
  let operator_mask =
    make_group_mask vector_group_operators
  in
  ...

It survived as a call in the hot loop, as did similar boundaries across the scanner. -O3 did not substitute for the annotations: removing them lost 10–30% depending on path and fixture (the corresponding table).

A parser example showed the same problem as an allocation. The original literal matcher contained a recursive local helper:

let expect_literal state ~start literal value =
  let literal_length = String.length literal in
  let rec loop offset =
    if offset = literal_length then
      require_boundary state (start + offset)
    else if state.input.[start + offset] = literal.[offset]
    then loop (offset + 1)
    else invalid_syntax ()
  in loop 0; value

Flambda2 did not lift loop, so each literal allocated a closure. Moving it to the top level removed the allocation.

Intrinsics did not always lower to the instructions they name

In the installed package snapshot,¹⁰ the popcount and count-trailing-zeros externals lacked [@@builtin]. The backend supported both, but popcount became an OCaml runtime call and then libgcc’s __popcountdi2.

A local [@@builtin] declaration produced one popcnt and improved the scanner by 10%; count-trailing-zeros needed the same fix. Later package updates restored the metadata. Only machine-code inspection exposed the issue.

Non-value layouts need library support

The first object representation used assoc# array. Here assoc# is an unboxed pair-like record that stores the key and JSON value directly with layout value & value, removing one record per object field. While parsing an object of unknown size, however, the association scratch buffer must append fields, grow when full, copy the completed slice into the owned object, clear references to parsed values, and reuse its capacity for the next parse.

Neither Stdlib nor Base exposed bulk copy and fill for this layout. A custom binding to internal %arrayblit expanded to an OCaml loop with two caml_modify calls per product store. Clearing required another such loop because no matching fill was available.

Boxed associations restore ordinary value arrays: slicing becomes one caml_array_blit, clearing one caml_array_fill, and the runtime batches barrier work.

Depending on the fixture, unboxed associations ranged from a 10% loss to a 10% win. A better element layout helps only when libraries support its full lifecycle.

Templates can define operations for several layouts, but case lists must repeat product kinds: the grammar contains kind-abbreviation syntax that is not exposed to users. Support for non-value layouts remains spotty, and complete template families are cumbersome to write and maintain.

Practical guidance is sparse

The new features are documented individually, but there is little practical material on using them together with Flambda2 for performance work. Much of this investigation started with ocamlopt -help, compiler source, profiles, and disassembly. End-to-end examples of performance engineering with these features would make the toolchain easier to learn.

Where the remaining gap comes from

The tape runs 2.7 times slower than C++ simdjson. Some of that gap is elbow grease: simdjson has years of target-specific tuning, while the number-parser checkpoint table shows that my newer implementation was still finding gains. Tuning is not the whole explanation, however. The OxCaml parser also executes more work.

On a complete canada.json parse, the OxCaml tape path executes about 33 instructions per input byte against about 16 for C++ simdjson (the figure below). The instruction-count ratio does not identify where the cycles are spent, but it shows that the OxCaml parser performs about twice as much work per byte. The decimal loop above shows one source: representation work and a runtime poll around the same digit recurrence.

Instruction count does not predict time, but the ratio confirms extra work across scalar parsing, representation, and runtime bookkeeping. Tuning can shrink some of it. The current OCaml representation and runtime require some of it.

Conclusions

SIMD, unboxed types, and integer intrinsics let me implement simdjson directly in OxCaml. The owned parser is five times faster than existing OCaml parsers, which suggests that regular OCaml can leave meaningful performance untapped when code cannot express the representations or operations it needs. Numeric variant blocks remain costly. I believe user-defined unboxed variants can bring the parser closer to ten times the OCaml baseline, but that estimate is unmeasured.

The algorithm mapped naturally, but the first version was still 3.5 times slower than the current one. CPU-level techniques transferred from C++. Finding where to apply them required benchmarks, profiles, and generated code.

Memory-related choices were runtime-specific. Stack allocation removed GC traffic, but inlining and scalar replacement made code fast by removing aggregates altogether. Reusable storage suited the tape. The owned tree needed small-container specialization and buffer strategies designed around the GC. Writing fast OxCaml therefore requires understanding both how the generated machine code runs on the CPU and how the OCaml runtime manages allocation and pointers. More predictable inlining and broader library support for new layouts would make that work easier.

Benchmarking and testing setup

Machine and harness

Unless noted, figures use the 15 August snapshot on Fedora 43 and OxCaml 5.2.0+ox. Benchmarks were pinned to one Golden Cove performance core, with the performance governor and ASLR disabled. Boost remained enabled, so small differences are noise. OCaml rows use Bechamel and C++ rows use Google Benchmark. Hardware counters use the same core. Throughput is rounded to 10 MB/s, except the structural-index table in GB/s.

Usage modes

Reusable creates one parser and keeps its grown buffers. C++ simdjson benchmarks this mode. One-shot creates parser state per document, including buffer allocation and initialization. Both return the same Json.t.

What the rows measure

Yojson, Jsont, simdjson-ox DOM, and RapidJSON return owned trees. The OxCaml, simdjson, and sajson tape rows return handles into parser-owned storage. Every row completes parsing, but ownership differs. Driver checksums catch missing work. They are not cross-language equivalence tests for integer or UTF-8 policy.

The seven-fixture geometric mean is a useful summary but hides real variation, as the per-fixture table shows. The borrowed tape is ahead of RapidJSON on object- and string-heavy fixtures and behind it on numbers, canada, and marine_ik. Those are also the cases where C++ simdjson loses much of its lead, because number parsing is scalar work.

Correctness

The parser passes all 105 vendored JSONChecker cases. Tests compare tape with tree, reusable with one-shot, and SIMD with an independent scalar scanner. QuickCheck covers complete values, strings, numeric edge cases, and invalid syntax. Errors lack stable positions and categories.

Use of AI tools

AI coding agents were part of the implementation workflow, primarily Codex with the ocaml-codex plugin. They worked best on tasks with an automatic check: tests, benchmark and corpus wiring, and well-specified experiments. They were also useful as fuzzy search across compiler and package sources. Tasks that depended on judgement, such as API design, representation choices, and deciding what to measure, were a poor fit. Even checkable work required steering. Tests, benchmarks, profiles, and disassembly decided which changes stayed.

Scope and limitations

The public library needs API polish, better errors, broader platform support, and production use. The scanner uses four 128-bit x86 SSE vectors per block. Wider AVX types need runtime dispatch, while ARM support waits on upstream SIMD interfaces.

References and notes