Comparative benchmarks
The ublk overhead ladder: kernel floors, userspace transport, and RAM targets, with latency and CPU costs.
On this page
The overhead ladder separates the kernel floor, userspace transport, and RAM data work. This completed run used Linux 6.12.111 in a KVM guest on 4 dedicated physical cores of a Ryzen 9 6900HX. Fio ran on vCPUs 0-3, and the server on 4-7, with disjoint CPU sets. There are 3 interleaved rounds per target and workload, with 264/264 valid observations.
Measured transport cost#
- ublksrv, libublk-rs, and inline go-ublk add about 13 µs of completion latency at qd1 over the kernel null_blk floor. At depth, random reads cost about 1,100 extra CPU-ns/IO, or 40% over that floor.
- go-ublk with
DeviceParams.Inlineties the C and Rust reference servers on transport cost in this run: 13.89 µs qd1 p50 and 4,027.94 CPU-ns/IO for random reads. Its default goroutine mode adds about 35 µs over the kernel floor. - lib-ublk’s serving path is currently the slowest at depth, with about 20 µs qd1 p50 and a 300k IOPS ceiling. Work on the serving path is ongoing.
- At depth, the C, Rust, and inline Go null targets reach the kernel rate: about 751k-757k IOPS for random reads, against null_blk’s 698k. Fio limits throughput there; CPU-ns/IO separates their transport costs.
No winner claims. The report’s qualification gates still lack quiet-window attestation and complete peer controls. These are VM measurements; cross-CPU wakeups cost more here than on bare metal. Null targets discard writes. lib-ublk’s null example also zero-fills reads, and ublksrv’s RAM target includes loop/tmpfs backend I/O. Kernel workers can run outside the userspace CPU sets. Rust’s RAM geometry is unmatched.
Reading the ladder#
L0 is null_blk without stored data. L1 adds kernel RAM storage through brd or memory-backed null_blk. L2 measures the userspace null targets, and L3 adds retaining-RAM targets.
The requested device geometry was 2 queues at depth 64. All userspace targets accepted it except libublk-rs’s RAM example, which fixes 1 queue at depth 128. Its null example accepted the requested geometry. brd has kernel-managed geometry.
Every qd1 p50 column comes from the separate 4 KiB random-read run with one job at depth 1, including the write and mixed tables. Kernel devices complete inline through io_uring; their reported roughly 0.1 µs measures completion latency alone and excludes submission. The tables show inline for those rows. The userspace p50 values also measure completion latency; they aren’t full round-trip times.
IOPS and CPU-ns/IO come from each named workload at 2 jobs × queue depth 32. Values are means of the primary runs; latency values are means of run-level quantiles. CPU-ns/IO includes fio and server user/system CPU plus attributed residual kernel CPU per completed I/O. It excludes kernel workers outside the benchmark cgroups, unattributed interrupts, and other guest or host work. The report gives the accounting method and sample standard deviations.
4 KiB random reads#
| Rung | Target | qd1 p50 µs | 2×qd32 IOPS | CPU-ns/IO |
|---|---|---|---|---|
| L0 | null_blk (memory_backed=0) | inline | 697,574.16 | 2,872.83 |
| L1 | brd | inline | 748,901.96 | 2,673.59 |
| L1 | null_blk (memory-backed) | inline | 518,206.85 | 3,869.35 |
| L2 | ublksrv (null) | 13.16 | 756,632.54 | 3,981.74 |
| L2 | libublk-rs (null) | 13.89 | 751,428.28 | 4,006.86 |
| L2 | go-ublk (null, inline) | 13.89 | 750,904.23 | 4,027.94 |
| L2 | go-ublk (null, default goroutine) | 35.41 | 627,270.72 | 5,701.59 |
| L2 | lib-ublk (null, zero-filled reads) | 19.75 | 301,149.68 | 7,045.35 |
| L3 | ublksrv (loop over tmpfs) | 20.44 | 367,384.13 | 5,680.39 |
| L3 | libublk-rs (RAM, unmatched) | 14.61 | 656,377.06 | 4,309.44 |
| L3 | go-ublk (RAM, inline) | 14.53 | 601,819.01 | 4,532.49 |
| L3 | go-ublk (RAM, default goroutine) | 36.78 | 568,548.05 | 6,920.21 |
| L3 | lib-ublk (RAM) | 20.35 | 283,428.33 | 7,200.78 |
4 KiB random writes#
| Rung | Target | qd1 p50 µs | 2×qd32 IOPS | CPU-ns/IO |
|---|---|---|---|---|
| L0 | null_blk (memory_backed=0) | inline | 665,755.53 | 3,008.38 |
| L1 | brd | inline | 689,646.76 | 2,903.98 |
| L1 | null_blk (memory-backed) | inline | 506,170.70 | 3,960.83 |
| L2 | ublksrv (null) | 13.16 | 712,006.11 | 4,223.56 |
| L2 | libublk-rs (null) | 13.89 | 706,629.18 | 4,257.47 |
| L2 | go-ublk (null, inline) | 13.89 | 706,909.94 | 4,279.23 |
| L2 | go-ublk (null, default goroutine) | 35.41 | 606,231.64 | 5,959.48 |
| L2 | lib-ublk (null, zero-filled reads) | 19.75 | 275,901.24 | 7,828.04 |
| L3 | ublksrv (loop over tmpfs) | 20.44 | 367,190.94 | 5,792.74 |
| L3 | libublk-rs (RAM, unmatched) | 14.61 | 670,473.17 | 4,362.64 |
| L3 | go-ublk (RAM, inline) | 14.53 | 591,022.87 | 4,640.91 |
| L3 | go-ublk (RAM, default goroutine) | 36.78 | 558,978.76 | 7,142.78 |
| L3 | lib-ublk (RAM) | 20.35 | 251,613.09 | 8,092.99 |
4 KiB mixed random I/O#
| Rung | Target | qd1 p50 µs | 2×qd32 IOPS | CPU-ns/IO |
|---|---|---|---|---|
| L0 | null_blk (memory_backed=0) | inline | 673,483.59 | 2,976.25 |
| L1 | brd | inline | 722,591.23 | 2,770.78 |
| L1 | null_blk (memory-backed) | inline | 514,477.61 | 3,895.04 |
| L2 | ublksrv (null) | 13.16 | 731,771.42 | 4,111.79 |
| L2 | libublk-rs (null) | 13.89 | 725,685.49 | 4,146.05 |
| L2 | go-ublk (null, inline) | 13.89 | 730,038.46 | 4,145.17 |
| L2 | go-ublk (null, default goroutine) | 35.41 | 605,853.07 | 5,864.79 |
| L2 | lib-ublk (null, zero-filled reads) | 19.75 | 278,881.15 | 7,531.09 |
| L3 | ublksrv (loop over tmpfs) | 20.44 | 362,288.20 | 5,790.70 |
| L3 | libublk-rs (RAM, unmatched) | 14.61 | 629,258.92 | 4,489.11 |
| L3 | go-ublk (RAM, inline) | 14.53 | 598,136.86 | 4,609.34 |
| L3 | go-ublk (RAM, default goroutine) | 36.78 | 560,703.84 | 7,041.87 |
| L3 | lib-ublk (RAM) | 20.35 | 262,404.24 | 7,728.19 |
Earlier shared-CPU baseline#
The earlier retaining-RAM baseline used the same VM with fio and the server sharing a 4-CPU budget. Its random and sequential workload tables remain available in the report. That CPU placement differs from the disjoint ladder run, so the two sets of means aren’t interchangeable.
Go dispatch experiment#
A separate interleaved A/B run on the same VM compares go-ublk’s default goroutine-per-request dispatch with DeviceParams.Inline. At queue depth one, RAM throughput changed from 20.3k to 43.5k IOPS. Those are medians from a separate experiment; the ladder tables report their own measured means.
Evidence files#
- Overhead ladder report: all workload means, dispersion, CPU accounting, rung deltas, qualification gates, source revisions, and binary hashes.
- Overhead ladder manifest: disjoint CPU placement, workload geometry, seed, runtime, warmup, and interleaved schedule.
- Shared-CPU baseline report and manifest: the earlier retaining-RAM experiment.
- Go dispatch results and metadata: per-round latency, syscall, and context-switch measurements.
The hosted reports and metadata are sanitized extracts. Raw fio JSON and guest lifecycle logs aren’t included in these downloads.