spin-machine releases, measured

The bench fleet measures each release once it is published (report.yml). It boots the release interleaved with the release before it, each by the spin-machine of its own commit, in the shape spin boots a workspace in. A release from before report.yml was measured by spin-machine-dev releases measure. Each record says the shape it booted, and a number is read only beside one in the same shape. The goal is 100 ms (AGENTS.md). A Δ is read only between two releases booted in the same run, because the bench moves between runs by as much as a release does: v20261120.01's restore to first login was 158.9 ms p50 in one run and 167.0 in another (labs 44641240861, 44727451080). So a release's Δ is against the release measured with it, with its 95% interval. A Δ whose interval includes 0 is within noise, whatever its size. spin-machine-dev releases compare A B puts two side by side.

First SSH login, p50 and p95, release by release 0 100 200 300 400 500 600 700 ms v20261119.02 v20261120.01 v20261120.02 v20261120.03 v20261120.04 v20261121.01 shape changesfrom here: 1 of 16 vCPUs, 128 of 8192 MiB, 16 hotplug disks, sized to 2048 MiB, spin's login goal 100 ms v20261119.02 cold: p50 518, p95 536 ms 518 v20261120.01 cold: p50 508, p95 516 ms 508 v20261120.02 cold: p50 502, p95 513 ms 502 v20261120.03 cold: p50 474, p95 485 ms 474 v20261120.04 cold: p50 456, p95 477 ms 456 v20261121.01 cold: p50 464, p95 486 ms 464 v20261119.02 restore: p50 181, p95 183 ms 181 v20261120.01 restore: p50 185, p95 305 ms 185 v20261120.02 restore: p50 182, p95 632 ms 182 v20261120.03 restore: p50 125, p95 132 ms 125 v20261120.04 restore: p50 108, p95 111 ms 108 v20261121.01 restore: p50 110, p95 116 ms 110 cold boot to first login restore to first login goal First SSH login, ms: the line is the p50, the band up to the p95.

The goal, release by release

First SSH login, ms p50/p95. Δ is the p50 against the release before it, booted interleaved with it in the same run, and its [95% interval]: within noise when the interval includes 0.

releasecold boot to first loginΔrestore to first loginΔcold, QEMU started aheadrestore, QEMU started ahead
v20261119.02518 / 536181 / 183476 / 485154 / 162
v20261120.01508 / 516-3.2 [-10.1, +4.6] within noise185 / 305+2.1 [-2.9, +9.6] within noise482 / 491156 / 172
v20261120.02502 / 513+1.2 [-4.0, +15.7] within noise182 / 632+1.1 [-2.0, +6.1] within noise472 / 484155 / 162
v20261120.03474 / 485-35.7 [-43.1, -27.6]125 / 132-46.7 [-48.2, -43.5]446 / 459102 / 106
v20261120.04456 / 477-14.3 [-20.4, -8.2]108 / 111-13.2 [-15.1, -11.4]428 / 44083.2 / 87.1
v20261121.01464 / 486+1.1 [-1.7, +9.9] within noise110 / 116-2.2 [-4.7, +1.6] within noise442 / 46183.7 / 90.8

v20261121.01 against the goal: cold +364 ms, restore +10 ms.

Diagnostics

Not targets: what moves behind the goal's numbers (AGENTS.md, "The goal"). p50.

releasesave msstate MiBkernel to "Freeing" msQEMU RSS MiB
v20261119.0218018390.2240
v20261120.0118118388.8240
v20261120.0217618289.3238
v20261120.0312217870.8234
v20261120.0412217879.9232
v20261121.0112517982.6232

Where each was measured

releasemeasuredhostshapeboots of each probeagainstrun
v20261119.022026-11-20c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 4 hotplug disks, sized to 2048 MiB, spin's login20v20261119.0144820465606
v20261120.012026-11-20c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 4 hotplug disks, sized to 2048 MiB, spin's login20v20261119.0244820465606
v20261120.022026-11-20c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 4 hotplug disks, sized to 2048 MiB, spin's login20v20261120.0144820465606
v20261120.032026-11-20c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 4 hotplug disks, sized to 2048 MiB, spin's login20v20261120.0244809542263
v20261120.042026-11-20c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 4 hotplug disks, sized to 2048 MiB, spin's login20v20261120.0344811347183
v20261121.012026-11-21c8id.4xlarge, nested1 of 16 vCPUs, 128 of 8192 MiB, 16 hotplug disks, sized to 2048 MiB, spin's login20v20261120.0444911249078