Reading the benchmark

Read the 1% low, not the average: the 1% low is the worst 1% of frames, where stutter shows, while two machines with the same average FPS can feel completely different. Compare runs of the same game, map, settings and length, change one thing between them, and expect the numbers to move a few percent on their own.

What it measures

Frame times from the game you are actually playing, read from the graphics stack itself rather than from an overlay hooked into the game. Start the capture, play normally, stop it.

Playing your own game matters. A synthetic scene tells you how the machine handles that scene; it cannot tell you whether your game stutters in the one place it always stutters.

What do average FPS, 1% low and frame time mean?

Read the lows, not the average. Going from 200 to 210 average changes nothing you can feel. Going from a 90 to a 140 in the 1% low changes everything.

How do you compare two benchmark runs honestly?

• Same game, same map, same settings, same length of run.
• Change one thing between runs. Change three and the comparison tells you nothing about which one mattered.
• Play the same way. A run spent looking at a wall and a run spent in a firefight are not comparable.

What is left out

Frames the game draws while it is not in front are not counted: the seconds between starting the capture and switching to the game, the seconds after you switch back to stop it, any alt-tab in between, and the first second after the game comes back to the front. A game in the background often draws at a fraction of its normal rate, and those frames alone can set the 1% low. The result says how many seconds were left out, and a run saved by an older w0a, which still counted them, is not judged against a newer one.

Starting and stopping from inside the game

The hotkey is off until you turn it on: while w0a runs, a registered combination is taken from every other program, so w0a does not take one on its own. Click the Hotkey button next to the game picker and press a combination, for example Alt+F11. Then press it in the game to start the capture and again to stop it, so you do not leave the game for it. w0a stays in the background: it does not come to the front or take the focus. The game does not receive the combination itself; CS2 binds neither Alt nor F11 by default, and every other key reaches the game as before. If a capture cannot start, the message waits until you switch back to w0a.

To change the keys later, click the Hotkey button again and press the new combination, holding Ctrl, Alt, Shift or Win with the key. Del turns the hotkey off, Esc cancels, and the choice is kept. If another program already holds the combination, a line under the button says so, and the hotkey does nothing until you pick other keys. w0a registers the combination with Windows and nothing else: it does not hook or read the keyboard.

Why do benchmark results change between runs?

They always do. Thermals, background work, where you walked, what the other players did. A few percent between two runs is normal and is not evidence of anything. If a change is real, it shows up repeatedly and it shows up in the lows.

After a driver or game update

A pair taken across a game update is not judged: the page names the two versions and leaves the difference uncoloured, as it already does across a driver change. A patch changes what the game itself does, so no difference across one can be put down to a setting. w0a reads the version from Counter-Strike 2's own files; for other games only the driver is compared.

The first match after a driver or game update can compile shaders the cache no longer holds. When the first run after the newest update came out worse than the next run on the same versions, by more than the two runs' own noise, a note under the results says so. Whether shader compilation caused it is not measured; the note only says the two fit. When the two runs do not differ, there is no note. So after an update, play one match before you measure, and compare settings from the second run on.

Settling it with an A/B series

One run on each side cannot tell a small change from that wander. An A/B series repeats the comparison until it can. You decide what A and B are (for example A: engines off, B: engines on); w0a changes no setting itself.

• Pilot first. Press Start pilot and measure the same scene ten times without changing anything. The pilot measures how far your 1% low moves from run to run on its own, and from that the smallest difference a series can see. It counts for the same game on the same hardware.
• Then the series. Press Start A/B series. The first capture is a warm-up on setting A: it is saved and shown, but not counted, because the first run after the game starts plays on cold caches and would always fall on A, making B look better. After it, before every capture the page names the side to measure, in the order A B B A, repeated. Switch to that setting, play the same scene, stop the capture. Each B is compared with the A measured beside it, so the slow warm-up of a machine over an evening cancels out.
• Every run is the same scene and at least 100 seconds long. In the pilot, delete the first run after starting the game: the caches are still warming up.
• The verdict comes from the 1% low alone, once there are at least six usable pairs (eight are better); the other rows are for information. Three answers must agree: a signed-rank test, a 95% interval of the median difference that leaves zero out, and a difference at least as large as the smallest one the series can see. Without a pilot, or ten A runs in the series to stand in for one, a difference is shown but not called.
• "Within the noise" is a result too: the change did not move the lows by more than they move on their own.

A pair the comparison card would not judge on its own (a short run, lost events, an engine changing state, other hardware) is not counted, and the series says how many were left out. Press End series when you are done; its result stays on the page.

How long the game waited for a processor

Under the list of long frame intervals, a card reads the kernel's own scheduler records for a short window once the game has been presenting frames for a while: how long the game's threads waited between being ready to run and getting a processor, and which other programs held the processor meanwhile. It needs administrator rights, and a window in which the kernel dropped events shows no figure rather than a wrong one. With two runs selected, it says whether the wait moved by more than the windows' own spread.

The same card names the processor the game's busiest thread ran on most (taken as its main thread) and the deferred procedure call (DPC) time measured where it ran, next to the average processor. A series adds one line: across the runs of one setting (the pilot, or the A side), whether that DPC time and the stutters rose and fell together. It describes co-occurrence only; which one causes the other is not measured.

The network during the capture

While a capture runs and the game is running, w0a sends a small probe to 1.1.1.1 ten times a second, the address the Live connection card on the Internet page uses. It is not your game's server: it measures your own link (the router, the Wi-Fi, your provider's network), the part every game's traffic crosses first. A probe not answered within a second counts as lost. If the network never answers ping, w0a switches to TCP handshakes on port 443 of the same address, five a second, and the pings before the switch leave the record: they were filtered, not lost. While no game is running nothing is sent, and outside a capture the recorder does not run at all.

With two runs selected, the comparison card gets a Network row: each run's p95 round trip, loss and spikes side by side, uncoloured, because the card compares frames, not networks. The copied result carries the same three figures. When a run was measured with handshakes the row says so, since a handshake reads a little higher than a ping. Its tooltip adds each run's p50, p99, highest round trip and spike line.

A spike is a stretch of probes above the spike line, counted once however long it lasts. The line is the upper quartile plus three interquartile ranges and never less than 10 ms above the median, so the ordinary wobble of a quiet link does not count. It needs at least 20 answered probes, as p95 does; p99 needs 100. Below that the figure is a dash, not a guess. A record stops after an hour of probing, or earlier if the path asks for fewer packets (ICMP source quench) or probing fails, and the tooltip says which. A run saved by an older w0a has no network record.

In the list of long frame intervals, each interval says how many probes were in the air in the same window, how many went unanswered and the longest round trip. A sentence above the list counts the intervals that share a window with an unanswered probe or one over the spike line. That is all it says: the two happened at the same time. Whether either one held up the other is not measured.

Saved runs feed the stutter scan

Runs you save are used as evidence by the stutter scan: frames measured clearly slower than the frames around them appear there as a finding of their own, next to the possible causes. A saved run from the game that stutters is the most direct evidence you can hand it.

Related

• What the stutter scan looks for