rask
Finding a race with sim mode
This program has a bug. Two tasks take money out of the same account at the same time, and sometimes the account ends up negative. A normal test run catches it about once in eight runs. rask test --sim catches it within a few seeds and gives you one that reproduces it every time.
The program
Alice has 100. One task sends 80 to Bob and another sends 80 to Carol. One of them should fail.
transfer reads the balance in one with block and subtracts in another. Each block holds the lock, so the two tasks never touch the balance at the same moment. But between the two blocks the other task can run, read the same 100, and subtract too.
loading…A normal test run
It passes most of the time. I ran it 400 times on a four-core machine and it failed 47 times. When it fails, all you get is the assertion, and running it again usually passes.
$ rask test bank.rk ✓ an account never goes below zero output: │ sees 100, sending 80 │ sees 20, sending 80 1 tests, 1 passed, 0 failed
A sim test run
--sim runs all the tasks on one thread, and a seed decides which task runs at each switch point. It's the same scheduler as a normal run. Only the choices come from the seed.
The same seed always gives the same order, so a failure always comes with a command that reproduces it. Seeds 1, 2 and 3 fail here, and 43 of the first 100 seeds do.
loading…
What happened
Add --trace bank.json to the replay command and sim records what each task did at each step: the locks it took, what it printed, and when it started and finished. Here's that file, one column per task.
read bank a read that was out of date by the time its task wrote, and that write
The fix
Do the check and the subtraction in the same with block. Then no other task can run in between, and 2000 seeds pass.
loading…$ rask test --sim --seeds 2000 bank.rk ✓ an account never goes below zero 1 tests, 1 passed, 0 failed (2000 runs)
About sim mode
FoundationDB and TigerBeetle test their concurrent code this way, with simulators they built themselves. In Rask it's a flag on rask test.
Sim also runs a virtual clock, so a 30-day sleep returns right away. It keeps file writes in memory, and it can inject I/O errors, dropped connections and clock jumps when a test asks for them. Details are in the sim spec.
The programs and traces on this page are checked in next to it. The compiler's test suite regenerates the traces and fails if they change.