How jobs are scheduled and checked

Picking a host

A request becomes a job. Ballast looks for a host that is eligible, which means all of these are true:

• Its last heartbeat is under 30 seconds old.
• It is not paused and not quarantined.
• Its latest report lists the model as loaded, and it has a free slot.
• Its GPU is under 88 C and has the VRAM free for the model.
• It accepts serverless jobs and is not currently rented.

Among those, the pick goes to the highest reliability score, then the quickest first token over the last hour, with a random tie-break. The pick and the claim happen in one database step, so two requests never take the same slot.

When a host fails

The host has 5 seconds to claim the job and 20 seconds to send a first token. If it does neither, or reports an error, the attempt fails, its score drops, and the job goes to another host. There are at most 3 attempts and never the same host twice. The renter sees one reply and pays once. If all three fail, the renter gets a 502 and nothing is billed.

The reliability score

Every device has a score from 0 to 100. Seventy percent is its success rate over its last 50 jobs and spot checks, 20 percent is how much of the last 24 hours it was online, and 10 percent is its latency against the model's median. A new device starts at 60 so one early failure cannot sink it.

Spot checks

A spot check, which Ballast also calls a canary, is a prompt whose right answer the server already knows, sent at temperature 0. They are sent about 2 percent of the time, once whenever a model first loads on a device, and once after a device has been gone for more than 10 minutes. A rented GPU is checked before the rental goes live and about every 30 minutes during it.

• A check passes when the expected answer is in the output and the token count the host reports is believable for that text.
• A host cannot tell a spot check from a real job, and renters are never billed for one.
• A failure lowers the score and reverses any earnings still in the 24 hour hold.
• A score under 50, or 3 failures in the last 20 checks, quarantines the device. It gets no jobs until it passes 3 checks in a row.

Real jobs also get a token sanity check before billing. Billed output tokens can never exceed the number of characters in the output.