Your Vast.ai Machine Went Offline: The 10-Minute Checklist

A machine that drops off the marketplace earns nothing and quietly damages your reliability score with every hour it stays down. The worst way to debug it is to poke at random. The fastest way is a fixed order of checks, from power up through disk, where each step either finds the cause or rules out a whole layer. Run this list top to bottom and you will usually know what happened inside ten minutes.

Why a fixed order beats intuition

Every layer of the stack depends on the ones below it. The Vast.ai daemon cannot report in without a network, the network does not matter if the machine lost power, and the GPUs are irrelevant if the disk filled up and services are crashing on writes. Checking in dependency order means you never waste time debugging a symptom three layers above the actual fault. Hosts who skip straight to reinstalling drivers regularly spend an evening fixing a problem that was a tripped breaker.

The order: power, network, daemon, GPUs, disk. Every time.

Check 1: Power and console access

Can you reach the machine at all? Try SSH first. If SSH fails, you need eyes on the box: a monitor, IPMI, a KVM, or a smart plug that reports power draw.

No power draw at all: tripped breaker, failed PSU, or a power strip someone borrowed. Physical problem, physical fix.
Powered but no console output: the machine crashed or hangs at boot. Note anything on screen before you reset it; a kernel panic message is worth a photo.
Console works, SSH does not: the machine is alive and the problem is network or sshd. Continue to check 2 from the console.

Rebooted overnight without you?

An unexpected reboot with no storm and no power cut often traces back to an automatic update replacing the kernel or restarting services. If last reboot shows a restart you did not order, read why auto updates kill GPU host uptime before this happens again.

Check 2: Network, gateway, and DNS

From the console, test connectivity in three quick layers:

ip a
ping -c 3 192.168.1.1
ping -c 3 8.8.8.8
ping -c 3 vast.ai

1ip a should show your interface up with the LAN address you expect. No address means a DHCP or netplan problem.
2Pinging the gateway (use your router's address) proves the LAN link. Failure here is a cable, switch, or interface issue.
3Pinging 8.8.8.8 proves routing to the internet. Success here but failure on ping vast.ai means DNS is broken, not the connection.

If the machine is online but the marketplace still cannot reach it, you have a reachability problem rather than an outage, and the port not listening diagnosis takes over from here.

Check 3: Is the Vast.ai daemon running

The Vast.ai daemon is what reports your machine to the marketplace. Machine healthy plus daemon dead equals offline listing. Check it:

systemctl status vastai
sudo journalctl -u vastai -n 50

A stopped daemon usually restarts cleanly with sudo systemctl restart vastai. If it crashes again within minutes, the journal output tells you why, and the answer is frequently one of the two checks below: it cannot see the GPUs, or it cannot write to disk. The Vast.ai docs cover daemon-specific errors in more depth.

Check 4: Do the drivers see every GPU

nvidia-smi

Three outcomes, three diagnoses:

All GPUs listed: drivers are fine, move on.
"NVIDIA-SMI has failed" or a driver/library version mismatch: the kernel and driver modules disagree, classically after a kernel update installed itself overnight. A reboot fixes the mismatch; preventing the surprise update fixes the pattern.
Some GPUs missing: a card fell off the bus. Check sudo dmesg | grep -i xid for hardware errors, then reseat power and riser cables before suspecting the card itself.

A machine listed with fewer GPUs than advertised is arguably worse than one that is fully offline, because renters get broken instances and rate you for it.

Check 5: Is the disk full

df -h

A disk at 100 percent breaks things in confusing, indirect ways: containers fail to start, logs stop, the daemon cannot update its state, and none of the error messages say "disk full." Old rental images are the usual culprit on a hosting machine. Reclaim space with docker system df to see what is eating it, then prune unused images during a quiet window. Aim to stay under 85 percent so the machine has headroom during large image pulls.

After the fix: close the loop

Once the machine is back, spend five more minutes making the same failure less likely and less expensive next time:

Write down the cause and the fix. Your own history is the best troubleshooting doc you will ever have.
If the cause was an automatic update, schedule updates on your own terms instead.
Confirm the machine is actually renting again, not just showing online. A listing can be visible and still unreachable.

The last point is where alerting earns its keep. Set up Discord and Telegram alerts so the next incident announces itself, and you open this checklist while the outage is still measured in minutes.