Solo Pool Failover: Our Zero-Downtime Response Pattern

It's 3am and forty S19s stop submitting shares at the same time. Nothing on the rack looks wrong — fans are spinning, lights are on — but the connection to the pool is dead. If you've run a solo fleet long enough, you've had a night like this. The difference between a five-minute blip and a six-hour revenue gap almost always comes down to whether you had a failover path ready before the alert fired.

Why we treat failover as a day-one setting

A lot of solo operators bolt failover on after their first bad outage. We'd rather build it in before the first miner ever boots. Solo mining already carries enough variance in block luck — you don't want operational downtime stacked on top of that variance, quietly eating hours you'll never see reflected in a luck report. Even a short endpoint hiccup, if it goes undetected for an hour, is an hour of hashrate that never touched a block template.

What the backup path actually needs

A real failover setup isn't just typing in a second URL and hoping. We keep two backup endpoints tested and ready, not just written down somewhere:

• A primary endpoint plus two backups that have actually been connected to and verified recently — not just copied from a wiki page a year ago.
• The same worker and credential naming convention across primary and backup pools, so nobody's guessing which worker string goes where at 3am.
• Timeout and retry settings documented per miner model, since firmware from different vendors handles a dead socket very differently. Some retry aggressively, some just sit there.
• Alerts specifically for reconnect loops and reject-rate spikes, separate from your general "worker offline" alert. A reconnect loop looks different from a dead worker and needs a different response.

What actually happens when it breaks

Here's the sequence we follow, roughly in order:

1. Catch it early using last-share recency and reject-rate alerts rather than waiting for a full offline flag — by the time a worker shows "offline," you've usually already lost several minutes.
2. Figure out fast whether it's your network, the miner's firmware, or the pool endpoint itself. A quick ping and a check of whether other workers on the same switch are affected tells you a lot in under a minute.
3. Flip to the backup endpoint and watch it closely for the first 10-15 minutes. That window is where you catch a bad failover — one that reconnects but then starts throwing stale shares because vardiff hasn't settled yet.
4. Log when it started, when it recovered, and what caused it. Writing up an incident at 3am feels like busywork, but six months of these logs is what tells you which endpoint is actually reliable and which one just looks reliable on paper.

Closing the loop

Failover you've never tested is a guess, not a plan. We run a controlled failover test roughly once a month — pick a low-stakes window, flip a few workers over on purpose, and confirm they come back to primary cleanly once the test ends. It's tedious. It's also the only way to know your backup endpoint won't choke the first time you actually need it under real load.

One honest caveat: failover fixes connectivity problems, not hardware problems. If a board is dying or a PSU is flaky, switching pools won't help — you'll just reproduce the same instability on a different endpoint. Rule that out separately before you blame the pool.

Teams that actually rehearse this tend to end up with a noticeably better effective hashrate than teams that only watch their nominal TH/s number, because effective hashrate accounts for every minute you weren't really connected. Our worker search dashboard lets you check accepted-share history and stability signals per worker, which is the fastest way to tell a real outage apart from a single flaky miner.