SWIM gossip membership in plain language

Heartbeats through a central server don't scale: the coordinator becomes a SPoF and a fan-in bottleneck. SWIM (Scalable Weakly-consistent Infection-style Process group Membership) spreads failure detection with gossip — the same family NotiQ uses for worker membership.

Ping → ack → suspect

Each node periodically picks a random peer and sends a ping. No ack within timeout? Indirect probe: ask other nodes to ping the target. Still silent? Mark suspect, then propagate via infection-style gossip.

The false-positive knob

Aggressive timeouts catch failures fast but flag healthy nodes on slow networks. Conservative timeouts delay failover. There is no free lunch — only a tunable trade-off.

The mental model

SWIM gives you eventually consistent membership — not a linearizable registry. Your routing layer (consistent hash ring, load balancer) must tolerate brief disagreement.

Ops checklist

timeline

Set probe intervals from **p99 network RTT**, not mean.

Cap gossip fan-out — infection-style, not broadcast storms.

On suspect → confirm before draining — flapping nodes hurt more than slow detection.

Log *why* a node was marked (direct vs indirect timeout) — postmortems need it.

For where membership meets data placement, read consistent hashing. For consensus after you know who's alive, Raft election.