Documentation

Overview #

Each node runs a prototype micro-model: a bounded set of labeled exemplars with nearest-neighbor prediction. Nodes learn from their own local, non-IID data (each sees only a slice of the classes) and periodically gossip a sample of their prototypes to peers. Because merging prototype sets is order-invariant (set union + prune), there is no weight-averaging step to go wrong and no need for a coordinator to synchronize rounds.

The result: a node ends up able to classify categories it never saw locally, pooled from the swarm — approaching centralized accuracy while every node keeps its data private and the network tolerates malicious peers.

Installation #

Python 3.10+. The learning core needs only numpy; optional extras add the real P2P stack and real-data validation.

git clone https://github.com/shaswata56/swarmint
cd swarmint
pip install -e ".[dev]"        # everything, for development

# or install just what you need:
pip install -e "."             # core (numpy-only, in-process simulator)
pip install -e ".[net]"        # + real UDP / Kademlia DHT / Ed25519 / msgpack
pip install -e ".[learn]"      # + scikit-learn for real-data + embeddings

Quickstart #

Verify the project does what it claims, then see the smallest end-to-end example:

# reproduce every headline claim as a PASS/FAIL table (~40s)
python benchmark.py

# smallest end-to-end: specialists gossip into a full-task model
python examples/quickstart.py
#  solo node (no gossip)  : 0.333
#  swarm node (gossip)    : 0.855

# real data, and real UDP + DHT across OS processes
python -m swarmint.sim.run_digits
python -m swarmint.sim.run_multiproc
python run_tests.py            # the test suite (no pytest needed)

Core concepts #

Prototype model

A model is a bounded set of (vector, label, weight) prototypes. Prediction is trust/weight-aware nearest-neighbor. Learning absorbs a new observation into a nearby same-label prototype or adds a new one; pruning keeps the set bounded. Distance radii (absorb / conflict / override) are derived from data statistics, not hardcoded.

Gossip federation

Every round, a node samples a few prototypes and pushes them to peers. Receivers merge them. Over many rounds, knowledge diffuses so each node approaches full-task accuracy.

Trust & corroboration

Each peer carries a reputation score. A prototype for a class the receiver can't verify locally is quarantined until it is corroborated by a majority of distinct senders — which defeats a lone (or small colluding) attacker. Verifiable poison is caught by merge-validation against the receiver's own holdout.

Byzantine robustness

Label-flip attackers see their trust collapse and their updates rejected. The swarm stays near its clean accuracy at roughly 5% malicious nodes; the safe fraction scales with how much honest redundancy covers each class.

Architecture #

swarmint is layered so the learning core stays transport-agnostic and synchronous, and all networking/asynchrony lives at the edges:

• core/ — PrototypeModel (learn / predict / merge / prune), a scalar logical clock, a signed tamper-evident update chain, and the shared genesis embedding.
• node/ — SwarmNode (synchronous, representation-agnostic: observe / gossip / receive / answer_query) and a NetNode that composes it with the network stack.
• network/ — the bus abstraction, an in-process simulator bus, a real UDP bus, msgpack wire format, Ed25519 identity, Kademlia discovery, NAT traversal, and relay.
• inference/ — trust-weighted aggregation and async request-collect RPC.
• sim/ — runnable simulations and real-dataset harnesses.

P2P networking #

The real stack runs over UDP with a msgpack wire format and Ed25519-signed envelopes (with a replay guard and a pubkey↔node-id binding, so a peer can't be impersonated).

• Discovery — a Kademlia DHT rendezvous plus peer-exchange (PEX); nodes advertise topics so gossip and queries can be routed to the relevant experts.
• NAT traversal — rendezvous-coordinated UDP hole-punching for peers behind home/mobile NATs.
• Relay fallback — when a direct path can't be opened (e.g. symmetric NAT), peers exchange gossip through a mutually reachable relay. The relayed payload is the original signed envelope, so the destination still verifies the true sender, and routing fails over across multiple relays so no single node is a point of failure.

Distributed inference #

A node can answer a query about classes it never learned by consulting the swarm: it fans the query out to peers, each returns a label and confidence (abstaining when off-manifold), and the answers are combined by a trust-weighted plurality. Trust is calibrated on the querier's own holdout, so confidently-wrong experts get down-weighted — the ensemble beats both local-only and naive-vote inference even with 30% liars.

Configuration #

Behavior is controlled by explicit fields and opt-in flags rather than hidden globals:

• Distance radii on the prototype model (absorb, conflict, override) — derive them from your data's same-class vs cross-class distance statistics.
• Byzantine gating — corroboration radius and abstain margin on the node.
• Networking — enable_nat, enable_relay, enable_chain on NetNode; a resource governor caps the multiprocess harness (configurable via flags or SWARMINT_* env vars).
• Shared genesis embedding — an optional supervised projection, fit once at genesis on a small public seed and distributed frozen, that lifts accuracy toward centralized on many-shot data. Every node must use the identical projection.

For exact APIs and field names, read the module docstrings and the README.