← Back to Home
Architecture Deep Dive

Decentralized Federation
Architecture & Protocol

Talx's federation layer turns every independently-hosted server into a node in a cryptographically trusted global network. Servers gossip a shared directory, deliver cross-instance DMs via signed HTTP push, and elect a new master automatically if the current one goes offline, all without any central coordinator.

● Ed25519 Signatures ● Gossip Directory ● Weighted Master Election ● Cross-Instance DMs ● 360-Day Uptime Tracking ● NOC Dashboard

The Big Picture

Each Talx binary is a fully self-contained node. One operator's deployment hosts the CA and directory bootstrap node on reserved subdomains. Every other instance auto-enrolls and joins the gossip ring.

Network Topology

talx.fyi (MASTER NODE) ca.talx.fyi Certificate Authority Ed25519 root keypair · signs leaf certs peer.talx.fyi Directory Bootstrap Node gossip seed · announce endpoint talx.fyi (app) Talx server · rooms · users PALACE_CA_ENABLED · PALACE_DIRECTORY_ENABLED Master Heartbeat broadcasts every 30s to all peers alice.example.com Enrolled peer · trusted state /.well-known/talx-federation /api/v1/federation/inbox bob.palace.net Enrolled peer · cross-instance DMs alice@talx.fyi ↔ bob@palace.net Outbox worker · signed push new.server.org Pending enrollment POST /federation/enroll → CSR or DM certificates@talx.fyi Self-Hosted Instance Your server, your rules PALACE_FEDERATION_ENABLED=1 PALACE_INSTANCE_DOMAIN=... Auto-enrolls via ca.talx.fyi Announces to peer.talx.fyi Gossip loop discovers peers Connect dialog shows all network spaces Election System Fires after 24h master silence Score = Age×100 + Ver×10 + Rel×1 Highest score wins Peers verify independently CA/directory URLs updated live No single point of failure signs cert seed peers gossip enroll CSR Cert issuance Directory gossip Peer gossip Heartbeat / Election Federated DMs

🔐 CA Service ca.talx.fyi

The network root of trust. Issues 26-year Ed25519 leaf certificates to enrolling instances. Runs in the same Go binary, activated by PALACE_CA_ENABLED=1. Enrollment works over plain HTTP (trust is signature-based, not TLS-based).

📡 Directory Node peer.talx.fyi

Bootstrap gossip seed and space aggregator. Enrolled instances announce here; the directory fans out to all peers. Powers the "Network" tab in the Connect dialog so every client sees all spaces across all servers.

✉️ Federation Inbox

Every instance exposes /api/v1/federation/inbox. Activities (DMs, friend requests, presence, directory updates) arrive here as signed JSON payloads. The inbox validates the sender's cert chain before dispatching.

📤 Outbox Worker

A background goroutine drains federation_outbox with exponential backoff. Delivery is fire-and-forget with retry; after 10 failed attempts the row is marked terminal so it never re-queues.

⚡ Election Monitor

Watches master heartbeats. If the master is silent for 24 hours, triggers a weighted score election. The winner promotes itself, broadcasts the new CA/directory URLs, and all peers update their config at runtime.

🕵️ NOC Dashboard

Real-time network operations dashboard at /admin/noc. Shows per-peer health, trust state, space counts, reliability graphs, and lets admins revoke or restore trust for any peer.

Ed25519 Certificate Authority

Trust in Talx is signature-based, not TLS-based. Every server-to-server call is signed with the sender's Ed25519 instance key and verified against a certificate that chains to the CA root. This means federation works equally well over plain HTTP (port 80) before TLS is configured.

HTTP Signature Fields
// Every outbound federation request is signed over:
// (request-target) = "post /api/v1/federation/inbox"
// host = "bob.palace.net"
// date = "Wed, 16 Sep 2026 21:00:00 GMT"
// digest = "SHA-256=<base64 body hash>"

Authorization: Signature keyId="alice.talx.fyi#instance",
  algorithm="ed25519",
  headers="(request-target) host date digest",
  signature="<base64 Ed25519 sig>"

Certificate Chain

CA Root Cert Ed25519 self-signed · 26 year validity ca.talx.fyi · baked into binary as trust anchor signs Leaf Cert A alice.example.com 26 year · auto-renews at 90d Leaf Cert B bob.palace.net 26 year · domain-bound pubkey Signed S2S Request Receiver verifies: sig matches pubkey AND cert chains to CA root
ComponentKey TypeValidityStorageNotes
CA root keypairEd2551926 yearspsdata/federation/ca/Self-signed; baked into binary as default trust anchor. Operator can add extra anchors via PALACE_TRUST_ANCHORS.
Instance keypairEd25519Permanentpsdata/federation/instance.keyGenerated once on first boot. Signs all outbound S2S requests.
Instance leaf certEd2551926 yearspsdata/federation/instance.crtIssued by CA at enrollment. Auto-renewed 90 days before expiry. Binds domain → pubkey.
Peer cert cacheEd25519Per-peerfederation_peers tableCached after first verified contact. Re-verified on every inbound message.

Two Ways to Join the Network

New instances join via auto-enrollment (one HTTP call) or peer-vouching (a DM to certificates@talx.fyi). Both paths end in the same signed certificate.

Path A Auto-Enroll via CA

1

Generate Ed25519 Keypair

On first boot with PALACE_FEDERATION_ENABLED=1, the server generates a persistent Ed25519 instance keypair under psdata/federation/instance.key. This never changes.

2

POST CSR to CA

Sends a signed Certificate Signing Request to https://ca.talx.fyi/federation/enroll. The CSR includes the instance domain and public key, signed as proof-of-possession. Falls back to http:// if HTTPS fails.

3

Receive Signed Certificate

CA verifies proof-of-possession, issues a 26-year leaf certificate chaining to the CA root, and returns it. The instance persists it as instance.crt.

4

Announce to Directory

Signed announce to https://peer.talx.fyi/federation/directory/announce. Directory verifies the cert chain, upserts the peer into federation_peers, and returns a seed peer set for gossip bootstrap.

Path B Peer-Vouched via DM

1

DM the certificates@ Bot

A new instance operator sends a DM to certificates@talx.fyi, a reserved bot account (similar to the loopback bot). The DM body contains the CSR payload.

2

Peer Forwards Vouch to CA

The existing trusted peer (talx.fyi) forwards the CSR with its own signed vouch to ca.talx.fyi/federation/vouch. The CA verifies the voucher is itself a trusted peer before signing.

3

CA Issues Certificate

CA issues the same 26-year leaf cert as Path A. The vouching peer receives it and forwards via DM reply to the new instance operator.

4

Admin Notification Fan-Out

On join (and on cert renewal/failure), the certificates@ bot DMs every server admin: the union of ListInstanceHosts() and PALACE_ADMIN_USERNAME entries, deduped.

Full Join Sequence (Path A)

new.server.org ca.talx.fyi peer.talx.fyi existing peers generate Ed25519 keypair (once) POST /federation/enroll {domain, pubkey, sig} verify proof-of-possession issue leaf cert (26yr) signed leaf cert (PEM) persist instance.crt POST /federation/directory/announce {signed} verify cert → CA root upsert federation_peers seed peer set + listed in directory gossip pull /.well-known/talx-federation from all peers certificates@ DMs all server admins "joined network ✓" heartbeat broadcast starts (30s interval) time

Self-Propagating Discovery

Each enrolled instance runs a background gossip loop. It pulls /.well-known/talx-federation and /api/v1/federation/directory from the bootstrap node and from every known trusted peer. New peers discovered transitively are added and their spaces merged into federation_directory. The aggregated directory is exposed as /serverlist/network.json and rendered in the Connect dialog's Network tab.

TablePurpose
federation_peersKnown instances: domain, pubkey, cert, trust state, last_seen, first_seen, software_version
federation_directoryCached remote spaces: peer_domain, tenant_id, display_name, sysop, users, rooms, url, updated_at
federation_remote_accountsResolved user@domain handles, cached with 1h staleness window
federation_outboxQueued signed S2S deliveries with retry state and exponential backoff

Gossip Ring Convergence

peer.talx.fyi directory seed talx.fyi (master) alice.ex .com bob.net peer new.org joining you.me self-host pull dir pull dir gossip

user@domain Addressing

Accounts are globally addressable as user@instance-domain. Cross-instance DMs travel as signed JSON activities via HTTP push, using a shadow-user pattern so the entire local DM pipeline works unchanged.

Cross-Instance DM Flow
// alice@talx.fyi sends a DM to bob@palace.net

// 1. talx.fyi: shadow-provision bob@palace.net as a local users row
shadowUser = CreateFederatedUser("bob@palace.net", "palace.net", "Bob")
thread = GetOrCreateDMThread(alice.ID, shadowUser.ID)
InsertDMMessage(thread.ID, alice.ID, "Hey Bob!")

// 2. Enqueue signed activity to outbox (fires async)
EnqueueFederationDelivery("palace.net", "dm.message", {
  from: "alice@talx.fyi", to: "bob@palace.net",
  body: "Hey Bob!"
})

// 3. OutboxWorker: sign + POST to bob's inbox (exponential backoff)
POST https://palace.net/api/v1/federation/inbox
  Authorization: Signature keyId="talx.fyi#instance" ...
  { type: "dm.message", from: "alice@talx.fyi", to: "bob@palace.net" }

// 4. palace.net inbox: verify sig + cert chain, shadow alice, push event to Bob
PushEventToAccount(bob.ID, EventDMMessage{thread, "Hey Bob!"})

🪞 Shadow Users

Remote senders are provisioned as local users rows with a user@domain username. This means friendships, DM threads, mailbox UI, and even user: bans all work unchanged, with no special-casing in the local stack.

🔍 Handle Resolution

Before sending, the outbox resolves user@domain via GET /.well-known/talx-federation/resolve?acct=user@domain on the remote server. Results are cached in federation_remote_accounts with a 1-hour staleness window.

🛡️ Spoofing Prevention

The inbox validates that activity.From's domain matches the HTTP-signature-verified sender domain. A trusted peer cannot send activities claiming to originate from a third instance.

🔄 Activity Types

Inbox handles: dm.message, friend.request, presence, directory.update. Unknown types are acknowledged and silently ignored so new activity types from newer peers never break older ones.

Decentralized Weighted Election

If the master node is silent for 24 hours, every peer independently computes a weighted score for all known peers. The highest scorer promotes itself, broadcasts the election result, and peers verify the claim by recalculating independently before accepting.

PHASE 1

Heartbeat Silence

Master broadcasts heartbeat every 30s. All peers record receipt with timestamp.

PHASE 2

Timeout Detected

Missing heartbeat for 2 min = blip. Missing for 24 hours = election triggered.

PHASE 3

Score Calculation

Each peer computes weighted scores for all known peers independently using the formula below.

PHASE 4

Winner Promotes

Highest scorer announces itself as new master, broadcasts new CA/directory URLs.

PHASE 5

Consensus Verify

All peers recalculate the winner's score independently. Only if valid does the network update CA/directory URLs at runtime.

Election Score = (100 × AgeScore) + (10 × VersionScore) + (1 × ReliabilityScore)
AgeScore (0–100): peer_age / oldest_peer_age × 100. Oldest instance wins ties (weight 100)
VersionScore (0–100): semantic version rank / total_peers × 100. Newest software preferred (weight 10)
ReliabilityScore (0–100): composite of 360-day uptime %, blip count & average blip duration (weight 1)
Given 3 peers:
Peer A: age=365d version=1.2.0 uptime=99.9% blips=5
Peer B: age=180d version=1.3.0 uptime=99.5% blips=200
Peer C: age=90d version=1.2.5 uptime=99.99% blips=0

Age Scores (oldest_age=365d):
A: 100 × (365/365) = 100.0
B: 100 × (180/365) = 49.3
C: 100 × (90/365) = 24.7

Version Scores (B=newest rank 3, C=rank 2, A=rank 1):
A: 100 × (1/3) = 33.3
B: 100 × (3/3) = 100.0
C: 100 × (2/3) = 66.7

Reliability Scores (360-day):
A: 90.5 B: 85.5 C: 95.0

Total Election Scores:
A: (100×100.0) + (10×33.3) + (1×90.5) = 10,423.8 ← WINNER
B: (100×49.3) + (10×100.0) + (1×85.5) = 6,015.5
C: (100×24.7) + (10×66.7) + (1×95.0) = 3,232.0

→ Peer A wins. Age dominates (weight 100). Even the oldest version can't overcome a 365-day head start.

Reliability Scores via Gossip Consensus

Each peer independently monitors every other peer's heartbeat. Downtime events are aggregated into daily summaries, and multi-peer consensus is required before a blip is accepted, preventing a single noisy peer from tanking another's reliability score.

📊 Data Pipeline

Raw heartbeat gaps → federation_peer_uptime_events → daily aggregator → federation_peer_downtime_summary → reliability calculator → federation_peer_reliability_score (360-day rolling window, pruned daily)

🤝 Gossip Consensus

Downtime summaries are gossiped between peers. A blip is only counted if multiple peers independently agree on the downtime window. Single-observer events are discarded. The detection_count field tracks how many peers observed the same window.

DB Schema (00026_election_uptime.sql)
-- Raw downtime events
CREATE TABLE federation_peer_uptime_events (
  peer_domain TEXT NOT NULL,
  event_type TEXT NOT NULL, -- 'down' | 'up'
  timestamp TIMESTAMP NOT NULL,
  detected_by TEXT NOT NULL -- who saw it
);

-- Daily aggregates (per-peer per-day)
CREATE TABLE federation_peer_downtime_summary (
  peer_domain TEXT NOT NULL,
  date DATE NOT NULL,
  downtime_minutes INTEGER DEFAULT 0,
  blip_count INTEGER DEFAULT 0,
  detection_count INTEGER DEFAULT 1,
  UNIQUE(peer_domain, date)
);

-- Pre-computed 360-day rolling scores
CREATE TABLE federation_peer_reliability_score (
  peer_domain TEXT PRIMARY KEY,
  score_360d REAL, -- 0-100 composite
  uptime_pct REAL, -- % uptime
  total_downtime_minutes INTEGER,
  total_blips INTEGER,
  avg_blip_duration REAL,
  last_calculated TIMESTAMP
);

Network Operations Center

Real-time visual dashboard at /admin/noc. Superadmin-only. Shows the health of the entire federation at a glance.

📊 /admin/noc

HTML dashboard with live auto-refresh (30s). Overall network health score (0–100%), per-peer status, Chart.js downtime graphs, trust state badges, space counts, and last-seen timestamps.

🔌 /admin/noc/status

JSON API returning { joined, certNotAfter, lastRenewed, peerCount, health, peers[] }. Used by the dashboard and available for external monitoring integrations.

🔒 Trust Management

POST /admin/noc/peers/<domain>/trust revokes or restores trust for any peer. Revoked peers can no longer deliver to the inbox or receive from the outbox. Requires active superadmin session.

📈 Uptime Graphs

Per-peer Chart.js downtime visualization from the 360-day history. REST endpoints serve pre-computed graph JSON. Reliability scores and blip counts are visualized alongside the election weight they would generate.

Federation API Reference

Every Instance

GET
/.well-known/talx-federation
Actor document: instance domain, pubkey, cert, inbox_url, directory_url, software/version. Used by peers to bootstrap gossip and verify signatures.
GET
/.well-known/talx-federation/resolve
?acct=user@domain resolves a local handle. Returns {handle, displayName, avatarUrl, exists}. Used for pre-send validation of cross-instance DMs.
POST
/api/v1/federation/inbox
Authenticated sink for signed activities (dm.message, friend.request, presence, directory.update). Validates HTTP signature + cert chain. Rejects spoofed From domains.
GET
/api/v1/federation/directory
This instance's shareable space list, the same data as the local Connect dialog. Used by peers during gossip pull.
GET
/serverlist/network.json
Aggregated directory of all federated spaces. Powers the Network tab in every client's Connect dialog.

CA Node Only (ca.<base>)

POST
/federation/enroll
CSR enrollment. Body: {domain, publicKey} signed by the CSR key. Returns a PEM leaf cert on success. Idempotent re-enrollment supported.
POST
/federation/vouch
Peer-vouched enrollment. Voucher must be a trusted peer. Issues the same cert as /enroll but through a trusted intermediary.
GET
/federation/ca/root
Returns the CA root certificate/pubkey. New instances pin this as their trust anchor. Baked into the binary as the default.

Directory Node Only (peer.<base>)

POST
/federation/directory/announce
Signed announce from enrolled instance. Verifies cert chain, upserts peer, returns seed peer set for gossip bootstrap.
GET
/federation/directory
Aggregated network directory and peer list. Used as the primary gossip bootstrap source.

Enable Federation on Your Server

Minimal .env for a Federated Instance
# Enable federation
PALACE_FEDERATION_ENABLED=1
PALACE_INSTANCE_DOMAIN=myserver.example.com

# Where the CA and directory live (defaults to talx.fyi network)
PALACE_CA_URL=https://ca.talx.fyi
PALACE_NODES_URL=https://peer.talx.fyi

# Superadmin to notify on join/renewal events
PALACE_ADMIN_USERNAME=yourname

# Run your own CA + directory (talx.fyi only, or private networks)
# PALACE_CA_ENABLED=1
# PALACE_DIRECTORY_ENABLED=1
# PALACE_TRUST_ANCHORS=<extra CA pubkeys, comma-separated>
VariableDefaultDescription
PALACE_FEDERATION_ENABLEDfalseMaster on/off. Safe to leave unset for self-hosted instances that don't want to federate.
PALACE_INSTANCE_DOMAINBASE_DOMAINCanonical federation identity domain. Becomes the @suffix for all accounts on this server.
PALACE_CA_URLhttps://ca.talx.fyiCA enrollment endpoint. Override to run a private network with your own CA.
PALACE_NODES_URLhttps://peer.talx.fyiDirectory bootstrap peer. 443→80 fallback automatic.
PALACE_CA_ENABLEDfalseHost the CA service on ca.<base>. Generates/loads root keypair under psdata/federation/ca/.
PALACE_DIRECTORY_ENABLEDfalseHost the directory node on peer.<base>.
PALACE_ADMIN_USERNAMEjonkSuperadmin username(s). DMs sent here on join/renewal/failure. Comma-separated for multiple.
PALACE_TRUST_ANCHORS(CA root)Additional CA pubkeys to trust. Use for private networks or adding extra anchors.
PALACE_FEDERATION_ALLOW_PRIVATE_CHALLENGEfalseAllow challenge fetches to loopback/private IPs. LAN testing only.