Event Bus (Kafka)

Blindstrader uses Apache Kafka as its event bus for asynchronous, decoupled communication between microservices. No service calls another service's HTTP endpoint for core business logic — they emit events and subscribe to events.

Why Kafka?

Benefit	Detail
Decoupling	Services evolve independently; a downstream service being offline doesn't block the upstream
Durability	Events are persisted on disk; consumers can replay from any offset
Ordering	Events within a partition are strictly ordered
Fan-out	Multiple services can consume the same event without the producer knowing
Audit trail	Kafka topics serve as an immutable log of everything that happened

Event Naming Convention

Events follow the pattern {ServiceDomain}{Action}:

BrandCatalogUpdated
BrandPermissionsUpdated
SupplierProductPublished
SupplierCatalogVersionAdopted
RetailerProductAdopted
OrderPlaced
OrderConfirmed
OrderDispatched
OrderCancelled
PaymentCaptured
PaymentRefunded
OrganisationCreated
OrganisationTerminated

Topic Structure

Topics are named blindstrader.{domain}.{action} using dot notation:

Topic	Producer	Consumers
blindstrader.brand.catalog-updated	Brand Service	Supplier Service
blindstrader.brand.permissions-updated	Brand Service	Supplier Service (cache invalidation)
blindstrader.supplier.product-published	Supplier Service	Retailer Service
blindstrader.order.placed	Retailer Service	Supply Chain, Payment, Notification
blindstrader.order.confirmed	Supply Chain Service	Notification, Retailer
blindstrader.payment.captured	Payment Service	Supply Chain, Platform Ops
blindstrader.identity.organisation-created	Identity Service	Brand, Supplier, Retailer, Platform

Event Schema

All events share a common envelope:

{
  "event": "BrandCatalogUpdated",
  "version": "1.0",
  "occurred_at": "2025-06-01T10:30:00Z",
  "service": "brand",
  "tenant_id": "louvolite",
  "payload": {
    ...event-specific data...
  }
}

Consumer Groups

Each service that consumes a topic uses its own consumer group. This ensures:

• Multiple replicas of the same service share the load (one message processed once per group).
• Different services each get their own copy of every message (fan-out).

Consumer group naming: blindstrader-{service}-{topic-slug} e.g. blindstrader-supplier-brand-catalog-updated.

Idempotent Consumers

All event consumers are written to be idempotent — processing the same event twice has the same effect as processing it once. This is achieved by:

• Storing the event_id in a processed_events table.
• Checking for the event_id before processing.
• Skipping duplicates silently.

This allows safe Kafka offset resets and event replays during incident recovery.

Dead Letter Queue

Events that fail processing after 3 retries are moved to a Dead Letter Topic (DLT):

blindstrader.{original-topic}.dlq

Platform Ops can inspect and replay DLT events from the management panel under Events → Dead Letter Queue.

Monitoring Kafka Lag

Consumer lag (how far behind a consumer group is from the latest message) is monitored via Kafbat UI at kafbat.blindstrader.com (internal only) and Prometheus metrics exposed to Grafana.

Alert thresholds:

• Warning: consumer lag > 1,000 messages
• Critical: consumer lag > 10,000 messages