Operations

How Magellan runs in production

A single-screen view of the operational shape of the platform — ingest, processing, push, observability. Use this to answer “what happens when a contract comes in?” without diving into a specific service repo.

System dataflow

A high-level view of how a single hotel contract flows through Magellan — from inbox or upload, through verification and matching, to a confirmed rate push and a Slack notification.

Hotel Staff
External · Email + PDF
Operator
Internal · Manual Upload
Magellan Agent
Service · O365 Poller
Contract Extractor
Service · PDF → JSON
Agent Interactions API
verify-contract · event-match · dupload · blockload · hotel-contact
HotelMap Booking
External · Rate API
Slack
External · Notifications
Magellan Dashboard
Service · Operator UI

Traffic legend

Primary pipeline
Hotel staff → O365 inbox → Magellan Agent Agent Interactions API → HotelMap Booking API. Fully autonomous. The agent uses OpenAI function calling to decide which tool to invoke at each turn.
Manual path
Operator → Contract Extractor → human review → downstream into the same API. Used when extraction needs supervision or for contracts that never hit the inbox.
Observability
Every service emits Slack notifications and metrics. The Magellan Dashboard aggregates supply-chain data and exposes dev tools for re-running any pipeline step.
Rate sourcing
Son of Duplo runs in parallel — automated browsers fetch competitor rates from booking sites and feed the HotelMap availability engine. Independent of the contract pipeline.

Email routing

The system dataflow above shows where email enters the pipeline. For the full Magellan Online Room Blocks routing flowchart — every REF email, decision gate, chase sequence, and go-live step — use the dedicated trace below.

Shared concerns

Patterns that span more than one service — async jobs, the data ownership map, observability through Slack, and the metrics collector that drives the daily report.

Async job pattern

Every long-running operation in the Agent Interactions API returns HTTP 202 immediately with a job_id. Clients poll GET /jobs/{id}. Jobs live in Valkey under job:{uuid} as JSON with a TTL of JOB_TTL_SECONDS. Hung coroutines time out; stale processing jobs are auto-failed on the next poll.

Data ownership

Store
Owner
Purpose
Tech
MongoDB
Contract Extractor
Extracted contract data, form state, PDFs in GridFS.
Mongo
PostgreSQL
Magellan Agent
Conversation history, stage transitions, actions, allowlists.
Postgres
Valkey
Agent Interactions
Async job state — pending → processing → completed/failed.
Redis
Elasticsearch
Agent Interactions
Events index for matching, hotels index for enrichment.
ES
MySQL
Magellan Dashboard
Supply-chain data, booking analytics, connections.
MySQL

Slack notifications

Slack is the universal notification channel. Every service emits events here; the daily report aggregates the lot into a single digest with tier breakdowns.

Push confirmation
Rates pushed via dupload or blockload.
Score alert
Event-match score below threshold — severity by tier.
Verification alert
Weighted verification score below acceptance.
Security rejection
Upstream extraction flagged the input as unsafe.
No-match
Zero viable event candidates returned from matching.
Failure
Unhandled pipeline error — fire-and-forget.
Daily report
Aggregate metrics snapshot with tier breakdown.

Metrics

MetricsCollector
Thread-safe in-memory counters. Records requests, success / failure, match scores, verification scores, and security rejections. snapshot_and_reset() drives the daily Slack report and resets the day's counters. A background reporter loop sleeps until the target time, posts, and resets.