Service Overview

The Deal Desk Service (rio-deal-desk, deployed as a FastAPI container on AWS ECS Fargate alongside serverless AWS Lambda and Step Functions workers) is the core processing engine for RIO’s revenue intelligence capabilities.

It ingests Deal Desk team communications—focusing exclusively on email threads involving Deal Desk team email addresses (including internal quote approval requests, discount exception discussions, rep-to-analyst negotiations, and buyer exchanges)—with RIO’s analytics engine. It manages per-user Google Workspace / Gmail OAuth connections, ingests email threads into encrypted S3, executes a 6-stage AI enrichment pipeline (Amazon Bedrock LLMs & Nova Pro / Titan Embeddings), and serves structured deal intelligence (MEDDPPICC scoring, pricing rationale, timelines, quote revision graphs) via REST APIs.

Key Service Deliverables

OAuth Connection Sequence

                  POST /authorize                          GET /callback
   Frontend ────────────────────────▶ FastAPI API ────────────────────────▶ FastAPI API
      │                                   │ (sign HMAC state,                    │ (verify state,
      │                                   │  call AgentCore)                     │  store in AgentCore,
      │   authorization_url               │                                      │  update DB, publish event)
      │◀──────────────────────────────────┘                                      │
      │                                                                          │
      └─▶ Redirect user ▶ Google Consent ▶ AgentCore Vault ▶ Redirect Callback ──┘
                                                                                 │
                                                                                 v
                                                             EventBridge: GoogleConnectionEstablished
                                                                                 │
                                                                                 v
                                                                 Step Functions: Email Backfill Pipeline

HTTP API Surface (10 Endpoints)

The FastAPI application running on ECS Fargate exposes ten routes across three core operational groups:

1. Integration & Connection Lifecycle (/activity/integrations)

• POST /activity/integrations/{provider}/connections/{person_id}/authorize: Initiates per-user OAuth flow. Returns signed HMAC state and Google authorization_url generated via AWS Bedrock AgentCore Identity.
• GET /activity/integrations/{provider}/callback: OAuth callback endpoint handling Google redirect. Validates HMAC state, registers OAuth tokens in AgentCore token vault, updates Postgres oauth_connection table, and publishes GoogleConnectionEstablished event.
• GET /activity/integrations/{provider}/connections/{person_id}: Retrieves connection status (connected, connected_at, granted scopes, error).
• GET /activity/integrations/google/account-summary: Returns summarized Gmail mailbox account metadata for the authenticated user.

2. Opportunity Email Threads (/activity/opportunities)

• GET /activity/opportunities/{id}/email-summary: Retrieves aggregated email activity metrics and key thread indicators for a target CRM opportunity.
• GET /activity/opportunities/{id}/threads: Returns paginated email threads linked to an opportunity, including participant lists, message dates, and thread resolution status.

3. Deal Intelligence Serving (/activity/deal-intelligence)

• GET /activity/deal-intelligence/{id}: Retrieves cached AI-generated deal intelligence from Postgres deal_intelligence table (cached with a 24-hour TTL).
• POST /activity/deal-intelligence/{id}/generate: Triggers an async Lambda worker to compute fresh AI deal intelligence for an opportunity (pricing justification, MEDDPPICC extraction, timeline, quote graph).

6-Stage Serverless Intelligence Pipeline

  +-------------------+       +-------------------+       +-------------------+
  | Stage 1: Ingest   | ───▶  | Stage 2: Parse    | ───▶  | Stage 3: Resolve  |
  | Gmail -> S3/CH    |       | Bedrock Converse  |       | 3-Lane Resolution |
  +-------------------+       +-------------------+       +-------------------+
                                                                    │
                                                                    ▼
  +-------------------+       +-------------------+       +-------------------+
  | Stage 6: Serving  | ◄───  | Stage 5: Synthesis| ◄───  | Stage 4: Distill  |
  | Deal Intelligence |       | Canonical/Pattern |       | Claude + Titan    |
  +-------------------+       +-------------------+       +-------------------+

Stage 1 — Ingestion & Corpus (Sync Schedule & Mechanics)

• Backfill Sync: Triggered immediately upon receiving GoogleConnectionEstablished event. Executes email_backfill.asl.json with 2-way parallel Map concurrency. Backfills up to 90 days (newer_than:90d query) of historical email messages.
• Scheduled Sync: EventBridge Scheduler cron (rate 1 day) launches email_sync.asl.json. Fetches incremental email deltas via Gmail historyId cursor stored in Postgres email_sync_state.
• Corpus Storage: Raw email content is stored durably in Amazon S3 (${DeployPrefix}-email-corpus), partitioned by tenant and date ({tenant_id}/year=/month=/day=/{msg}.json), encrypted with a dedicated per-environment Customer Managed Key (CMK) via Server-Side Encryption (SSE-KMS).
• Corpus Index: Metadata is logged to ClickHouse dd_corpus_index table (deduplicated on external_id).
• Attachment Pipeline: Asynchronous trigger via EmailIngestionSucceeded event launches email_attachment_pipeline.asl.json to store attachments in S3 and log metadata in Postgres email_attachment.

Stage 2 — Parsing

• Orchestration: Scheduled via email_parsing_pipeline.asl.json.
• LLM Engine: Amazon Bedrock Converse API (inline for under 100 artifacts, Bedrock Batch for over 100).
• 3-Level Parsing Resilience:
  1. Attempt 1: Parse body + all attachments.
  2. Attempt 2: If content filtering/size limits fail, retry with small attachments only (under 4MB).
  3. Attempt 3: If still failing, fallback to body-only text extraction.
  4. Permanent Failure: Record entry in Postgres parsed_email_failure and emit EmailParsingFailed event.
• Output: Parsed text saved to S3 & Postgres parsed_email. Idempotency tracked in DynamoDB pipeline-idempotency.

Stage 3 — Attribution & Resolution

• 3a. Attribution (email_attribution_pipeline.asl.json): Reads S3 message headers and maps sender/recipient email addresses to CRM person and account IDs using AddressResolver. Populates ClickHouse dd_fact_artifact, dd_fact_artifact_participant, and dd_fact_thread.
• 3b. Resolution (email_resolution_pipeline.asl.json): Matches email threads to CRM entities using 3 evidence lanes:
  • CRM Lane: Direct structural lookup (Account -> Opportunities, Products, Quotes, Regions).
  • LLM Lane: Claude extracts candidate entities from text.
  • Text Lane: Regex/heuristics matching SKUs, deal sizes, region codes.
• Claim Fusion: Fusion engine weighs lane confidence scores to resolve Opportunity, Product, Quote, and Region per thread.
• Persistence: Writes resolved threads to Postgres thread_resolution and ClickHouse dd_fact_thread / dd_fact_thread_learning.

Stage 4 — Distillation & Embedding

• Orchestration: email_distillation_pipeline.asl.json.
• LLM Processing: Amazon Bedrock Claude extracts key business learnings, risks, and positive indicators per thread.
• Vector Generation: Bedrock Titan Embeddings model generates 1024-dimensional vector embeddings for each distilled learning.
• Output: Written to ClickHouse dd_fact_thread_learning with slice tags (product, region, deal_type, deal_size_band). Emits EmbeddingsBatchReady event.

Stage 5 — Knowledge Base (Canonical Learning & Pattern Synthesis)

• 5a. Canonical Learning (canonical_learning_pipeline.asl.json): Triggered by EmbeddingsBatchReady event. Performs Hierarchical Navigable Small World (HNSW) vector similarity search in ClickHouse to cluster near-duplicate learnings into canonical rows in dd_canonical_learning. Retries collapse deterministically via uuid5(slice + learning_id).
• 5b. Pattern Synthesis (pattern_synthesis_pipeline.asl.json): Scheduled pipeline using Bedrock Claude to synthesize high-level business patterns per slice (product x region x deal_type x deal_size_band) into ClickHouse dd_synthesized_pattern.

Stage 6 — Deal Intelligence Serving

• Trigger: Async Lambda triggered on-demand via POST /activity/deal-intelligence/{id}/generate or GET cache miss.
• LLM Reasoning: Combines CRM opportunity details, quote revision histories, and distilled email learnings using Bedrock Claude.
• Generated Outputs:
  • Pricing Justification: Net ask, price drivers, discount justification.
  • MEDDPPICC Extraction: Full methodology scoring (Metrics, Economic Buyer, Decision Criteria, Decision Process, Paper Process, Identify Pain, Champion, Competition).
  • Chronological Deal Timeline: Visual deal milestones.
  • Quote Revision Graph: Historical quote changes and discount iterations.
• Caching: Stored in Postgres deal_intelligence with a 24-hour cache TTL.

Data Stores & Schemas

PostgreSQL (Transactional State — RDS)

• oauth_connection: Per-user OAuth connection state, tokens vault reference, granted scopes.
• email_sync_state: Gmail historyId watermark cursor and backfill status per user.
• parsed_email: Body/attachment parse status and S3 object keys.
• parsed_email_failure: Attachment and message parsing failure audit logs.
• email_attachment: Metadata for downloaded email attachments.
• thread_resolution: Thread-to-CRM entity resolution outputs, risks, positive indicators.
• deal_intelligence: Cached daily AI deal intelligence results.

ClickHouse (Analytics Warehouse — dd_ Tables)

• dd_corpus_index: Index of all ingested raw S3 email artifacts.
• dd_email_sync_run_log: Audit execution logs for sync and backfill passes.
• dd_fact_artifact: Per-email attribution facts (sender, account, timestamp).
• dd_fact_artifact_participant: Email participant mappings (To, Cc, Bcc).
• dd_fact_thread: Rollup metrics and resolution attributes per email thread.
• dd_fact_thread_learning: Thread learnings with 1024-dim Titan vector embeddings.
• dd_canonical_learning: Clustered canonical learnings from vector similarity search.
• dd_synthesized_pattern: Synthesized business patterns per product/region/deal slice.

DynamoDB & S3

• DynamoDB (pipeline-idempotency): Stage-level idempotency lock table preventing duplicate processing across re-runs.
• Amazon S3 (prod-rio-email-corpus): Server-Side Encrypted (SSE-KMS) bucket storing raw normalized email payloads.

Observability, Event Routing & Security

Event Sources

• rio.activity: Used by API endpoints, deal intelligence triggers, and pipeline failure alerts.
• rio.core.activity: Used exclusively by the email ingestion Lambda for audit events.

Notification Alert Routing

Ingestion failure events append detail.target_service = "rio.platform.notification" into their payload. The Notification Service EventBridge rule filters on this field and detail.event_name to route alerts to technical notification channels.

High-Level Service Architecture

The diagram below shows the high-level architecture — Ingestion & Corpus, Attribution & Distillation, Knowledge Base, Output, Feedback Flywheel, and Key Design Principles — for the RIO Deal Desk Service.

Event Wiring (Service Message Flow)

Auto-generated view of which events this service sends and receives on the EventBridge bus.