The Outlook inbox is where the conversations live: cargo nominations, charterer questions, supplier disputes, class society reminders, flag administration follow-ups, master-to-shore reports, near-miss escalations. The data inside it is what the rest of the pipeline reads against — but only if it’s accessible. The Email Surveillance pipeline does two jobs:Documentation Index
Fetch the complete documentation index at: https://docs.appliedaifoundation.org/llms.txt
Use this file to discover all available pages before exploring further.
- Sync — pull the inbox into a local searchable store so other pipelines can reference it without hitting Outlook every query.
- Topic tracking — for the recurring topics the fleet generates email about (PMS, class, SIRE/CDI, technical forms, COC dispensations, MoM, TMSA, performance, fleet-wide notifications), track the latest mail sent and the pending follow-ups.
Where the data comes from
| Source | What it provides |
|---|---|
| Microsoft Outlook (via Graph API) | Email metadata, body, attachments, folder labels, conversation linkage |
| MSAL token cache | Authentication for delta-query reads |
| Local SQLite index | The synced corpus — written by this pipeline, read by every other |
Sync
Outlook isn’t a useful real-time queryable surface — Microsoft Graph API rate limits, latency, and the requirement to authenticate per query make it slow and expensive. The sync turns the live inbox into a local SQLite index plus on-disk attachments.| Layer | Stored | Used for |
|---|---|---|
| Email metadata | sender, recipients, subject, dates, folder, labels | Search, classification |
| Email body | HTML and plain-text | Content extraction, entity recognition |
| Attachments | PDFs, Excel files, screenshots | Read by other skills (e.g. PDF to markdown) |
| Thread linkage | conversation-id grouping | Reply-chain reconstruction |
| Sync state | last sync timestamp, sequence number | Incremental sync — no double-fetching |
Topic tracking
Beyond raw sync, the pipeline tracks 20 recurring email topics. Each topic surfaces:“When was the last mail sent on this topic, what did it say, and what’s the response status?”The topics, organised by audience:
Class & statutory
Class & statutory
- Class status — class society correspondence: surveys, conditions, certificate updates
- CoC and dispensation — Conditions of Class and dispensation requests / approvals
- Medical chest certificate — annual certificate correspondence
Vetting & commercial
Vetting & commercial
- SIRE and CDI — current inspection cycle correspondence
- SIRE / CDI status due in 2 months — pre-inspection preparation
- VIR status fleet-wise — charterer inspections across the fleet
- VIR status due in 1 month — pre-inspection preparation
- TMSA mail — Tanker Management and Self-Assessment correspondence
Technical & operational
Technical & operational
- PMS mail — maintenance correspondence with vessels
- Performance mail — engine / vessel performance correspondence
- Technical form submission status — chasing late forms
- Missing technical form status — escalations on never-submitted forms
- LO shore analysis — lubricant lab correspondence
- Consumption log review status — emissions data quality follow-up
- LO Nissen Kaiun report status — owner-specific report correspondence
Operational regulatory
Operational regulatory
- Vessels calling Paris MoU ports — pre-inspection prep mail to masters
- Vessels calling Australia — pre-inspection prep mail (AMSA)
- MoM mail — Minutes of Meeting follow-ups
Data model
Inside the SQLite index:What the senior review (or daily digest) contains
For an operations or technical superintendent, the daily digest is the most useful surface:- New mails of interest — count, sender breakdown, top subjects.
- Topic dashboard — every tracked topic with status (Responded / Awaiting / Never sent / Follow-up due), latest sent date, days-since.
- Follow-ups due — the focused list: topics that have gone quiet beyond their response window.
- Attachments to process — PDFs, Excels, BDNs, lab reports, certificates received but not yet routed to other pipelines.
- Thread alerts — conversations that grew unusually fast (10+ replies in 24h often means an incident).
- Fleet-wide patterns — the same topic chased across multiple vessels in the same week (procedural drift across the fleet).
Cross-pipeline integration
The email index is read by other pipelines:- Class reads class-society correspondence to surface dispensations and survey responses.
- Compliance reads SIRE / CDI / VIR correspondence to track operator-response status.
- Forms reads form-submission chasing emails to distinguish “engineer didn’t submit” from “engineer submitted to wrong recipient”.
- Maritime report generator pulls incident-related email threads when generating investigation reports.
Authentication and rate-limit handling
Microsoft Graph API enforces tenant-level rate limits (10,000 requests / 10 min default). The sync respects the limit by using delta queries (only fetching changed items) and adaptive back-off when the API returns 429 Too Many Requests. Auth uses the existing MSAL token cache. The pipeline does not store credentials directly — token refresh follows the standard MSAL flow and the cache is reused across sync runs.What the pipeline does not do
- It does not send emails on behalf of users.
- It does not auto-archive or delete.
- It does not modify mail content in place.
The single most consequential improvement most fleets can make to their email-driven processes is subject-line standardisation. The topic-tracking templates rely on subject patterns; a pre-agreed prefix per topic (“[CLASS]”, “[PMS-CHASE]”, “[VIR-PRE]”) increases tracker recall from ~70% to >95%.
References
Templates: email-processing
20 templates covering the recurring topic trackers — class, PMS, SIRE/CDI, VIR, TMSA, technical forms, performance, MoM, AMSA / Paris MoU, and more.
Related: Class
Class correspondence is read out of this index.
Related: Compliance
SIRE / CDI / VIR response tracking pulls from email threads here.
Related: Maritime Report Generator
Investigation reports cite specific email threads — they’re sourced from this index.