Automating 12% of customer inquiries for a print-on-demand company

I built a read-only order-context service for a print-on-demand company. The primary success criterion was simple: reduce human handoffs for order-status questions without weakening ownership or disclosure controls.

The measured result I can defend is a 12 percentage-point increase in overall support deflection after release. The order-status conversations improved too, but the reporting did not isolate the service as the only cause of the overall change. That attribution limit matters. This is evidence of a useful intervention, not a claim that one workflow created every point of improvement.

The service checked merchant ownership, joined four operational systems, filtered the result before the model saw it, and returned one explanation through Fin (opens in a new tab) in Intercom Messenger (opens in a new tab). It did not give the model broad access or let a read-only question become an accidental write.

The result was a 12-point change, not a promise of savings

The primary KPI was support deflection: a conversation resolved without being handed to a human agent. The business outcome was additional support capacity. It was not automatically realized cost savings.

Measure	Baseline or basis	Observed result	Business translation	Evidence or limit
Primary KPI: support deflection	Before and after release	+12 percentage points overall	More conversations resolved without a human handoff	The service contributed to the change, but the reporting did not isolate it as the only cause
Derived capacity	10,000 conversations × 12 points	About 1,200 additional deflections	Roughly €6,000 of outsourced handling capacity at an estimated €5 per conversation	An estimate, not realized savings; it depends on staffing or BPO costs flexing with volume
Secondary operational benefit	Human agents previously assembled context across systems	One read-only explanation also became available to internal agents through Slack (opens in a new tab)	Less context switching and less need to buy broad Jira (opens in a new tab) access for support	Useful operational benefit; not separately quantified

The opportunity estimate behind the target was 15% of conversations × 80% realistically answerable = 12 percentage points. At peak volume, the equivalent capacity could be larger, but that is still a capacity estimate. I would not call it annual savings unless finance confirmed that staffing or provider spend actually fell.

The domain model decided what could be disclosed

The answer was not “find the order.” It was “find the right order, for the right merchant, then explain only the facts that merchant may receive.” These were the concepts I had to keep consistent:

Concept	Type	Description
Merchant	Actor	The authenticated party asking through Intercom Messenger (opens in a new tab). A merchant owns its orders and must not learn whether another merchant’s order exists.
Order	Entity	The merchant-facing order identifier and source of order-level state. One order can relate to several internal packages.
Package	Entity	An internal production unit linked to one order and one facility. The internal package ID is not a carrier tracking number.
Production facility	System boundary	The place where a package is produced. Its status can explain a delay but is not itself a customer-facing authority.
Order-context service	System	The read-only boundary between Fin (opens in a new tab) and internal systems. It authenticates the request, checks ownership, fetches permitted facts, and returns an explanation context.
Relation index	Entity	A prepared mapping between conversation context, package IDs, and operational records. It lets the service find related context without asking the model to search systems.
Operational or technical incident	Entity	A permitted summary from the separate Intercom (opens in a new tab) workspace or Jira (opens in a new tab). The service may use the summary only after the order is authorized.

The vocabulary prevented a common category error: authentication established who was calling, but ownership established which order the caller could open.

flowchart TD
accTitle: Order context domain model
accDescr: A merchant owns an order. An order contains packages made at production facilities. A read-only service checks that ownership before it combines order, tracking, and permitted incident context for Fin.
Merchant[Merchant] --> Order[Merchant order]
Order --> PackageA[Internal package A]
Order --> PackageB[Internal package B]
PackageA --> FacilityA[Production facility A]
PackageB --> FacilityB[Production facility B]
Order --> Service[Read-only order-context service]
Service --> Fin[Fin in Intercom Messenger]

Ownership was the first trust boundary

The person talking to Fin (opens in a new tab) was the merchant, not the merchant’s customer. The merchant could see its own order and delivery details, but should never be able to see another merchant’s order.

The request carried two inputs with different trust levels.

The following payload is illustrative; identifiers are redacted and the trust boundary is the production mechanism being shown:

{
  "authenticated_account_id": "merchant-account-123",
  "order_id_from_message": "123456789"
}

The account ID came from the verified Intercom Messenger (opens in a new tab) session. The order ID came from the merchant’s message and was extracted by Fin (opens in a new tab), so the service treated it as untrusted input.

The service performed one lookup filtered by both values:

find order
where order_id = order_id_from_message
and merchant_account_id = authenticated_account_id

If the lookup returned nothing, Fin (opens in a new tab) received the same generic response whether the order did not exist or belonged to somebody else. Only a successful match allowed the service to retrieve packages and continue.

sequenceDiagram
accTitle: Ownership is checked before order context is fetched
accDescr: Fin sends an authenticated merchant account and an untrusted order ID to the read-only service. The service checks both before retrieving any order or incident context.
participant Merchant
participant Fin
participant Service as Read-only service
participant Orders as Order data
Merchant->>Fin: Ask about an order
Fin->>Service: Account ID + order ID
Service->>Orders: Match both values
Orders-->>Service: Match or no match
Service-->>Fin: Generic denial or permitted request
Fin-->>Merchant: Explain only permitted facts

The plain-language rule was: knowing an order number is not proof that you may open it. OWASP describes (opens in a new tab) the underlying risk as Broken Object Level Authorization, but the practical defence was an ownership check on every request.

At the time, the integration used Intercom’s Messenger Identity Verification (opens in a new tab) with an HMAC (opens in a new tab) because that was the available security mechanism. Today I would use Intercom’s JWT-based Messenger authentication (opens in a new tab) for expiry, stronger control over signed attributes, and alignment with the current recommendation. The boundary stays the same: authentication does not replace resource ownership.

Four sources became one explanation

The company did not lack information. It lacked one controlled way to assemble the right information for the right merchant.

Source	What it contributed	Why it mattered
Internal order data	Order state, package state, original dates, and revised estimates	It supplied the structured facts already known by the company
AfterShip (opens in a new tab) tracking data	Tracking events from the shipping integration	It explained what happened after a package left production
Separate Intercom (opens in a new tab) workspace	Conversations between production facilities and operations	It added human operational context that support could not otherwise see
Jira (opens in a new tab)	Technical issues affecting an order or package	It explained failures such as a print file not reaching production

The service joined those sources only after the merchant and order had passed the ownership check. It then returned a small explanation context rather than forwarding raw records to the model.

At the time, the relation index was maintained outside Intercom (opens in a new tab) because the service needed a dependable package-to-conversation lookup. Today I would first test whether Intercom’s search and custom-attribute features (opens in a new tab) could hold enough of that relation directly. If a separate index were still needed, I would evaluate n8n Data Tables (opens in a new tab) before reaching for a separate database for a small normalized index. I would keep a real database when stricter access boundaries, uniqueness guarantees, or heavier concurrency justified it.

Disclosure happened before the model saw context

The service did not fetch every incident and ask the model to decide what was safe. It checked a sensitivity field before reading full contents. The safe branch removed unnecessary personal data and returned a bounded summary.

The following is a simplified representation of the production rule; the example keeps the mechanism while omitting implementation-specific details.

if source.sensitive == true:
    stop_before_fetching_full_contents()
    return "No safe operational detail available"

context = fetch_permitted_source(source)
context = redact_unnecessary_fields(context)
return context

The important limitation was uncomfortable: the sensitive field defaulted to false, and a human had to turn it true. That meant an unclassified source could enter the permitted branch. The rule was deterministic, but the classification input still depended on human governance.

Today I would keep the explicit field but add fail-closed treatment for known high-risk incident types. That trades some coverage and operations convenience for a smaller chance that an unclassified record is disclosed. I would make the decision with the process owner and security owner, not hide it inside a model prompt.

Latency changed the shape of the design

Fin (opens in a new tab) expected the tool to return quickly. That constraint changed what belonged in the live request:

• The request path handled ownership, source reads, filtering, and one concise explanation context.
• Package-to-conversation relation work happened asynchronously before the request needed it.
• A slow or unavailable source produced a bounded answer or a human handoff, not a confident sentence assembled from half a response.

This is a useful boundary for other systems too: do the work that makes the answer safe in the request path, and move expensive enrichment out of it when the user does not need to wait for that enrichment.

Logging was another data-design decision

The workflow execution history made troubleshooting possible, but it also became another place where operational context could exist. In a current version I would decide this alongside the data model:

The operational lesson is broader than this service: observability is a system boundary. Make it inspectable without making it an ungoverned second database.

What generalizes beyond order status

The pattern is not “put an AI agent in front of four systems.” It is a bounded read path:

This pattern fits when a reader needs a trustworthy explanation assembled from several systems and the capability can remain read-only. It does not fit when the core problem is a financial write, an irreversible action, or a workflow that needs long-running compensation and approval. Those need a different architecture, even if a model still helps with language.

The related architecture deep dive on governed skill libraries generalizes the same boundary-first reasoning to shared guidance rather than order context.

Common questions

Why was the service read-only?

The problem was to explain an order, not change one. Read-only scope reduced the consequence of an incorrect model response and let the strongest controls focus on authentication, ownership, disclosure, and context assembly. Refunds, cancellations, and other writes need approval, idempotency, limits, audit trails, and recovery from partial writes.

How did the service stop one merchant seeing another merchant’s order?

It authenticated the request, checked that the order belonged to the merchant, and only then looked up related records. Authentication did not grant access to every order. Ownership was its own gate before source fetches and before the model saw context.

What was the model actually allowed to see?

Only the context needed to explain the order: structured order facts, tracking events, and permitted operational or technical summaries. Sensitive records were stopped before their contents were fetched, unnecessary personal data was removed, and the model never received credentials or unrestricted source access.

The takeaway

• The primary result was a measured 12-point change in support deflection; the financial translation is estimated capacity, not claimed realized savings.
• The durable architecture was identity → ownership → permitted context → explanation, with writes outside the boundary.
• The accepted trade-off was operational coverage versus stricter disclosure classification and latency.

If your team is trying to turn fragmented operational context into a safe, useful read path, message me and tell me what the system currently makes people assemble by hand.