DataBP POC
DataBP POC / Documents / Design document

Morgan, the POC's agent: design document

A pluggable agent service: core, persona and route.

Morgan, the agent (DIG-656). Built from Pilot knowledge 111 item 817, version 12, as kept while building; private details such as addresses, identifiers and paths are masked here.

Design Document

Branch Delivery Document — Level 3 — Design Document

Effort DIG-656 (Morgan: build), under DIG-650 — The DataBP POC: Morgan, an agent that answers from the governed metrics — branch rayserrano2735/dig-656-morgan-build in Integrationals/databp-dbt
Governed by Standard: Branch Delivery Documents (K#12 I#27)
References Profiling: none: Morgan reads no source system; it reads the dbt Semantic Layer, whose metrics the reconciliation's design defines (knowledge 111 item 806) — related records: DV Digest's design for Sage, the agent copied here (knowledge 35 item 324) and its code (rayserrano2735/digest-datavault, infra/aws and docs/ask, read at 47c4608); the reconciliation's design (knowledge 111 item 806); the build tutorial, step 26 (knowledge 111 item 463); DataBP project setup (knowledge 111 item 802); the site's look (knowledge 111 item 811)
Status v1 — DRAFT — written 2026-11-20 13:31Z (not presented for agreement: its compliance section is not built)
Audience Ray Serrano, who decides; the session that builds Morgan
Author Ray Serrano; edited by Claude — 2026-11-20
Classification Internal — Data Management
Official copy method Copied from this governed template by AGENTIC.DATASKILLS.KNOWLEDGE_ITEM_FROM_TEMPLATE; template SHA-256 recorded in the created item

0. Target ERD

NA: Morgan stores no data; it reads governed metrics and keeps the conversation only in the page.

1. Purpose

Bin Luo, DataBP's CTO, can ask the POC's questions in his own words and get answers whose every figure comes from a governed metric, with the query behind it on the page. Morgan is the person he talks to: an approachable analyst who walks him through how a disagreement was found and what it costs. It is the DV Digest pattern, Sage, copied to DataBP (Ray, 2026-11-20 08:05 ET: "Now we're going to add the interactive agent"; earlier: "mostly cut and paste").

2. Consumer questions

2.1 Questions from consumers

ID Asked by Asked at Origin Question in the asker's words Answer Decision(s) Test probe State and reason
Q-1 Ray Serrano 2026-11-20 08:05 ET DataBP session "Now we're going to add the interactive agent" A chat on the POC site where Morgan answers from the Semantic Layer and shows the query behind each figure D-1, D-2, D-5 P-1 Answered in design
Q-2 Ray Serrano 2026-11-20 08:23 ET DataBP session "It has to be personable, somebody you'd want to talk to, a financial advisor" Morgan, the approachable analyst, in the site's colors D-4 P-2 Proven: Ray asked Morgan on the live page, 2026-11-20 09:22 ET: "it is working, bravo"

2.2 Questions to consumers

ID Asked of Raised by and when Question Decision blocked Interim rule Measured stakes and clock State or answer
C-1 Bin Luo Ray Serrano, 2026-11-18 Which of DataBP's pains the POC should speak to (the reconciliation design's C-1) Morgan's suggested questions (D-5) The reconciliation's own questions Unknown before the CTO round Open

2.3 Questions to source owners

ID Asked of Raised by and when Question Decision blocked Interim rule Measured stakes and clock State or answer
None None: Morgan reads only the Semantic Layer None None None None None None

3. Concept

A page sends the conversation, with an access code, to a function on AWS Lambda. The function gives Claude Morgan's instructions, the current catalogue of metrics and dimensions, and three tools: list the metrics, get a metric's dimensions, and query metrics. Each tool calls dbt's hosted Semantic Layer in Production, the same service Hex and dbt's MCP read. The answer streams back to the page as it is written, and the page shows, under the final answer, each query that produced a figure. A reply that states a figure without a query is not shown.

4. Data reality

Profile result Measured fact Design consequence
Sage's code, read 2026-11-20 at 47c4608 A conversation function of 463 lines, web functions (access code, daily allowance, secrets from AWS Parameter Store), a deployment script, and an Ask page; Claude reaches the metrics through three tools, list-metrics, get-dimensions and query-metrics, served by a MetricFlow backend on Snowflake Copied; the MetricFlow backend and its MCP server are replaced by calls to dbt's hosted Semantic Layer (D-2)
The Semantic Layer, read 2026-11-20 12:12 UTC 17 metrics on Production (counted through the GraphQL interface with Token 2, 2026-11-20 13:46 UTC), among them the five disagreement counts, the two dollar figures and the four document unit counts; dimensions include subscriber, product, unit price and month The catalogue Morgan is given (D-3)
The vault, names read 2026-11-20 "DataBP Semantic Layer Credential Token 1" and "Token 2"; "dv-digest Anthropic API key"; "dv-digest QA access code"; "claude-ws AWS Key" Morgan's secrets come from items like these (D-6)
AWS, read from Sage's deployment script Sage deploys to AWS [id], where the lab's agent function digestcards-agent also runs, through the claude-ws key (read with that key, 2026-11-20 13:29 UTC) Morgan deploys beside them (D-7)

5. Decisions

Decision ids are D-1, D-2, D-3, numbered in order within the document.

ID Decision Rationale Supporting evidence
D-1 Morgan is Sage copied: the conversation function, the web functions, the streaming of answers as they are written, the guard that serves no figure without a query (Sage's D-13), the access code, the daily allowance and the Ask page; only what DataBP changes is new Sage is proven in production and Ray asked for a copy Ray, 2026-11-20 08:10 ET, answer 2y; DV Digest design D-1 to D-14
D-2 Morgan's tools call dbt's hosted Semantic Layer in Production (environment [id]) through its GraphQL interface, in place of Sage's MetricFlow backend on Snowflake One route to the metrics: Morgan's figures are Hex's and dbt's MCP's by construction; no Snowflake identity inside Lambda. Cost: Morgan stops when that service stops, as on 2026-11-20 from 03:26 to 04:25 UTC Ray, 2026-11-20 08:19 ET, answer 1a
D-3 The catalogue (metrics, their descriptions and dimensions) is read from the Semantic Layer when a function starts and kept for 15 minutes Sage keys its catalogue to the dbt run behind it; the hosted Semantic Layer gives no run identity through these calls, so a short life keeps a changed metric from going stale for long Sage's D-9
D-4 Morgan is the approachable analyst: a woman with dark hair in a bun, navy blazer, yellow scarf and earrings, in the site's navy, sky and yellow; her instructions make her walk the user through how a disagreement was found and what it costs, in plain terms, and say that all data is synthetic Ray: a person, not a logo, "somebody you'd want to talk to" (08:23 ET); "Let's go with approachable" (08:25 ET) Tutorial step 26, item 463 v126
D-5 The page is a new page of the POC site, "Ask Morgan", with suggested questions taken from the reconciliation: what October's disagreements cost, why subscriber 43 is unbilled, which check fails most, and how November compares The site is bare until its beautification phase (Ray, 08:29 ET), so the page carries function first Ray, 2026-11-20 08:29 ET
D-6 Secrets live in AWS Parameter Store under the prefix /databp/morgan, written at deploy from vault items by Sage's ensure_secret: the Anthropic key, the Semantic Layer token and the access code. They come from the vault items "dv-digest Anthropic API key", "DataBP Semantic Layer Credential Token 2" and "Agent Code - Morgan" (Ray, 2026-11-20 08:36 ET, answers 1b, 2a, 3a; Morgan's own vault item named by Ray, 2026-11-20 13:50 ET) Nothing secret in code, files or the page Sage's web_functions.secret and deploy_functions.ensure_secret
D-7 One deployment, beside Sage's, in AWS [id], region us-east-1: function databp-morgan-conversation, Python 3.12, 300 seconds, 512 MB, reserved concurrency 2, response streaming, origins https://integrationals.github.io and http://localhost:8000; daily allowance 200 conversations A POC has one environment, as the reconciliation's QA waiver reasons; values are Sage's Sage's deploy_functions.py; the test plan's QA waiver (item 814, 5.1)
D-8 The model is Sage's default, claude-sonnet-5-5 Proven with this prompt shape and tool set Sage's conversation_function.py
D-9 The agent is a pluggable service: a core (the conversation loop, the tools, streaming, the guard, the access code and allowance) apart from two parts plugged into it, a persona (name, instructions, suggested questions, look) and a route to the metrics (dbt's hosted Semantic Layer for Morgan; Sage's MetricFlow on Snowflake could be another). Morgan is one configuration of the core; the code lives in Integrationals/databp-dbt The same agent can serve another client or another semantic layer by a new configuration, not a new copy Ray, 2026-11-20 08:36 ET: "we should think of the agent as a pluggable service", answer 4a

6. Domain specification

6.1 source-to-target map

Profile verdict Source concept and column Target area Destination table and column Identity recipe Pattern Additive? Decision Deliberate collapse, omission, or deferral
NA None: Morgan maps no source column None None None None No D-2 Morgan reads metrics; it writes no table

Completion check: every profiling Integrate row has a destination; every Defer and Reject is named as absent; every written target column traces to a source column or stated derivation.

6.2 Dictionary

Term Business meaning source-to-target mapping Mapping state Decision Verified example
Morgan The POC's agent: answers questions about the reconciliation from the governed metrics None Not mapped D-1, D-4 None yet
Query evidence The metric, grouping and filter of each Semantic Layer query behind a figure, shown under the answer None Not mapped D-1 None yet

6.3 Domain content

What Morgan knows besides the catalogue: one data provider, data provider 3; four documents per subscriber, product and month (license, entitlement, usage report, invoice); the five checks and the two dollar figures as the reconciliation's design defines them (item 806, D-2, D-10, D-12, D-22); months October 2026 to September 2027, with reports and invoices generated for each (D-23); all data synthetic, generated for the POC.

6.4 Instance models

NA: Morgan holds no rows.

6.5 Worked examples

Bin asks: "Why is subscriber 43 unbilled in October?" Morgan queries licensed_units, reported_units, units_reported_beyond_licensed and unbilled_revenue by subscriber and month for subscriber 43 and October 2026, and answers that its usage report counted 17 units with no license behind them, which at 50 dollars a unit is 850 dollars unbilled; under the answer the page shows the query. These are the answer key's figures (generator/phase_one_b.py).

7. Deferred decisions

ID Deferred decision Reason Unblocking condition Owner
None DD-1 to DD-4 settled by Ray, 2026-11-20 08:36 ET: the Anthropic key DV Digest's (1b), the Semantic Layer token Token 2 (2a), the access code its own vault item, Agent Code - Morgan (3a; named 2026-11-20 13:50 ET), the build issue DIG-656 (5y) None None None

8. Testing plan

Layer Required proof or artifact Applicability and reason
1: built-in dbt tests NA No dbt model changes
2: dbt-expectations NA No dbt model changes
3: PCM unit tests Sage's unit tests, kept and run, with the Semantic Layer calls mocked: access code, allowance, history shaping, the no-figure guard, streaming events The function's logic is Sage's, tested the same way
4: logic assertions A figure in a reply without a query is never served (Sage's D-13), tested with a planted reply The page's promise
5: detectors CloudWatch events for start, catalogue, model call, metric query and upstream error, 14 days, as Sage Operational visibility
6: reconciliation Each figure Morgan gives for the probes equals the same metric read through dbt's MCP and the answer key Morgan, Hex and the MCP must agree
7: semantic surface The catalogue Morgan is given lists the 15 metrics; each probe's query evidence names real metrics The semantic surface is Morgan's only input
Full-build guardrails, not a methodology layer Deployment checks: reserved concurrency 2, daily allowance, origins, streaming mode; no secret in the package; a live read of the page after each change Cost, access and the page

Consumer-question probes

Question Runnable probe Expected result
Q-1 On the page: "What did October's disagreements cost?" then "Why is subscriber 43 unbilled?" 1,410 dollars unbilled and 2,070 at risk; subscriber 43: 17 units reported beyond licensed, 850 dollars; query evidence under each
Q-2 Ray reads the page and talks to Morgan Ray's approval of the voice and the look

Checks from the profile

Profile result Check Layer Planned artifact
None: no profile None None None

9. Handoff boundary

The implementation guide decides how: the files copied from Sage and each change, the Semantic Layer client, the prompt in full, the page, the deployment, and the tests. This design decides what and why.

Finance workbook and initial UAT

For finance reporting branches, state the workbook's consumer, scope, business questions, expected behavior, preliminary-review conditions and final acceptance criteria under K#12 I#27. Otherwise state NA with a reason.

NA: not a finance reporting branch.

Compliance record

Built by the author last, before the document is presented for agreement (Standard: Branch Delivery Documents, K#12 I#27, "Compliance record before agreement"): every section and field of this template shown present and filled, by infra/workstation/pilot/compliance_sections.py; every rule of that standard and of the standards it cites, each with the passage that meets it or marked NOT MET; and the decisions the document carries. While this section reads NOT BUILT, the document is not presented for agreement.

NOT BUILT.