Context
The operating situation
Audit and compliance questions about historical trading activity had to be routed to engineers who could write the query. The wait was measured in hours, and the people who needed the answer could not check the work.
Challenge
The operating problem
Audit and compliance staff needed answers from historical trading data but could not write SQL, and the obvious fix - letting a model generate queries directly - would have put unreviewable free-form output in front of a regulated workflow.
Approach
Decisions that shaped it
- Defined a strict JSON query schema and made the model's only job the translation of a question into that schema.
- Reused the schema that already backed the manual UI search, so AI-assisted and hand-built queries resolved through one validated path.
- Evaluated two providers against the same protocol so the design did not depend on a single vendor.
- Refused unmappable questions rather than letting the model approximate an answer.
Solution
What was delivered
A model-agnostic query layer: natural language in, a validated JSON query schema out, executed through the same search path the manual UI already used. Nothing the model produced reached the data store unvalidated.
Outcome
Documented outcome
A successful two-month proof of concept in which query construction fell from hours to seconds, with AI-assisted and hand-built queries resolving through one validated path.
This was a two-month proof of concept and did not reach production, so no adoption, usage, or business-outcome figure is claimed for it. The schema constrains output and narrows unsupported answers; that is a design property, not a measured accuracy result, and no evaluation data is published. The former employer stays anonymized.
Controls
What kept it safe to operate
- Model output is constrained to a validated schema over known fields and sources; it is never executed as free-form SQL.
- Questions that cannot be mapped to the schema are refused rather than guessed at.
- AI-assisted and manually built queries resolve through one validated path, so there is a single place to audit.
- Two providers were evaluated against the same protocol, so no single model's behaviour is load-bearing.