Forward Deployed Engineer, live answer, requirement 3 of 12

Retrieval-augmented generation

Chat answers over a customer's own documents, returned with the matched document names, after a check that the caller may use that persona.

Built by Gus Assuncao of Gus IT LLC for a Forward Deployed Engineer application. Luca is the AI platform Gus IT builds and operates.

Try it

Ask about the engineering use cases I have published on gusit.de. The demo retrieves with the hybrid method described here, answers only from the passages it finds, and shows the score behind each one. If nothing clears the threshold, it says so instead of answering.

Public demo on Cloudflare Workers AI. Inputs are screened by Llama Guard 3 before any other model runs, requests are rate limited, and nothing here connects to a Luca or customer system.

The requirementBuild retrieval-augmented generation over a customer's own documents, so answers are grounded in their sources and cite them.

How it works

Scope the sourcesI sit with the data owner, list every document source, and record who may read each one. No document is embedded until those access rules are agreed.
Embed on a local GPULuca splits each uploaded file into overlapping passages of about 1,200 characters, then embeds them. By default it calls an embedding model on a local GPU, so the text stays inside the environment. If the local model is unreachable or the call fails, Luca falls back to a hosted embedding model. I tell the data owner about that fallback before any restricted document is loaded.
Hybrid retrievalLuca blends semantic similarity with keyword overlap, weighted 85 to 15 by default, then diversifies the results with Maximal Marginal Relevance. By default it keeps at most two passages from one document, so one long document cannot crowd out the rest. Passages under a minimum similarity score are dropped first, and I test that cut-off against the customer's own questions.
Answer with sourcesLuca adds the retrieved passages to the persona's chat prompt and tells the model to mention the document name when it cites one. The reply comes back with the matched document names and their scores. Luca does not refuse on its own when nothing matches, so I add that rule to the persona prompt and test it.

Tools

Tool	What it does here	Status
Luca Express RAG engineAvailable	Uploads and embeds documents into one store per persona, with a query API that checks persona and document access.	Available
Luca persona chatAvailable	Adds retrieved passages to a persona's chat prompt and returns the matched document names and scores with the reply.	Available
Luca OnEarly access	Embeddable website chat that answers from a knowledge base built from each customer's site and documents.	Early access
Cloudflare Workers AIDemo only	Runs this public demo: bge-base embeddings, Llama 3.3 70B for the answer, and Llama Guard 3 to screen each question.	Demo only

Available means shipped and in use. Early access means built but not generally available. Roadmap means designed, not built. Demo only means it powers the public demo, not customer deployments.

The write-up has every step of how I deliver this at a customer, with the tools and their status.

Read the write-up on gusit.de

The other answers

LLM applications in productionOn Luca, model calls normally go through a gateway that holds provider keys, records each chat call and sets a model per task.Prompt engineeringI write the output shape first, measure the prompt against each model's context window, and check replies against that shape.AI agents and orchestrationI build agents that call your systems through registered tools, still in early access, and workflows that start on business events.Request classification and routingI build automation that classifies each inbound request, plans bounded tasks, and holds high-risk tasks for approval; the dispatcher is in early access.AI workflow orchestrationI map the steps, API contracts, and data flows between a customer's systems and the models before I build the workflow.API and integration designI publish the API contract first, gate it with keys and scopes, then build integrations against it.Hybrid enterprise integrationI connect AI services to your identity, applications and data, and run the platform on Kubernetes on Azure Local.Prototype to productionI take a prototype through generated tests and a two-approval merge gate to a production deploy that rolls back if the new pod keeps crashing or cannot start.Approvals and exception handlingIn chat, I hold each Gatekeeper tool marked destructive until the signed-in user approves it, with expiry and a record. This path is in early access.Evals and output validationI built a versioned rubric judge for a tutoring app, and I trace failed or slow model calls to their provider.Client-facing deliveryI turn engineering work into updates that client architects, security leads and business owners can each act on.

Built by Gus Assuncao of Gus IT LLC for a Forward Deployed Engineer application. Luca is the AI platform Gus IT builds and operates. Contact gus@gusit.de. Claude is a product of Anthropic PBC. Cloudflare and Workers AI are trademarks of Cloudflare, Inc. Llama is a trademark of Meta Platforms, Inc. Other names are trademarks of their respective owners. Gus IT LLC is an independent engineering firm and is not affiliated with or endorsed by these companies.