Your Fabric data agent is answering from world knowledge — and not telling you

2026-08-25 · Nikhil “Nick” Harinath · ~4 min

You wire a Fabric data agent into an Azure AI Foundry agent, ask it a question about your own warehouse, and get back a fluent, confident, plausible answer. Someone on the business side spots that the number is wrong. You go looking for a bad join or a stale semantic model.

It may not be your data at all. It may be that your agent never reached your data, and answered from the model's general knowledge instead — without saying so.

Why it happens: OBO means the agent is not the caller you think

The Fabric data agent tool uses on-behalf-of (OBO) authentication. The query against the Fabric data plane does not execute as the agent, or as a service principal you configured, or as whoever owns the Foundry project. It executes as the identity that invoked the agent.

That's a sensible security design — row-level security and workspace permissions follow the human asking the question. But it produces a failure mode that is easy to misread, because the identity doing the work is often not the identity you were thinking about while you built the thing.

The common shape of it:

• Your Azure subscription owner — the account you build and click with — is a B2B guest in the tenant, something like you_gmail.com#EXT#@tenant.onmicrosoft.com.
• Fabric workspace access was granted to a member account in that tenant.
• Everything in the portal looks fine, because portal access and Fabric data-plane access are different grants.
• You invoke the agent as the guest. OBO passes the guest identity to Fabric. Fabric declines. Retrieval returns nothing.

The dangerous part is the fallback, not the failure

A retrieval tool returning an error is a normal, recoverable condition. The problem is what most agent orchestrations do next: they treat the empty tool result as "no additional context" and let the model answer anyway.

So instead of "I could not reach the data", you get a fully-formed answer synthesized from pretraining. For questions about public topics, that answer will be superficially reasonable — which is exactly what makes it expensive. A loud failure gets fixed in ten minutes. A silent one gets quoted in a deck.

Grounded retrieval that fails open is worse than no retrieval at all, because it looks identical to retrieval that succeeded.

How to confirm it in a couple of minutes

1. Read the trace, not the answer. Confirm the data-agent tool was actually called and what it returned. An empty or errored tool result under a confident answer is your diagnosis.
2. Print the effective identity. Check the UPN that reached the data plane. If it contains #EXT#, you are running as a guest.
3. Re-run as a tenant member that holds explicit Fabric workspace access. If the same question now returns real numbers, it was never a data problem.

Fixes, in the order I'd try them

• Run as a member identity with Fabric workspace access. Simplest and usually correct. Note that the two identities may hold different rights elsewhere — the member may lack the subscription-level RBAC you need for capacity operations, so you end up switching between them deliberately rather than merging them.
• Make retrieval failure fail closed. If the retrieval tool errors or returns empty, the agent should say so rather than answer. This is a prompt-and-orchestration change, and it is the fix that keeps paying off — it converts every future grounding failure from silent to visible.
• Assert grounding in your evals. Add a check that answers cite retrieved rows. An eval that only scores plausibility will happily pass a hallucination.

The general lesson

Most of the hard part of enterprise AI is not the model. It's identity, permissions, and what your system does when a dependency quietly returns nothing. OBO is a good default, but it moves the security boundary to a place where a misconfiguration doesn't look like an error — it looks like an answer.

When an agent gives you a confident answer, the first question is not "is this right?" It's "what did it actually read?"

← All notes  ·  Work with me