How Microsoft Foundry Toolboxes Reduce Your AI Agent Token Costs

In this blog post How Microsoft Foundry Toolboxes Reduce Your AI Agent Token Costs we will explain why capable AI agents often become more expensive as new tools are added, and how a better toolbox design can control that cost without limiting what the agent can do.

The problem is rarely one expensive question. It is thousands of small, unnecessary charges caused by an agent repeatedly loading instructions about tools it does not need. This waste can remain hidden until the agent moves from a small trial into everyday business use.

What is a Microsoft Foundry Toolbox?

Microsoft Foundry is a platform for building, managing and monitoring business AI applications and agents. An agent is an AI assistant that can reason through a task and use approved tools to retrieve information or take action.

A Microsoft Foundry Toolbox packages a controlled collection of those tools behind one managed connection. Tools might include web search, document search, code execution, internal business APIs or external systems connected through Model Context Protocol, commonly called MCP.

MCP is a standard way for AI applications to discover and use external capabilities. Think of it as a common plug that lets an agent connect to different business systems without every connection being built from scratch.

We cover those integration options in more detail in our guide to connecting Foundry agents to business systems with MCP and OpenAPI.

Where the hidden token cost comes from

AI models process text as tokens, which are small pieces of words and sentences. Providers generally charge for the tokens sent to the model and the tokens generated in response.

Before an agent can choose a tool, the model may need to receive each tool’s name, description and input format. If the agent has three tools, that overhead may be modest. If it has 30 or 50 tools, the model can receive a large catalogue on every turn.

This creates three business problems:

• Higher AI bills: You pay to send tool definitions that have nothing to do with the current task.
• Slower responses: The model must work through more information before deciding what to do.
• More mistakes: Similar or poorly described tools can cause the agent to select the wrong action.

A toolbox alone does not automatically remove all this overhead. The real savings come from combining a well-organised toolbox with concise tool definitions and Foundry’s Tool Search capability.

How Tool Search reduces unnecessary context

Tool Search lets the agent discover capabilities when it needs them instead of receiving the full tool catalogue at the start of every request. In plain English, the agent asks, “Which tool can check this customer’s invoice?” and Foundry returns only the relevant options.

Without Tool Search, a large toolbox can place every definition into the model’s working context. With Tool Search enabled, the model initially sees two focused capabilities: one for finding an appropriate tool and another for calling the selected tool.

A simplified configuration looks like this:

{
 "tools": [
 {
 "type": "toolbox_search_preview"
 }
 ]
}

The full toolbox remains available, but its contents do not all need to occupy the model’s attention at once. Microsoft currently suggests considering Tool Search when a toolbox grows beyond roughly 10 to 15 tools or when different requests require very different capabilities.

At the time of writing, Tool Search is a preview feature. It should be tested carefully before becoming a dependency for a critical production process.

Five practical ways to lower agent token costs

1. Measure the current cost before changing anything

Start by recording the input tokens, output tokens, model calls and tool calls for common business tasks. Measure complete tasks rather than isolated prompts.

For example, “resolve a customer billing question” may involve six model calls, two searches and one API request. That is the unit you need to cost because it reflects what the business is actually paying to complete.

2. Build toolboxes around business intent

Avoid creating one enormous company-wide toolbox simply because it is convenient. Group tools around clear purposes such as customer support, finance operations, employee onboarding or security investigation.

This reduces the number of possible tools, improves selection accuracy and makes ownership clearer. A finance agent should not need to evaluate software deployment tools when it is checking an overdue invoice.

3. Use Tool Search for larger or mixed toolboxes

Tool Search is most valuable when an agent has access to many tools but normally needs only one or two for each request. It allows the agent to find the relevant capability dynamically rather than carrying the entire catalogue through the conversation.

However, do not enable it and assume the work is finished. Test realistic questions, confirm the correct tools are discovered and compare the total tokens used before and after the change.

4. Shorten tool descriptions and input formats

Tool definitions should be clear, but they do not need to read like operating manuals. Long descriptions, repeated warnings and oversized input formats can add thousands of tokens across a multi-step interaction.

Give each tool one clear purpose. Remove duplicated wording, unnecessary examples and fields the agent will never use. If two tools sound almost identical, rename them so the business difference is obvious.

5. Control how much information tools return

Token waste does not stop after a tool is selected. A database or search tool that returns 200 records when the agent needs five can create a much larger model request.

Apply filters, pagination and response limits before the information reaches the model. Return only the fields needed for the decision, and summarise large results outside the main conversation where possible.

What the savings could look like

Consider an illustrative 200-person professional services firm with an internal operations agent. The agent has 30 tools covering customer records, projects, invoices, policies and employee requests.

If each tool definition averages 250 tokens, loading all 30 tools adds around 7,500 input tokens to a model turn. An eight-turn workflow could consume 60,000 tokens in tool definitions before counting the user’s question, conversation history or tool results.

At 5,000 workflows per month, that represents up to 300 million input tokens of potential tool-definition overhead. Actual charges depend on the model and how the application manages context, but the example shows why a low-cost pilot can produce a surprisingly large production bill.

With an intent-based toolbox and Tool Search, the model can start with a small discovery layer and load only the few definitions relevant to the task. The business outcome is not just a lower token bill. Responses can also become faster and tool selection more reliable.

Cost control must include security and governance

Reducing tokens should never mean giving an agent broad, unmanaged access. Each agent still needs permission to use only the systems and data required for its job.

Foundry Toolboxes centralise connections, authentication and versioning, making it easier to update tools without rewriting every agent. Managed identity can also remove stored passwords and access keys, while role-based access control limits who can manage or use each resource.

Our guide to securing Microsoft Foundry projects with RBAC and managed identity explains how these controls work in practice.

Toolbox versions should be tested before promotion, particularly when a change can affect financial records, customer data or external actions. This gives teams a controlled way to improve costs without introducing unexpected behaviour.

Start with one high-volume workflow

The best place to begin is not your most impressive AI demonstration. Choose a repetitive, high-volume workflow where costs and outcomes can be measured, such as service desk triage, policy questions or customer record lookup.

Record the baseline, remove unnecessary tools, shorten definitions, limit returned data and test Tool Search where appropriate. Then compare cost per completed task, response time and accuracy.

CloudProInc brings more than 20 years of enterprise IT experience to this work. As a Melbourne-based Microsoft Partner and Wiz Security Integrator, we help organisations build practical Foundry, Azure, Microsoft 365 and security solutions without the overhead of a large, faceless provider.

If your AI agent costs are climbing or you cannot clearly explain what each workflow costs, we are happy to review the design and identify where tokens are being wasted — no strings attached.