Ranked #4 of 710/18 · Last reviewed

AI gateways

A proxy between applications or agents and model providers that routes, logs and filters model requests: which models, which keys, what data goes out.

Verdict: Governs the model channel well; does not see or stop the agent's actions.

Who it suits

Platform teams that need to govern model access, spend and data sent to providers, especially for internal AI applications.

Examples

Self-hosted or managed LLM gateways and proxies, such as LiteLLM or Portkey

Cost

Open-source gateways are free to self-host, with infrastructure to run; managed gateways price by usage or plan: check each vendor's pricing page.

Pairs with

An AASB for the actions; the gateway for models, keys and spend. Unbound works alongside gateways such as LiteLLM and Portkey.

How it handles MCP connections
  • Discovery: Sees MCP tools only as tool definitions inside model requests.
  • Control: Can strip or block tool calls in model responses, without the server's context.
  • Record: Model request logs; not the MCP call or its result.

Strengths

  • Central control of models, keys and spend
  • Can redact or block data in prompts on their way out
  • A log of model requests and responses

Limits

  • Sees model traffic, not what the agent does on the machine afterwards
  • Local terminal commands and stdio MCP servers never cross it
  • Agents on some vendor subscriptions cannot be rerouted through it

Scores, with reasons

Each criterion is scored 0 to 3 as defined on the criteria page.

CriterionScoreWhy
Visibility 2Sees every model request routed through it: prompts, responses, tokens, models.
Enforcement 1Acts on the model request, by blocking or redacting, not on the command or tool call the agent then runs.
Developer friction 2Each agent needs pointing at the gateway; some agents and plans do not allow it.
Coverage of terminal and MCP actions 1Sees tool calls only as text in model responses, without their execution or result on the machine.
Audit evidence 2Request logs of the model channel, without the actions that followed.
Deployment effort 2Infrastructure to run plus per-agent configuration on every machine.
Total10/18Ranked #4 of 7. See the full table.
Other approaches