One compatible endpoint
Point existing OpenAI or Anthropic-compatible clients at the broker and select governed model aliases instead of vendor-specific endpoints.
PRODUCT 01 / AGENT ACCESS MANAGER
On-Prem AI Gateway and Multi-Provider LLM Broker: One endpoint. Every LLM vendor.
Agent Access Manager is an on-prem AI gateway and broker: applications use one OpenAI-compatible endpoint and virtual keys while the broker controls provider credentials, routing, usage, guardrails, and audit.
ON-PREM REQUEST PATHOPENAI-COMPATIBLE
01
APPLICATIONOpenAI SDK / AI application
AUTHENTICATEDAuthorization: Bearer sk-virtual-key POST /v1/chat/completions · model: fast
02
AI GATEWAY / BROKERAgent Access Manager
GOVERNEDkey / budget / guardrail / route / meterSCOPE
org / team / project · RPM / TPM · PII / secret policyPRIMARY DEPLOYMENTAnthropic / Geminitranslated request
FALLBACK DEPLOYMENTOpenAI / Groq / Ollamahealth-aware route
DURABLE AUDITVirtual key + provider + model + tokens + outcome
call_8F21ACONTROL OUTCOMES
Replace direct vendor integrations with one governed AI access layer.
Virtual application keys
Issue vendor-agnostic keys that can be scoped, revoked, suspended, and expired while master vendor credentials remain encrypted at rest.
Resilient model routing
Map aliases to several deployments, translate provider formats, spread load, track health, and fall back across vendors on failure or budget pressure.
Metered, guarded calls
Apply scoped budgets, RPM/TPM limits, request and response guardrails, and durable call-level usage and audit records.
REFERENCE ARCHITECTURE
Configure once, then route every application call through the broker.
Register providers
Store each vendor endpoint, protocol, and encrypted credential in the on-prem catalog.
Define deployments
Map a stable model alias to one or more real upstream models for load balancing and fallback.
Mint virtual keys
Issue revocable application credentials scoped to an organization, team, or project with budgets and rate limits.
Proxy and record
Authenticate, guard, route, stream, meter, and record the provider, model, tokens, cost, and final outcome.
GATEWAY TELEMETRY
ILLUSTRATIVE REFERENCE VIEWTIMEVIRTUAL KEYMODEL ALIASPROVIDEROUTCOME
14:32:08vk_finance_07fastanthropicSUCCESS
14:31:55vk_support_02reasoningopenaiFALLBACK
14:31:41vk_research_12summarizegeminiBLOCK
PRODUCT SPECIFICATION
One deployable. Two interfaces. Enforced module boundaries.
A Spring Modulith architecture keeps the operational path simple while separating access, routing, governance, and analytics responsibilities.
POST /admin/providersPOST /admin/deploymentsPOST /admin/keysPOST /admin/budgetsRegister vendors and model aliases, encrypt provider credentials, mint application keys, and configure scoped controls.
POST /v1/chat/completionsPOST /v1/responsesPOST /v1/messagesPOST /v1/embeddingsUse existing OpenAI or Anthropic-compatible clients with a broker-issued virtual key; streaming is supported.
iamVirtual keys and /v1 authentication
catalogProviders, deployments, encrypted credentials
directoryOrganizations, teams, and projects
providersOpenAI, Anthropic, Gemini, and Vertex adapters
routingLoad balancing, health, and fallback
gatewayOpenAI-compatible request pipeline
analyticsPostgres call audit and usage records
billingBudgets, spend, RPM, and TPM
guardrailsPII, secret, and denylist policy
siemOptional OpenSearch event projection
soarOptional reversible abuse containment
sharedEvents and common value types
ON-PREM FIRST / POSTGRES SYSTEM OF RECORD / OPTIONAL OPENSEARCH + VALKEY + KEYCLOAK
TRUST ARCHITECTURE
Built for security review, not security theater.
Map virtual-key access, encrypted provider credentials, guardrail decisions, and audit evidence to the controls your enterprise already operates.
SOC 2 control alignment
Architecture supports evidence collection for access, change, and monitoring controls.
ISO 27001 readiness
Map credential, access, policy, and audit practices to ISMS control objectives.
NIST AI RMF mapping
Operational visibility across governed model access, guardrails, usage, and provider outcomes.
Control-alignment statements describe product architecture and are not claims of EnvisionAI certification.
WORKS WITHOpenAIAnthropicGeminiVertex AIGroqOllama
TECHNICAL EVALUATION
Review the control path with your security and platform teams.
Map the architecture to your providers, application clients, governance scopes, data boundaries, and operating requirements.