One stable endpoint
Expose an OpenAI-compatible interface while platform teams control providers, deployments, credentials, and economics behind it.
PLATFORM / ENTERPRISE LLM GATEWAY
On-Prem Enterprise LLM Gateway and Multi-Provider AI Broker: Route every model request. Keep every call accountable.
Agent Access Manager puts virtual-key access, model aliases, provider translation, budgets, rate limits, guardrails, metering, and audit into one on-prem request path.
CONTROL OUTCOMES
Centralize access, routing, safety, spend, and evidence.
Governance before upstream
Verify the virtual key and evaluate scoped budgets, RPM/TPM limits, and request guardrails before a vendor call proceeds.
Resilient routing
Select providers by policy and maintain fallback routes based on availability, latency, capacity, or cost.
End-to-end evidence
Connect virtual key, organization scope, model alias, provider, tokens, cost, guardrail decision, and outcome in one audit record.
REFERENCE ARCHITECTURE
Every call follows one governed request pipeline.
Authenticate virtual key
Verify the broker-issued key and resolve its organization, team, project, status, and applicable scopes.
Evaluate control policy
Check model policy, budget headroom, RPM/TPM windows, and pre-call PII, secret, or denylist guardrails.
Select secure route
Resolve the encrypted vendor credential and route the translated request to a healthy primary or fallback deployment.
Record the outcome
Normalize or stream the response and emit provider, model, token, cost, guardrail, latency, and outcome evidence.
GATEWAY TELEMETRY
ILLUSTRATIVE REFERENCE VIEWTIMEVIRTUAL KEYMODEL ALIASPROVIDEROUTCOME
14:32:08vk_finance_07fastanthropicSUCCESS
14:31:55vk_support_02reasoningopenaiFALLBACK
14:31:41vk_research_12summarizegeminiBLOCK
PRODUCT SPECIFICATION
One deployable. Two interfaces. Enforced module boundaries.
A Spring Modulith architecture keeps the operational path simple while separating access, routing, governance, and analytics responsibilities.
POST /admin/providersPOST /admin/deploymentsPOST /admin/keysPOST /admin/budgetsRegister vendors and model aliases, encrypt provider credentials, mint application keys, and configure scoped controls.
POST /v1/chat/completionsPOST /v1/responsesPOST /v1/messagesPOST /v1/embeddingsUse existing OpenAI or Anthropic-compatible clients with a broker-issued virtual key; streaming is supported.
iamVirtual keys and /v1 authentication
catalogProviders, deployments, encrypted credentials
directoryOrganizations, teams, and projects
providersOpenAI, Anthropic, Gemini, and Vertex adapters
routingLoad balancing, health, and fallback
gatewayOpenAI-compatible request pipeline
analyticsPostgres call audit and usage records
billingBudgets, spend, RPM, and TPM
guardrailsPII, secret, and denylist policy
siemOptional OpenSearch event projection
soarOptional reversible abuse containment
sharedEvents and common value types
ON-PREM FIRST / POSTGRES SYSTEM OF RECORD / OPTIONAL OPENSEARCH + VALKEY + KEYCLOAK
TRUST ARCHITECTURE
Built for security review, not security theater.
Map virtual-key access, encrypted provider credentials, guardrail decisions, and audit evidence to the controls your enterprise already operates.
SOC 2 control alignment
Architecture supports evidence collection for access, change, and monitoring controls.
ISO 27001 readiness
Map credential, access, policy, and audit practices to ISMS control objectives.
NIST AI RMF mapping
Operational visibility across governed model access, guardrails, usage, and provider outcomes.
Control-alignment statements describe product architecture and are not claims of EnvisionAI certification.
WORKS WITHOpenAIAnthropicGeminiVertex AIGroqOllama
TECHNICAL EVALUATION
Review the control path with your security and platform teams.
Map the architecture to your providers, application clients, governance scopes, data boundaries, and operating requirements.