Documentation

Serve your first model in 10 minutes.

Concept guides, copy-paste tutorials, and exhaustive reference for the inference engine, policy engine, and SDK.

Get started

Quickstart

Install the CLI, authenticate, and serve your first NIM model in under 10 minutes.

Read the quickstart

Build

Model Serving DSL

Compose Triton, TensorRT-LLM, and NIM serving graphs with shared memory.

Workflow guide

Govern

Policy engine

Declarative policies that evaluate before every tool call. Spend caps, escalation, two-person.

Policy reference

Operate

Observability

DCGM GPU telemetry, token throughput, GPU utilization, cost attribution.

Observability docs

Connect

Integrations & MCP

Wire up NVIDIA NIM, Triton, Salesforce, or any MCP server.

Connector catalog

Trust

Security & compliance

SOC 2, ISO 27001, HIPAA, FedRAMP, EU AI Act conformity guides.

Trust center

Quickstart

Hello, model.

Install the CLI: npm i -g @mandarum/cli
Authenticate: mandarum login
Scaffold: mandarum init my-nim-pipeline
Run: mandarum serve --model llama-3.1-70b-instruct · view GPU metrics in the dashboard

// my-nim-pipeline/index.ts
import { pipeline, model, policy } from "@mandarum/sdk";

export default pipeline({
  name: "nim-inference",
  policy: "production.guardrails.v1",
  steps: [
    model.nim({ model: "meta/llama-3.1-70b-instruct" }),
    model.verify({ evalSet: "accuracy/golden-v3" }),
    model.route({ fallback: "mistralai/mixtral-8x7b" })
  ]
});