Arize Phoenix: open-source tracing and evals on OpenInference

Phoenix is Arize AI's open-source workbench for tracing and evaluating LLM applications and agents. Version 20.21.1 shipped on 12 October 2026. The server, the evals library, the OpenTelemetry helper and the client SDKs all release from one repository on their own version numbers.

Phoenix is one container that does four jobs:

• Collects traces. It accepts OpenTelemetry (OTLP) over HTTP at /v1/traces on port 6006 and over gRPC on port 4317, and serves the UI on 6006.
• Evaluates. A Python and TypeScript evals library runs LLM-as-a-judge and code evaluators over traces, datasets or any dataframe, and logs the results back onto spans as annotations. Evaluators can also live on the server and score every experiment automatically.
• Runs experiments. Datasets of examples, tasks run against them, scores compared run against run.
• Iterates prompts. Versioned, taggable prompts, a playground that compares prompts and models side by side, and span replay that reopens a real production call with different inputs.

In the stack it is the observability and evaluation layer, and the most self-contained one: SQLite by default, Postgres for teams, nothing else to run.

Phoenix speaks OpenInference, Arize's open semantic conventions for AI traces (llm.*, input.value, openinference.span.kind), with auto-instrumentors for most frameworks and providers. Spans that follow the OpenTelemetry GenAI conventions (gen_ai.*) also arrive, but the docs say they do not get the same UI treatment as OpenInference spans.

The server and the evals library use the Elastic License 2.0. You may self-host it free on your own infrastructure with no feature gates. You may not offer it to third parties as a hosted or managed service, or circumvent licence-key functionality. The instrumentation (OpenInference), arize-phoenix-otel and the client SDKs are Apache-2.0. Arize sells a separate managed platform, Arize AX, built on the same standards.

Pick Phoenix when:

• You want everything free and self-hosted: tracing, evals, datasets, experiments, prompt management and the playground, with no paid tier inside Phoenix.
• Data cannot leave your network. Phoenix runs air-gapped: with PHOENIX_ALLOW_EXTERNAL_RESOURCES=false it makes no public-internet requests of its own (fonts, telemetry, sandbox downloads), and evals call only the model endpoints you configure.
• You work in notebooks and scripts. uvx arize-phoenix serve gives you a full instance in seconds, and evals run over pandas dataframes.
• Your stack is covered by OpenInference instrumentors: OpenAI, Anthropic, Bedrock, LangChain, LlamaIndex, DSPy, the Claude Agent SDK, the OpenAI Agents SDK, Strands and many more.

Look elsewhere when:

• You plan to offer it to customers as a hosted service: ELv2 forbids that.
• You need continuous evals on production traffic with alerting, monitors and dashboards for non-engineers. Arize documents those as Arize AX features.
• Many teams need isolation inside one deployment. One Phoenix instance is one tenant; the docs say to run several instances, while Langfuse separates organizations and projects in one deployment.

Spans and conventions. Every operation is an OpenTelemetry span with an OpenInference span kind: LLM, CHAIN, TOOL, AGENT, RETRIEVER, EMBEDDING, RERANKER, GUARDRAIL, EVALUATOR, PROMPT, DECISION or UNKNOWN. LLM spans carry input and output messages, token counts and model parameters. Traces live in projects; spans that share a session.id form a session, and user.id attributes the work to a user.

Setup is one call. phoenix.otel.register() builds the OpenTelemetry tracer provider and exporter from PHOENIX_COLLECTOR_ENDPOINT and PHOENIX_API_KEY, and auto_instrument=True turns on every installed OpenInference instrumentor. Context managers (using_session, using_user, using_metadata, using_tags, using_attributes) stamp attributes onto every span created inside them. In Python, register() defaults to batch=False, which exports each span as it ends; production code should pass batch=True.

Evals are structured. LLM evaluators call the judge through tool calling, so the judge must pick one of your labels instead of writing free text. Executors retry on rate limits and adjust concurrency, and every evaluator run is itself traced to a Phoenix project, so you can inspect what the judge saw.

Experiments run a task function over a dataset, apply evaluators, and store every output and score so the UI can diff runs. Storage is SQLite under ~/.phoenix (or PHOENIX_WORKING_DIR) by default, or Postgres 14 and later via PHOENIX_SQL_DATABASE_URL. Retention defaults to forever (PHOENIX_DEFAULT_RETENTION_POLICY_DAYS=0) unless you set a policy.