Skip to main content

​
Telemetry Playbook

Agnitra treats telemetry as a first-class artifact. Every optimization captures before/after metrics so engineering, infra, and finance teams agree on the impact of a rollout. This guide explains how telemetry is produced and how to route it to your observability stack.

​
What the CLI & SDK Emit

Both CLI and SDK expose the same data so you can automate pipelines or drive notebooks without format drift.

​
Routing Telemetry

  1. File drops — agnitra optimize --telemetry-out telemetry.json writes a structured JSON file. Persist it to S3, GCS, or your artifact store.
  2. Programmatic export — Use agnitra.telemetry_collector and agnitra.telemetry.usage_meter helpers to push directly to HTTP, Kafka, or Snowflake.
  3. Marketplace dispatchers — Extras like agnitra[marketplace] register AWS, GCP, and Stripe exporters (StripeUsageDispatcher, AwsMarketplaceDispatcher) that run asynchronously after each optimization.
Telemetry payloads contain deterministic keys for project_id, model_name, and timestamps so you can join them in downstream jobs.

​
Dashboards & Alerting

  • agnitra-dashboard renders telemetry bundles locally, highlighting speedups, GPU hour savings, and license compliance.
  • Push aggregated snapshots into your metrics system (Prometheus, Datadog, Grafana) to track optimization coverage and ROI over time.
  • Alert when expected_speedup_pct drops below target or when usage_event.status != "delivered" to catch marketplace backoffs.

​
Best Practices

  • Store raw telemetry before aggregating so you can retroactively re-price or inspect kernels.
  • Sign usage events before dispatching to marketplaces to meet compliance requirements.
  • Attach job_metadata (CLI flag) or metadata (SDK argument) to correlate runs with CI pipelines, pull requests, or customer tenants.
  • Rotate AGNITRA_API_KEY and audit outbound webhook targets to avoid leaking telemetry to untrusted endpoints.