# OpenAI Monitoring with OpenTelemetry

This guide walks you through setting up OpenAI monitoring using OpenTelemetry and SigNoz. You will instrument your OpenAI applications to capture traces, logs, and metrics and visualize them in real time in SigNoz.

OpenAI monitoring with SigNoz gives you visibility into:

- **Token usage** — track input and output tokens per request to understand cost drivers
- **Model latency** — measure response times across models and request types
- **Error rates** — detect API failures, rate limit errors, and timeouts before they impact users
- **Request traces** — follow individual LLM calls through your full application stack

## Requirements

- Python 3.8 or newer
- OpenAI Python library (`openai >= 1.0.0`)
- Valid OpenAI API key
- SigNoz setup (choose one):
  - [SigNoz Cloud account](/content/teams/index.html) with an active ingestion key
  - Self-hosted SigNoz instance

## Setup

### Step 1. Create a virtual environment

```bash
python3 -m venv .venv
source .venv/bin/activate
```

### Step 2. Install the OpenTelemetry dependencies

```bash
pip install opentelemetry-distro~=0.51b0
pip install opentelemetry-exporter-otlp~=1.30.0
pip install opentelemetry-instrumentation-openai-v2
```

### Step 3. Add automatic instrumentation

```bash
opentelemetry-bootstrap --action=install
```

### Step 4. Run your application

```bash
OTEL_SERVICE_NAME=<service-name> \
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.us.signoz.cloud:443" \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>" \
OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true \
OPENAI_API_KEY=<your-openai-key> \
opentelemetry-instrument <your_run_command>
```

**Environment Variables Explained:**

- `OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED` 
  - Enables automatic logging instrumentation
- `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT` 
  - Captures prompts and completions as logs
- `us`: Your [SigNoz Cloud region](/content/docs/ingestion/signoz-cloud/overview/#endpoint/index.html)
- `<your-ingestion-key>`: Your SigNoz [ingestion key](/content/docs/ingestion/signoz-cloud/keys/index.html)
- Replace `<service-name>` with the name of your service
- Replace `<your-openai-key>` with your OpenAI API key

## Requirements for Java

- Java 8 or newer
- OpenAI Java SDK (`openai-java >= 1.1.0`)
- Valid OpenAI API key
- SigNoz setup (choose one):
  - [SigNoz Cloud account](/content/teams/index.html) with an active ingestion key
  - Self-hosted SigNoz instance

## Setup for Java

### Step 1. Add dependencies

**For Gradle (build.gradle.kts):**

```gradle
dependencies {
    implementation("com.openai:openai-java:3.6.1")
    implementation("io.github.cdimascio:dotenv-java:3.0.0")
    implementation("io.opentelemetry:opentelemetry-api:1.54.1")
    implementation("io.opentelemetry:opentelemetry-sdk:1.54.1")
    implementation("io.opentelemetry:opentelemetry-sdk-trace:1.54.1")
    implementation("io.opentelemetry:opentelemetry-sdk-metrics:1.54.1")
    implementation("io.opentelemetry:opentelemetry-sdk-logs:1.54.1")
    implementation("io.opentelemetry:opentelemetry-exporter-otlp:1.54.1")
    implementation("io.opentelemetry:opentelemetry-semconv:1.27.0-alpha")
    implementation(platform("io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom-alpha:2.20.1-alpha"))
    implementation("io.opentelemetry.instrumentation:opentelemetry-openai-java-1.1")
}
```

**For Maven (pom.xml):**

```xml
<dependencies>
    <dependency>
        <groupId>com.openai</groupId>
        <artifactId>openai-java</artifactId>
        <version>3.6.1</version>
    </dependency>
    <dependency>
        <groupId>io.github.cdimascio</groupId>
        <artifactId>dotenv-java</artifactId>
        <version>3.0.0</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-api</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-sdk</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-sdk-trace</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-sdk-metrics</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-sdk-logs</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-exporter-otlp</artifactId>
        <version>1.54.1</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry</groupId>
        <artifactId>opentelemetry-semconv</artifactId>
        <version>1.27.0-alpha</version>
    </dependency>
    <dependency>
        <groupId>io.opentelemetry.instrumentation</groupId>
        <artifactId>opentelemetry-openai-java-1.1</artifactId>
        <version>2.20.1-alpha</version>
    </dependency>
</dependencies>
```

### Step 2. Import the necessary modules in your Java application

**OpenTelemetry:**

```java
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.instrumentation.openai.v1_1.OpenAITelemetry;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.semconv.ServiceAttributes;
```

**Traces:**

```java
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter;
```

**Logs:**

```java
import io.opentelemetry.sdk.logs.SdkLoggerProvider;
import io.opentelemetry.sdk.logs.export.BatchLogRecordProcessor;
import io.opentelemetry.exporter.otlp.http.logs.OtlpHttpLogRecordExporter;
```

**Metrics:**

```java
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.exporter.otlp.http.metrics.OtlpHttpMetricExporter;
```

### Step 3. Setup Traces

```java
import io.github.cdimascio.dotenv.Dotenv;

Dotenv dotenv = Dotenv.load();

String serviceName = "<service_name>";

Resource resource = Resource.getDefault()
            .toBuilder()
            .put(ServiceAttributes.SERVICE_NAME, serviceName)
            .build();

OtlpHttpSpanExporter otlpSpanExporter = OtlpHttpSpanExporter.builder()
            .setEndpoint(System.getenv("OTEL_EXPORTER_TRACES_ENDPOINT"))
            .addHeader("signoz-ingestion-key", System.getenv("SIGNOZ_INGESTION_KEY"))
            .build();

SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
            .setResource(resource)
            .addSpanProcessor(BatchSpanProcessor.builder(otlpSpanExporter).build())
            .build();
```

### Step 4. Setup Logs

```java
OtlpHttpLogRecordExporter otlpLogExporter = OtlpHttpLogRecordExporter.builder()
            .setEndpoint(System.getenv("OTEL_EXPORTER_LOGS_ENDPOINT"))
            .addHeader("signoz-ingestion-key", System.getenv("SIGNOZ_INGESTION_KEY"))
            .build();

SdkLoggerProvider loggerProvider = SdkLoggerProvider.builder()
            .setResource(resource)
            .addLogRecordProcessor(BatchLogRecordProcessor.builder(otlpLogExporter).build())
            .build();
```

### Step 5. Setup Metrics

```java
OtlpHttpMetricExporter otlpMetricExporter = OtlpHttpMetricExporter.builder()
            .setEndpoint(System.getenv("OTEL_EXPORTER_METRICS_ENDPOINT"))
            .addHeader("signoz-ingestion-key", System.getenv("SIGNOZ_INGESTION_KEY"))
            .build();

SdkMeterProvider meterProvider = SdkMeterProvider.builder()
            .setResource(resource)
            .registerMetricReader(PeriodicMetricReader.builder(otlpMetricExporter).build())
            .build();
```

### Step 6. Setup OpenTelemetry SDK with your created providers

```java
OpenTelemetrySdk openTelemetry = OpenTelemetrySdk.builder()
            .setTracerProvider(tracerProvider)
            .setMeterProvider(meterProvider)
            .setLoggerProvider(loggerProvider)
            .build();
```

### Step 7. Run an Example using OpenAI Instrumentor

```java
OpenAIClient client = OpenAIOkHttpClient.builder()
            .apiKey(System.getenv("OPENAI_API_KEY"))
            .build();

OpenAIClient otelClient = OpenAITelemetry.builder(openTelemetry)
            .build()
            .wrap(client);

ChatCompletionCreateParams params = ChatCompletionCreateParams.builder()
            .addUserMessage("What is SigNoz?")
            .model("gpt-4.1")
            .build();

ChatCompletion chatCompletion = otelClient.chat().completions().create(params);
System.out.println(chatCompletion);

// Force flush with timeout to ensure all telemetry is exported before shutdown
sdk.getSdkTracerProvider().forceFlush().join(10, java.util.concurrent.TimeUnit.SECONDS);
sdk.getSdkMeterProvider().forceFlush().join(10, java.util.concurrent.TimeUnit.SECONDS);
sdk.getSdkLoggerProvider().forceFlush().join(10, java.util.concurrent.TimeUnit.SECONDS);

sdk.getSdkTracerProvider().shutdown();
sdk.getSdkMeterProvider().shutdown();
sdk.getSdkLoggerProvider().shutdown();
```

## What Does OpenAI Monitoring Capture?

This instrumentation captures comprehensive telemetry data for your OpenAI applications:

### Traces

- **Spans** for each OpenAI API call with timing information
- **Operation details** like model name, max tokens etc.
- **Token usage** including input and output
- **Request and response metadata** such as finish reason and model used

### Logs

- **Structured logs** for each API call when `OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true` is set
- **Message content logs** (prompts and completions) when `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` is enabled
- **Error logs** for failed API calls with detailed error information and stack traces
- **Performance logs** showing request duration and timing information

### Metrics

- **Duration metrics** showing how long OpenAI calls take
- **Token usage metrics** tracking consumption over time
- **Request rate metrics** showing API call frequency
- **Error rate metrics** for monitoring API failures

## Validating Your OpenAI Monitoring Setup

**Validate your traces, logs, and metrics in SigNoz:**

- Trigger OpenAI API calls in your app. Make several API calls to generate some data. Then, wait for some time.
- In SigNoz, open the `Services` tab. Hit the `Refresh` button on the top right corner, and your application should appear in the list of `Applications`.
- Go to the `Traces` tab, and apply relevant filters to see your application's traces.
- Check the `Logs` tab to see captured logs from your OpenAI calls.
- Visit the `Metrics` tab to view token usage and performance metrics.