traceloop / openllmetry-js

Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry
https://www.traceloop.com/openllmetry
Apache License 2.0
261 stars 26 forks source link

Bug: OpenAI parse method crashes the process #462

Open Kovbo opened 2 weeks ago

Kovbo commented 2 weeks ago

Along with structured outputs, OpenAI introduced the parse method openai.beta.chat.completions.parse(), which returns a type-safe completion based on a provided zod schema.

Calling this method with openLLMetry initialized, results in the following error:

An error occurred: TypeError: this._client.chat.completions.create(...)._thenUnwrap is not a function
    at Completions.parse (.../node_modules/.pnpm/openai@4.62.1_zod@3.23.8/node_modules/openai/src/resources/beta/chat/completions.ts:77:8)

It captures the trace but crashes the whole process.

Minimal reproducible example:

import traceloop from '@traceloop/node-server-sdk';
import OpenAI from 'openai';
import { z } from 'zod';
import { zodResponseFormat } from 'openai/helpers/zod';

const openai = new OpenAI();

traceloop.initialize({ disableBatch: true });

async function main() {
  const completion = await openai.beta.chat.completions.parse({
    messages: [
      {
        role: 'system',
        content:
          'You are a helpful math tutor. Guide the user through the solution step by step.',
      },
      { role: 'user', content: 'how can I solve 8x + 7 = -23' },
    ],
    model: 'gpt-4o-2024-08-06',
    response_format: zodResponseFormat(
      z.object({
        final_answer: z.string(),
      }),
      'math_answer'
    ),
  });

  console.log(completion);
}

main();
nirga commented 2 weeks ago

Thanks for reporting @Kovbo! Mind sharing the OpenAI version you're using so we can reproduce?

Kovbo commented 2 weeks ago

It happens on the latest OpenAI version: 4.59.0 - 4.63.0. I added a minimal reproducible code example.