WhisperClient throwing NullReferenceException when calling TranscribeAudio, and ArgumentException: boundary when calling TranscribeAudioAsync

[paulorenanmelo]

Service

OpenAI

Describe the bug

I'm having the error below when trying to use the AudioClient whisper-1, as described in your readme.

Here's my code (used in Unity 2022.3 LTS):

OpenAIClient openAIClient = new OpenAIClient(new ApiKeyCredential("sk-proj-key"));
var assistantClient = openAIClient.GetAssistantClient();
var assistant = assistantClient.GetAssistant("asst_id").Value;
Debug.Log(assistant.Name);
AudioClient ttsClient = openAIClient.GetAudioClient("tts-1");
AudioClient whisperClient = openAIClient.GetAudioClient("whisper-1");

string audioFilePath = Path.Combine(Application.streamingAssetsPath, "test.mp3");
AudioTranscriptionOptions options = new()
{
    ResponseFormat = AudioTranscriptionFormat.Verbose,
    TimestampGranularities = AudioTimestampGranularities.Word | AudioTimestampGranularities.Segment,
};

And below is the stack trace of the error in AudioTranscription transcription = whisperClient.TranscribeAudio(audioFilePath, options);.

NullReferenceException: Object reference not set to an instance of an object System.Net.Http.MultipartContent.Dispose (System.Boolean disposing) (at :0) System.Net.Http.HttpContent.Dispose () (at :0) OpenAI.MultipartFormDataBinaryContent.Dispose () (at <84f990b59862492894fc2b32532747bf>:0) OpenAI.Audio.AudioClient.TranscribeAudio (System.IO.Stream audio, System.String audioFilename, OpenAI.Audio.AudioTranscriptionOptions options, System.Threading.CancellationToken cancellationToken) (at <84f990b59862492894fc2b32532747bf>:0) OpenAI.Audio.AudioClient.TranscribeAudio (System.String audioFilePath, OpenAI.Audio.AudioTranscriptionOptions options) (at <84f990b59862492894fc2b32532747bf>:0)

I tried alternatively to do it using async, with the snippet below

var transcription = whisperClient.TranscribeAudioAsync(audioFilePath, options);
while (!transcription.IsFaulted && !transcription.IsCanceled && !transcription.IsCompleted && !transcription.IsCompletedSuccessfully)
{
    yield return new WaitForEndOfFrame();
}
Debug.Log(transcription.Status);
Debug.Log(transcription.Exception);
Debug.Log(transcription.Result);

The status comes up as Failed, and below is the stack trace of the error in Debug.Log(transcription.Result);

ArgumentException: boundary System.Net.Http.MultipartContent..ctor (System.String subtype, System.String boundary) (at :0) System.Net.Http.MultipartFormDataContent..ctor (System.String boundary) (at :0) OpenAI.MultipartFormDataBinaryContent..ctor () (at <84f990b59862492894fc2b32532747bf>:0) OpenAI.Audio.AudioTranscriptionOptions.ToMultipartContent (System.IO.Stream audio, System.String audioFilename) (at <84f990b59862492894fc2b32532747bf>:0) OpenAI.Audio.AudioClient.TranscribeAudioAsync (System.IO.Stream audio, System.String audioFilename, OpenAI.Audio.AudioTranscriptionOptions options, System.Threading.CancellationToken cancellationToken) (at <84f990b59862492894fc2b32532747bf>:0) OpenAI.Audio.AudioClient.TranscribeAudioAsync (System.String audioFilePath, OpenAI.Audio.AudioTranscriptionOptions options) (at <84f990b59862492894fc2b32532747bf>:0) Rethrow as AggregateException: One or more errors occurred. (boundary) System.Threading.Tasks.Task.ThrowIfExceptional (System.Boolean includeTaskCanceledExceptions) (at :0) System.Threading.Tasks.Task1[TResult].GetResultCore (System.Boolean waitCompletionNotification) (at <b11ba2a8fbf24f219f7cc98532a11304>:0) System.Threading.Tasks.Task1[TResult].get_Result () (at :0)

Steps to reproduce

1. Create a unity project
2. Import OpenAI.dll into Plugins folder
3. Import all dependencies from nuget.org into a subfolder Dependencies inside Plugins folder (
4. Create a monobehaviour script, add it to the camera on the scene hierarchy (create a camera if none is there)
5. Edit script adding relevant snippet inside Start() method, adding needed "using" statements
6. Save changes, and click "Play" button, and watch console for logs

Code snippets

No response

OS

Windows 10

.NET version

.NET Standard 2.1 / .NET Framework 4.8 (tried both)

Library version

2.0.0 latest

[paulorenanmelo]

To replicate the issue:

1. Open an empty unity project using unity 2022.3 (LTS)
2. Click Assets > Import Package > Custom Package, and select the file from link below
3. In Project view, double click to open Scenes/TestChatGPT
4. In Hierarchy view, click "TestChatGPT" object, then in the Inspector view, double click the grayed out script "TestChatGPT" to open the script in visual studio (or whatever you have setup as IDE). Put your API key and Assistant ID, and save the file
5. Back in Unity, open the Console view, and click Play at the center top, the error will show in the console

https://drive.google.com/file/d/1aydM-ihuAA-bFYZaMpk3aKy3-PvcID2J/view?usp=sharing