search

srcnalt / OpenAI-Unity

An unofficial OpenAI Unity Package that aims to help you use OpenAI API directly in Unity Game engine.
MIT License
694 stars 153 forks source link

Chat GPT in VR working? #117

Open xmaymo opened 7 months ago

xmaymo commented 7 months ago

Hey guys,

Did anyone tried the project in VR?

For me everything works fine from the Unity editor but not on the Meta Quest build. I dit a test with the "microphone" permisions and it work well on the meta quest. And the openAI credentials are on the code, so I guess there's another problem. Basically the var response = await openAI.CreateChatCompletion(request); it is not called.

Is there something I need to change to make it work for VR? Any help would be appreciate it!

Thanks!

Firyfly commented 6 months ago

Sadly i am stuck at the exact same problem and havent been able to find a working solution.

I noticed that node apparently got updated, through which a configuration object is used instead of the private key, but i havent been able to make it work for now. @srcnalt Maybe you have knowledge if the node update (if i saw it correctly) has impacted the working of this function.

So definitely would love to hear feedback or ideas from other people. Everything is running fine for me in the unity editor, but as soon as i build it for android and sideload it on the quest 2, it stops at that function.

Hope there will be found a solution soon. ~Greetings from Germany

xmaymo commented 6 months ago

Sadly i am stuck at the exact same problem and havent been able to find a working solution.

I noticed that node apparently got updated, through which a configuration object is used instead of the private key, but i havent been able to make it work for now. @srcnalt Maybe you have knowledge if the node update (if i saw it correctly) has impacted the working of this function.

So definitely would love to hear feedback or ideas from other people. Everything is running fine for me in the unity editor, but as soon as i build it for android and sideload it on the quest 2, it stops at that function.

Hope there will be found a solution soon. ~Greetings from Germany

Hey Firyfly,

I figured it out after reading some other people's similar problems, the error is caused due the "Internet premission". The trick is to be sure to disable "Force Remove Internet Permission" on Project Settings -> XR Plug-In Management -> Open XR -> Meta Quest Support, before making the build. There was a bug in the old OpenXR plugin were that permision kept reseting to true automatically. With the newest OpenXR version 1.10.0 seems that they fixed that problem.

I am using Unity 2022.3.11f1 but it should work with the newest versions as well.

Kind regards from Barcelona!

StevenLe2012 commented 6 months ago

Have you implemented Whisper Speech to Text with the GPT model in VR @xmaymo? For me, I did so and it works well on Quest Link; however, when I go build it to Quest, I am unable to get the microphone to even pick up that I'm speaking. Super confused on why this is. I chose every single possible option from the Microphone Dropdown list, but none of them work on my Quest Build.