search

art-from-the-machine / Mantella

Mantella is a Skyrim and Fallout 4 mod which allows you to naturally speak to NPCs using Whisper (speech-to-text), LLMs (text generation), and xVASynth / XTTS (text-to-speech).
https://art-from-the-machine.github.io/Mantella/
GNU Affero General Public License v3.0
184 stars 44 forks source link

ValueError: Unknown encoding cl100k_base #352

Open overtimepog opened 2 months ago

overtimepog commented 2 months ago

Mantella currently running for SkyrimVR (C:\Steam\steamapps\common\SkyrimVR). Mantella mod located in C:\skyrim\MadGod\mods\Mantella - Bring NPCs to Life with AI Running Mantella with 'gpt-3.5-turbo'. The language model can be changed in MantellaSoftware/config.ini 23:37:06.756 ERROR: Error loading model. If you are using an alternative to OpenAI, please find the setting llm_api in MantellaSoftware/config.ini and follow the instructions to change this setting 23:37:06.757 ERROR: Traceback (most recent call last): File "src\llm\openai_client.py", line 106, in init File "tiktoken\model.py", line 75, in encoding_for_model File "tiktoken\registry.py", line 60, in get_encoding ValueError: Unknown encoding cl100k_base

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 21, in File "src\setup.py", line 113, in initialise File "src\llm\openai_client.py", line 110, in init File "tiktoken\model.py", line 75, in encoding_for_model File "tiktoken\registry.py", line 60, in get_encoding ValueError: Unknown encoding cl100k_base

I have tried everything, OpenRouter and OpenAI cause this no matter what, I have no idea

Leidtier commented 2 months ago

Looks like your tiktoken registry doesn't know the encoding cl100k_base for gpt-3.5-turbo. Similar to this issue over here https://github.com/openai/tiktoken/issues/80. As you are the very first person with this problem I expect there to be a specific issue with your system somehow... But I have no clue what it could be exactly.

Have you tried using a different model than gpt-3.5-turbo? Like the free ones from OpenRouter?

art-from-the-machine commented 2 months ago

@Leidtier these hidden imports mentioned in the linked issue are already included in the exe, so I'm not sure what else could be causing this!

danniesim commented 2 months ago

I am having the same issue. I am using Mad God Overhaul via Mode Organizer 2 (MO2). The problem seems to be caused by running Mantella in MO2's Virtual File System, as mantella.exe runs fine outside it.

Leidtier commented 2 months ago

@danniesim thanks for the report and the investigation. We had a couple of people installing MGO lately and they didn't report a problem like this... After you ran Mantella.exe outside of the MO2-VFS and it installed everything, have you been able to run the MGO one without issues?