search

aespinilla / Tiktoken

Openai's tiktoken implementation written in Swift
MIT License
53 stars 15 forks source link

Crash #4

Open twodayslate opened 11 months ago

twodayslate commented 11 months ago

P_H6tZDEBzZsR2oaxFFQn.xccrashpoint.zip

twodayslate commented 11 months ago

Here is another BAExrjYqL9nOLQdVlItnJp.xccrashpoint.zip

twodayslate commented 11 months ago
Screenshot 2023-11-06 at 3 18 51 AM
aespinilla commented 11 months ago

Could you give me the text which you would like to encode?

Thanks

twodayslate commented 11 months ago

One happened as I was tying "Hello". The other I don't know since it wasn't my device. I do not have reproducible steps.

LeaderBoy commented 11 months ago

I have the same problem many times, is it possible to fix it. Thanks a lot

aespinilla commented 11 months ago

@LeaderBoy could you tell me which encoding and prompt are you using? Thanks

aespinilla commented 11 months ago

Hello @twodayslate @LeaderBoy , are you trying to encode with gpt3? If so, try the gpt2 value instead gpt3 is the same encoder. I will fix it on next release.

twodayslate commented 11 months ago

Was using gpt-4 I believe when I got the crash.

LeaderBoy commented 10 months ago

@LeaderBoy could you tell me which encoding and prompt are you using? Thanks

Hello @twodayslate @LeaderBoy , are you trying to encode with gpt3? If so, try the gpt2 value instead gpt3 is the same encoder. I will fix it on next release.

I'm using gpt-4,I don't know which prompts are causing this, these crashes are what I collected from users.