Open emeeks opened 10 months ago
Have the same problem. Input is a string with ~ 20.000 characters. My workaround is to .free()
the encoder and try it again.
EDIT: No that does not work. I just create a new encoder. If I .free()
the old one the new one has the same "null pointer passed to rust" error
I debugged this a bit and I can reproduce the error if I .free()
the encoding before calling encode
. I get the same error message:
const encoding = get_encoding(encoding)
encoding.free()
encoding.encode(content) // <-- throws here with "null pointer passed to rust"
Calling .free()
two times after another produces the same error:
const encoding = get_encoding(encoding)
encoding.free()
encoding.free() // <-- throws here with "null pointer passed to rust"
I get this error after I refactored my code. I had the issue that calling get_encoding
took 50 ms every time I called it. Calculating multiple messages took sometimes up to multiple seconds. So I started caching the encoding in a Map()
and reused it. Maybe some........
Oh my god 🤦♂️ while writing this I realized that I called .free()
in another method, too. This method used the cached encoding. So, well my fault :D @emeeks Maybe you have a similar issue?
Hi, I'm having the same issue. "null pointer passed to rust" for a regular string. Does anyone have any workarounds?
TikToken was working fine encoding and decoding tokens for me then without any changes (just demoing what was working before) it failed with this error:
From the console:
In this case,
fullContext
is just a long string that is not null or undefined. What's very strange is it was just working fine yesterday and now does this. I tried fresh installs and everything but any tips would be appreciated.