Open chimezie opened 9 months ago
Finally got a moment to look into this. I've never used conda, but I can't reproduce on 3.11 using regular virtual env. Strangely, I don't get any output for your "How old are the Igbo people?" question using openhermes-2.5-mistral-7b-16k.Q5 but when I change the question to "Where do the Igbo people live?" I get a good answer.
Just reading the error you're getting it seems to think there is an unpicklable object when crossing the fork boundary, but I can't think what that might be, and usually the error is clearer in terms of what can't be pickled.
I originally got this error in my code while iterating over the results gathered asynchronously from a call to oapi.wrap_for_multiproc(prompt_to_chat('.. prompt ..'), **model_params). I was able to successfully reproduce it by running demo/chat_web_selects.py from the main branch:
This is how it was run in my command line
I'm running Python 3.10.12 in a Conda environment with OgbujiPT built from the main branch