search

LucienShui / huggingface-vscode-endpoint-server

starcoder server for huggingface-vscdoe custom endpoint
Apache License 2.0
166 stars 56 forks source link

support ReplitCode #6

Closed luckyfish0826 closed 1 year ago

luckyfish0826 commented 1 year ago

Hi Lucien, Thanks for your work. I saw your code support replitcode，yet seems it could not load the model as I tried.

I am still try to fix it, how I can work with you on the. Replit is also a good model.

Hope you can see this.

By the way, I currently encountered below error msg: ValueError: Unrecognized configuration class <class 'transformers_modules.replit.replit-code-v1-3b.a97a3fd23f03aa31b2a35a71b000ef67c60d0071.configuration_mpt.MPTConfig'> to build an AutoTokenizer.

luckyfish0826 commented 1 year ago

update: above error is because of the transformers version. 4.28.1 not working, 2.29.2 working.

thus fix: pip3 uninstall transformers && pip3 install transformers==4.29.2

luckyfish0826 commented 1 year ago

Oh I saw your requirements.txt specified the transformers version as 4.28.1, maybe for replit it need 4.29

luckyfish0826 commented 1 year ago

also for replit, it needs sentencepiece module

luckyfish0826 commented 1 year ago

server is up, but failed at first try:

curl -X POST http://localhost:8000/api/generate/ -d '{"inputs": "hellow_world()", "parameters": {"max_new_tokens": 64}}'

response: 500 Internal Server Error

stack trace: File "/home/notebook/code/group/huggingface-vscode-endpoint-server-main/generators.py", line 69, in generate params.pop('stop') KeyError: 'stop'

luckyfish0826 commented 1 year ago

update: just comment that line (params.pop('stop')) , it works well.