Open THU-mjx opened 9 months ago
Hope this issue is helpful https://github.com/THUDM/ChatGLM2-6B/issues/243.
when i add "trust_remode_code=True" in preprocessing/&postprocessing/1/model.py,
another error occurred : Failed to initialize Python stub: AttributeError: can‘t set attribute ’pad_token
I tried to fix it with this methods
The server could be built, but can't be used!
What branch do you use? If you don't use the latest main branch, please try it. Please remember updating the tensorrt_llm, too. Also, you need to rebuild the tensorrt_llm and the engine when you update the codes.
Using the latest tensorrt_llm and tensorrtllm_backend, I failed to launch the server with the above error:
I fixed this issue. The latest version of trtllm-backend have to manually set max_batch_size.
I have another question : how to set these values?
You can set 1 first.
Another error happened in preprocessing/postprocessing/model.py (I test several models, the errors are same!)
IN model.py, i just added 'trust_remote_code=True'
Please refer https://github.com/THUDM/ChatGLM3/issues/152.
A possible workaround is preventing assign the pad_token, and replace all pad_token by eos_token.
when i built a ChatGLM2-6B engine to launch the triton-server, an error occurred: Tokenizer class ChatGLMTokenizer does not exist