-
Sorry for a newb question, I don't find an answer. I succeeded in launching the server with unquantised Mistral7B:
```
python3 -m sglang.launch_server --model-path mistralai/Mistral-7B-Instruct-v0.2…
-
Thanks for your awesome work.
[swift](https://github.com/modelscope/swift) now supports inference, training of InternVL-Chat-V1.5 model
For more information, please refer to our document
- [Eng…
-
### System Info
peft==0.4.0.dev0
I'm not sure if this should be a bug report, so sorry if this is not convenient.
According to the `save_pretrained`method docstring, this saves the adapter model…
azayz updated
5 months ago
-
### System Info
Cuda : 12.1
OS : Windows x64
pip : 24.0
python : 3.10.10
transformers : 4.40.0
bitsandbytes: 0.43.1
### Who can help?
Hey, there @younesbelkada , @amyeroberts I am getting…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
按照 huggingface 的 README 启动服务:…
-
### System Info
Hi,
I am fine tuning Mistral7b model. I am getting long automated text generation using the fine tuned model. I have kept the eos_token=True. Can someone please tell me how to add a…
-
Using the latest commit d3184ec, I was able to make my own 4bpw quant of dbrx-instruct. I am running into problems trying to load the model in text-generation-webui (using that same commit of exllamav…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
### Describe the bug
![e341993d05015208d204…
xxg98 updated
4 months ago
-
Hello,
I woul like to use pipeline serialization. Below is my code for the llm
```
llm:
init_parameters:
huggingface_pipeline_kwargs:
model: mistralai/Mixtral-8x7B-Instru…
-
I tried to apply 4-bit training on Lora for EsmModel. However, there has been an error specifically for 4-bit training. The error disappears perfectly once `load_in_4bit=True` is commented out.
Cod…