-
### Description
When i wanted to run the basic_quant_quik.py, i got a error with "ModuleNotFoundError: No module named 'mixlib'"
### System Info
```
(llama_env_py39) liyunjie@icrsmp-nodeg:~/MIXQ…
-
Hi all, I am trying to fine-tune models in extremely long contexts.
I've tested the training setup below, and I managed to finetune:
- llama3.1-1B with a max_sequence_length of 128 * 1024 tokens
…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### How would you like to use vllm
Hi
I want to attach lora using docker command
docker run --runtime nv…
-
### System Info
I am finetuning Llama3-8b-Instruct model. Here is the Jupyter Notebook of the steps, i followed to perform the finetuning:
https://gitlab.com/keerti4p/llama3-8b-instruct-finetune/-…
-
During the use of LoRA fine-tuning, everything was normal, but the following issue arose during full-scale fine-tuning.
I use the following script for full fine-tuning :
```shell
#!/bin/bash
N…
-
**Command: tune run lora_finetune_single_device --config llama3_1/8B_lora_single_device**
**Output**:
```
INFO:torchtune.utils._logging:Running LoRAFinetuneRecipeSingleDevice with resolved config:…
-
### Proposal to improve performance
The spec dec performance of Eagleis worse than expected as shown below:
Model: [meta-llama/Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Llam…
-
I find Microsoft's Phi 3.5 vision instruct performs much better than Florence 2. Since it's an instruct model, it also has the benefit of taking text instruction as input to help describing the images…
-
Hi, Thanks for your wonderful work.
I am struggling using my lora tuned model.
I conducted following steps
1. finetuning with lora
- Undi95/Meta-Llama-3-8B-Instruct-hf model base
- llama3 …
-
## Summary
I am considering integrating sLM into the WelsonJS project. The leading candidate currently under review is Phi-3.5-mini.
If there are any updates on this matter, I will comment them in…