-
Segmentation fault when using the dev container to train the llm finetune recipe:
```
nemo.collections.llm.api.finetune/0 [NeMo I 2024-08-28 07:01:29 strategies:244] Fixing mis-match between ddp-conf…
-
Hi @VikParuchuri ,
Great project, I have been using it and it works for almost every use case of mine.
However I am now having some very complex documents and I want to finetune the layout detection…
-
Thank you very much for your excellent work!
I would like to ask you a question. I currently have two ideas, one is to finetune a multispeaker model and another one is to finetune the model alone.
T…
-
Could you please provide your code of experiments using LoRA?
-
is there any end-to-end script for finetuning procedure?
-
- Llama : https://llama.meta.com/docs/how-to-guides/fine-tuning
- Quantization : 실수형 변수(float)를 정수형 변수(int)로 변환
- 효과
- 모델 사이즈 축소
- 모델 연산량 감소
- 효율적인 하드웨어 사용
- Parameter Efficient Fi…
-
Great work! This is really impressive. Is there any chance to release training or fine-tuning (LoRA) code? Appreciate it if possible, thanks!
-
Noticed that `adapter_config` is created in `lora_finetune_fsdp2.py`: https://github.com/pytorch/torchtune/blob/277fbf881933127671e85fc4b5d39d80746a7868/recipes/dev/lora_finetune_fsdp2.py#L569
but …
-
Hello,
How should I run the LoRA finetuning with an own pretrained ViT and pretrained adapter? Do I use `--pretrain_mllm` to give the full pretrained model checkpoint which I get from `pretrain.sh…
-
### 🚀 The feature, motivation and pitch
The fine-tuning with only FSDP works well and sharded checkpoints are saved as `__0_*.distcp, .metadata, and train_params.yam`l. I can see the loss drop reas…