Potential 'unused_parameters' Error During Model Tuning with DDP.

X-LANCE / SLAM-LLM

Speech, Language, Audio, Music Processing with Large Language Model

MIT License

579 stars 52 forks source link

Closed Exgc closed 7 months ago

Exgc commented 8 months ago

Problem: When fine-tuning a model, encountering an 'unused_parameters' error due to unutilized parameters. Source of the Problem: A typo located at finetune.py: https://github.com/ddlBoJack/SLAM-LLM/blob/1e1c4e89068cb040cfcb606a0567c5eb4e869f0e/src/llama_recipes/pipeline/finetune.py#L183 Solution: Rectified the typo and incorporated the relevant config in .yaml file.

ddlBoJack commented 7 months ago

Please resolve the conflicts and re-PR