search

alibaba / Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
Apache License 2.0
674 stars 94 forks source link

微调llama时遇到的的bug #166

Closed Emperorizzis closed 5 months ago

Emperorizzis commented 6 months ago

您好，我在运行run_finetune_megatron_llama.sh时遇到了下面的bug：

Traceback (most recent call last):
  File "finetune_megatron_llama.py", line 5, in <module>
    from megatron_patch.data.finetune_dataset import LLamaDataset
ModuleNotFoundError: No module named 'megatron_patch.data.finetune_dataset'

好像与issue 119是一样的问题？

lwmlyy commented 5 months ago

您好，这个是由于llama模型的训练链路代码未更新导致。这个我们近期更新下。

建议使用llama2文件夹下的run_finetune_megatron_llama_withGA.sh进行模型微调，llama2效果更佳。

jerryli1981 commented 5 months ago

您好，我在运行run_finetune_megatron_llama.sh时遇到了下面的bug：

Traceback (most recent call last):
  File "finetune_megatron_llama.py", line 5, in <module>
    from megatron_patch.data.finetune_dataset import LLamaDataset
ModuleNotFoundError: No module named 'megatron_patch.data.finetune_dataset'

好像与issue 119是一样的问题？

您好，请先尝试下Readme中的流程，这个README我们验证过是能跑通的：LLama2的流程：https://github.com/alibaba/Pai-Megatron-Patch/blob/main/examples/llama2/README.md

jerryli1981 commented 5 months ago

就是完全按照这个README先走一遍看看：LLama2的流程：https://github.com/alibaba/Pai-Megatron-Patch/blob/main/examples/llama2/README.md