I am currently working on fine-tuning the CuMo model following the instructions in the "Getting Started" section of the repository. After downloading the necessary datasets and JSON files, the next step involves running the pre-training script for the MLP connector as shown below:
According to this script, deepspeed should execute the file llava/train/train_mem.py with specific parameters. However, I noticed that there is no llava directory in the repository. Instead, there is a cumo directory that contains a train directory with a train_mem.py file.
Could you please clarify if this is a typo in the script, and the path should actually be cumo/train/train_mem.py? Or is there an additional llava directory that needs to be downloaded separately?
Hello,
I am currently working on fine-tuning the CuMo model following the instructions in the "Getting Started" section of the repository. After downloading the necessary datasets and JSON files, the next step involves running the pre-training script for the MLP connector as shown below:
bash scripts/cumo/mistral_7b/pretrain_mistral_7b.sh
The contents of the pretrain_mistral_7b.sh script are as follows:
According to this script, deepspeed should execute the file llava/train/train_mem.py with specific parameters. However, I noticed that there is no llava directory in the repository. Instead, there is a cumo directory that contains a train directory with a train_mem.py file.
Could you please clarify if this is a typo in the script, and the path should actually be cumo/train/train_mem.py? Or is there an additional llava directory that needs to be downloaded separately?
Thank you for your assistance!