search

alibaba / Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
Apache License 2.0
707 stars 100 forks source link

llava run error #330

Open yangzhipeng1108 opened 2 months ago

yangzhipeng1108 commented 2 months ago

cd /workspace/Pai-Megatron-Patch/examples/llava/ sh run_pretrain_megatron_llava.sh \ dsw \ /workspace/Pai-Megatron-Patch \ 7B \ 4 \ 32 \ 1e-3 \ 1e-4 \ 2048 \ 2048 \ 0 \ bf16 \ 1 \ 1 \ sel \ true \ true \ true \ true \ 100000 \ /workspace/data/LLaVA-Pretrain/blip_laion_cc_sbu_558k.json \ /workspace/model_ckpts/vicuna-7b-v1.5-to-mg-tp1-pp1 \ 10000000000 \ 100000000 \ /workspace/output_patch_test

image

jerryli1981 commented 2 months ago

您好，在run_pretrain_megatron_llava.sh下面加一个--dataset LLava-Pretrain-Raw 试试

yangzhipeng1108 commented 2 months ago

image image

hxmzy520 commented 2 weeks ago

我也遇到过同样的问题， 请问/workspace/model_ckpts/vicuna-7b-v1.5-to-mg-tp1-pp1是使用什么脚本转换的？

yangzhipeng1108 commented 1 week ago

我也遇到过同样的问题， 请问/workspace/model_ckpts/vicuna-7b-v1.5-to-mg-tp1-pp1是使用什么脚本转换的？

llama2