search

luogen1996 / LaVIN

[NeurIPS 2023] Official implementations of "Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models"
502 stars 36 forks source link

Beginner cuestion #27

Open mauridev777 opened 1 year ago

mauridev777 commented 1 year ago

Hello. I been trying the entire day and I cannot find how to run the pretrained ScienceQA for testing: LLAMA2-7B (https://drive.google.com/file/d/10X2qCBYrLH1grZOHwHRMXLUoz-S6MSgV/view?usp=share_link)

I have downloaded the file but I am not sure where to add it. I have created a /content folder and a 7B folder placing the model inside of it, and using the default LLAMA7B params.json but I get errors.

Is there any guide for making the inference using the pretrained models?

Any help or guidance is really appreciated

Thanks!

luogen1996 commented 10 months ago

Please make sure the data folder is built correctly as here. Then you can run the script: CUDA_VISIBLE_DEVICES=2 torchrun --nproc_per_node 1 --master_port 11111 eval.py \ --ckpt_dir ../data/weights/ \ --llm_model 7B\ --tokenizer_path ../data/weights/tokenizer.model \ --data_root ../data \ --caption_file ../data/captions.json \ --adapter_path ./LaVIN-7B/checkpoint-19.pth \ --adapter_type attn \ --adapter_dim 8 \ --adapter_scale 1 \ --prompt_format QCM-ALE \ --max_batch_size 64\ --max_seq_len 512 \ --split test \ --n_prompt 6 \ --temperature 10.\ --visual_adapter_type router It's better to list your detailed errors so that i can know how to address it.