search

boheumd / MA-LMM

(2024CVPR) MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding
https://boheumd.github.io/MA-LMM/
MIT License
221 stars 26 forks source link

test for qa #16

Closed YALYAshley closed 3 months ago

YALYAshley commented 4 months ago

When I choose to run test_qa.sh, the following error occurs: ValueError: Input length of input_ids is 0, but max_length is set to -43. This can lead to unexpected behavior. You should consider increasing max_length or, better yet, setting max_new_tokens.

This is my test_qa.sh： checkpoint_path='/saved_model/MSVD_qa/checkpoint_best.pth' torchrun --nproc_per_node=4 \ --master_port=34652 \ train.py \ --cfg-path lavis/projects/malmm/qa_msvd.yaml \ --options \ model.arch blip2_vicuna_instruct \ model.model_type vicuna7b \ model.load_finetuned False \ model.load_pretrained True \ model.num_query_token 32 \ model.vit_precision fp16 \ model.freeze_vit True \ model.memory_bank_length 10 \ model.num_frames 80 \ run.init_lr 1e-4 \ run.max_epoch 5 \ run.num_beams 5 \ run.batch_size_train 8 \ run.batch_size_eval 8 \ run.accum_grad_iters 1 \ run.num_workers 12 \ run.seed 42 \ run.evaluate True \ run.valid_splits "['val', 'test']" \ run.report_metric True \ run.prefix test \ run.resume_ckpt_path ${checkpoint_path}

YALYAshley commented 4 months ago

Hope to get help, thank you.

chhchan commented 3 months ago

I also met this problem. May I ask if you have resolved it?

ZhixuanWuA commented 3 months ago

I also met this problem. May I ask if you have resolved it?

Not yet.

joslefaure commented 3 months ago

In the call to the generate function, in the blip2_vicuna_instruct.py file, amend the call to the generate function, L465 to include the max_new_tokens as follows:

outputs = self.llm_model.generate( inputs_embeds=inputs_embeds, attention_mask=attention_mask, do_sample=use_nucleus_sampling, top_p=top_p, temperature=temperature, num_beams=num_beams, max_length=max_length, min_length=min_length,

eos_token_id=self.eos_token_id,

repetition_penalty=repetition_penalty, length_penalty=length_penalty, num_return_sequences=num_captions, max_new_tokens=max_length, # Here )

ZhixuanWuA commented 3 months ago

In the call to the generate function, in the file, amend the call to the generate function, L465 to include the as follows:blip2_vicuna_instruct.py``max_new_tokens

outputs = self.llm_model.generate( inputs_embeds=inputs_embeds, attention_mask=attention_mask, do_sample=use_nucleus_sampling, top_p=top_p, temperature=temperature, num_beams=num_beams, max_length=max_length, min_length=min_length,

eos_token_id=self.eos_token_id,

repetition_penalty=repetition_penalty, length_penalty=length_penalty, num_return_sequences=num_captions, max_new_tokens=max_length, # Here )

Thank you. I will try it.