search

facebookresearch / SIEVE

SIEVE: Multimodal Dataset Pruning using Image-Captioning Models (CVPR 2024)
MIT License
14 stars 0 forks source link

blip_model_without_ddp generate error #3

Closed Allenallen926 closed 3 weeks ago

Allenallen926 commented 2 months ago

Hi,

I am trying to run the first step (Compute alignment scores) but it reports the bug:

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer. RuntimeError: IntArrayRef contains an int that cannot be represented as a SymInt: -6578592183139129989

on this line: https://github.com/facebookresearch/SIEVE/blob/042705a26dec22097dac7a2a661214b583a3f77e/webdataset_inference.py#L168

Could you please help to check what is wrong here?

Thank you for the help.

nazMahmoud commented 2 months ago

Thank you for your interest in our work. The first step is to create a conda environment using the provided environment.yml file. This is the environment we used for this project.

Did you create a conda environment using:

conda env create --name Sieve --file=environment.yml
Allenallen926 commented 2 months ago

Thanks for the reply. Yes, I am using the conda env from the provided yml file.

More detailed error traceback:

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.
Traceback (most recent call last):
  File "/zhome/22/3/210280/code/SIEVE/webdataset_inference.py", line 522, in <module>
    main()
  File "/zhome/22/3/210280/code/SIEVE/webdataset_inference.py", line 516, in main
    result_dict = inference_func(args)
  File "/zhome/22/3/210280/code/SIEVE/webdataset_inference.py", line 215, in inference_func
    generated_caption = blip_model_without_ddp.generate(
  File "/zhome/22/3/210280/code/SIEVE/BLIP/models/blip.py", line 144, in generate
    outputs = self.text_decoder.generate(input_ids=input_ids,
  File "/miniconda3/envs/Sieve/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/miniconda3/envs/Sieve/lib/python3.10/site-packages/transformers/generation/utils.py", line 1564, in generate
    input_ids, model_kwargs = self._expand_inputs_for_generation(
  File "/miniconda3/envs/Sieve/lib/python3.10/site-packages/transformers/generation/utils.py", line 723, in _expand_inputs_for_generation
    input_ids = input_ids.repeat_interleave(expand_size, dim=0)
RuntimeError: IntArrayRef contains an int that cannot be represented as a SymInt: -6578592183139129989