search

Haochen-Luo / CroPA

19 stars 2 forks source link

Missing multiple details about the pretrained network, dataset, missing file and an error on importing the module #2

Open miladabd opened 3 months ago

miladabd commented 3 months ago

Dear Authors,

Thanks for providing the code for your paper.

While working with your code I encountered several errors that prevented running your code. I list the errors as follows:

Issue 1:

In the main.py file, line 440, running the import_module throws a module does not exist error. Based on the provided files, I suppose this line should changed to something like this:

module = importlib.import_module(f"modles.{config_args.model_name}")

This updated version works for me.

Issue 2:

It is not clear which checkpoint (for the pretrained VLMs) is used in the paper and where should we download it from. Could you please include more details about this?

As of now, for Open_Flamingo, I used the updated version of the weights in hugging face, but not sure which one is compatible with your code:

https://huggingface.co/openflamingo/updated-weights/tree/main/openflamingo

Issue 3:

The details of the datasets used for the experiments and how we can get them are also missing. For example, for VQA-v2 dataset used in Open_Flamingo, I followed the dataset that you have cited in your paper:

[Goyal et al., 2017] Making the v in vqa matter: Elevating the role of image understanding in visual question answering

However, the name of the files, the number of the images, and the captions seem to be slightly different from what your code is expecting. As an example, after debugging your code, I realized it expects a file with the following name:

filtered_v2_OpenEnded_mscoco_val2014_questions.json

However, this file does not exist in the VQA-v2 dataset that is cited in your paper: https://visualqa.org/download.html

Issue 4:

After replacing all this missing information with some estimated placeholder, I finally got the following error which is related to a missing file:

FileNotFoundError: [Eror 2] No such file or directory: 'data/icl_indices.json'

Thanks for your time and I hope that by providing missing information, people can use your repo more efficiently.

ranck626 commented 3 weeks ago

Thank you for your work and making it easier for me to run the code. I am curious how you constructed the dataset. I didn't understand the author's statement "The dataset used in the paper is a subset of MSCOCO validation set. we provide the dataset in the data/ directory." I appreciate your interest and I hope to be guided by you.