Request for detailed tutorial

[ruarua6666]

Hi, Barry Menglong Yao,

First and foremost, I'd like to express my admiration for the incredible work you've done on Mocheg. It's evident that a lot of effort and dedication went into it, and I truly appreciate your contribution to the community.

I am genuinely interested in diving deeper into your project and leveraging it for my own research/work. However, I've encountered some challenges in setting everything up. Would it be possible for you to provide a more detailed tutorial? Specifically, clarity on the data set storage path, the pre-trained model storage path, and any other configuration details would be extremely beneficial.

Understanding that you are busy, I hope this isn't too much of an inconvenience. Your guidance would significantly assist many of us in the community who are keen to make the most out of your wonderful project.

Thank you so much for considering my request, and once again, hats off to your outstanding work!

Warm regards.

[Barry-Menglong-Yao]

Thanks for your interest! It will be helpful to add a detailed tutorial. I will work on it.

In order to solve the challenges in your case, do you mind listing these challenges?

[ruarua6666]

Thank you for your time! I would like to know how I should place the dataset in the folder? For example, should I create a folder called "data" and put all the files in the dataset in it?

[Barry-Menglong-Yao]

Yes. A folder named "data" for the dataset and a folder named "checkpoint" for checkpoints.

In fact, you can also use other paths as long as you pass your corresponding paths as Python arguments. They are Python arguments defined in corresponding scripts like retrieve_train.py.

Could you list other challenges so others can also benefit from your experience? It will also help me to understand how detailed the tutorial should be.

[ruarua6666]

Thank you for your patience and help. I'm at an academic conference right now and can't start the experiment right away, I'll be coming in next week to start the experiment and I'll list the challenges I'm having then.

[ruarua6666]

Hi,

While running the command python verify.py --model_type=CLAIM_TEXT_IMAGE_attention_5_4 --batch_size=2048 --lr=0.001 --loss_weight_power=2, I encountered an error in Mocheg-main/verification/train_verify.py at the line output = model(claim_img_encod, device):

RuntimeError: The size of tensor a (85) must match the size of tensor b (77) at non-singleton dimension 1.

I traced the issue back to the file /Mocheg-main/verification/util/data_util.py where the line self.processor(text=claim_and_text_evidence_list, images=img_list, return_tensors="pt", padding=True, truncation=True, max_length=77) did not specify a truncation length. After setting max_length to 77, the code ran without issues.

Is this the correct solution? Or is there an issue with the model I'm loading?

Thank you for your assistance!

[Barry-Menglong-Yao]

Yes. The key is to truncate the text under 77 tokens.

It seems I truncated the text during another preprocess. But your solution is better. I have merged your modification into the repository. Thanks!