Open memory4963 opened 2 years ago
I'm sorry, the README.md is too brief.
The reason for the error is that the image feature's shape isn't 3-dimension. You can see the correct feature shape image_feat_shape.png and read the scripts/README.md before extracting the image feature.
Question1: data/dict.en2fr_mask*.txt
will be updated.
Question2: the meaning of mask0
is training models on origin multi30k text (no mask token).
Thank you very much and sorry for my late reply.
Actually, I have followed the scripts/README.md
, maybe I did something wrong? I will check my code again and will tell you if it still has errors.
BTW, about the data/dict.en2fr_mask*.txt
, maybe you should also change README.md
and preprocess_mmt.sh
?
in the last line:
--srcdict data/dict.en2de_$mask.txt
Hi, thank you very much for the excellent work!
However, I am facing an error when I try to train the network with
train_mmt.sh
:I followed your instruction for preprocessing data, and the only thing I changed is in
train_mmt.sh
:image_feat=vit_tiny_patch16_384
->image_feat=vit_base_patch16_384
mask_data=mask0
->mask_data=mask1
My data tree is shown below:
I didn't show mask 2-4, c, p and fr for simplicity.
Could you tell me how to fix it?
By the way, I have 2 more questions want to confirm with you:
data/dict.en2de_mask*.txt
but nodata/dict.en2fr_mask*.txt
, should I use these files to preprocess fr text too?mask_data
intrain_mmt.sh
is set tomask0
defaultly, but in the README, the choices ofmask
do not containmask0
, as shown below. Could you tell me what's the meaning ofmask0
and how to generate its data?