jchenghu
commented
1 month ago Hi,
I'm not sure I'm not sure I understanding correctly the question
Dear sir， In order to enhance the image features, I extracted features.hdf5 file to add 2D positional encoding operation, and then training, this is my first step of training 8 epoch results, I would like to know why the checkpoint file size is only more than 400MB,
This is normal if you just finished the first training step since the checkpoint does not include the swin-transf. visual backbone
I remember that when I do not add the encoding is more than 2GB,
this is odd. If you're talking about the pre-trained models that can be downloaded from Google Drive, they are complete checkpoints (including the backbone) and are the results of the training steps 3 and 6 in the readme, this is why the checkpoints weight 2GB. When you train only on the first step, it contains only the fusion model, which is 400MB
but according to the results show and I do not add before compared to not good, but I run test.py to validate this checkpoint score is very low,
The last image you posted (I cannot see the first one) showcases a reasonable score. Keep in mind that you should multiply each value by x100 to get the actual score. For instance, the CIDEr-D score of 1.23 is actually 123.
and why is it the same as the training process.
I did not get what you meant here, maybe your first image for the matter, was important but I cannot visualize it
But when I run test.py to validate this checkpoint, the score is very low
If you got 1.23 (123 CIDEr) it should be fine.
, why is it different from the score of the training process.
I did not understand this bit
Note that the dataset I'm using is low resolution mscoco, only 2GB.
I find this odd, if the dataset was scaled down to only 2GB, I didn't expect the score to be that high! It means is performing exceptionally.... But again, I might not be understanding the situation correctly
Dear sir， In order to enhance the image features, I extracted features.hdf5 file to add 2D positional encoding operation, and then training, this is my first step of training 8 epoch results, I would like to know why the checkpoint file size is only more than 400MB, I remember that when I do not add the encoding is more than 2GB, but according to the results show and I do not add before compared to not good, but I run test.py to validate this checkpoint score is very low, and why is it the same as the training process. But when I run test.py to validate this checkpoint, the score is very low, why is it different from the score of the training process. Note that the dataset I'm using is low resolution mscoco, only 2GB.