I have a question

[daeing]

Can this code satisfied different size of image use your pretrained model? I found you use pretrained model from deit, and resize each image to 224 * 224? So can I define imgH and imgW another number and use pretrained model?

[roatienza]

The input image can be any size BUT it has to be resized to 224x224 before feeding it to the pre-trained model since the network has a fixed input dims. So, the pretrained model is only valid for imgH and imgW that are both 224.

[daeing]

The input image can be any size BUT it has to be resized to 224x224 before feeding it to the pre-trained model since the network has a fixed input dims. So, the pretrained model is only valid for imgH and imgW that are both 224.

Thank you for your reply. yes, I got what you mean, I wonder if a picture which size is (32, 100) resized to (224, 224) (according to dataset.py). This picture might be distortion. Have you ever meet this situation？Or does transformer copy this scene well？Many thanks.

[roatienza]

There will be distortion when the image is resized to 224x224. Pls note that the dataset images are not all 32x100. They are of various sizes and orientations. So, regardless of input size, there will always be distortion. Distortion in 32x100 is less compared to 224x224. Since 224x224 is more of upsampling, there is minimal information loss from the pt of view of neural networks.

[Ferry-Li]

The input image can be any size BUT it has to be resized to 224x224 before feeding it to the pre-trained model since the network has a fixed input dims. So, the pretrained model is only valid for imgH and imgW that are both 224.

So if I train on the Imdb dataset which is created with my own data(image), there is no need to pay attention to the image size, right? And when I test on other datasets with my own trained model, the image size is also not in my consideration?

[daeing]

There will be distortion when the image is resized to 224x224. Pls note that the dataset images are not all 32x100. They are of various sizes and orientations. So, regardless of input size, there will always be distortion. Distortion in 32x100 is less compared to 224x224. Since 224x224 is more of upsampling, there is minimal information loss from the pt of view of neural networks.

OK, how long have you been train your own dataset? I finetune it on my own dataset(resize img to 224*224) it work well, but I train it on a new size, pretrained model can't be use fully, It cost many days and can't get a good result.

[roatienza]

The input image can be any size BUT it has to be resized to 224x224 before feeding it to the pre-trained model since the network has a fixed input dims. So, the pretrained model is only valid for imgH and imgW that are both 224.

So if I train on the Imdb dataset which is created with my own data(image), there is no need to pay attention to the image size, right? And when I test on other datasets with my own trained model, the image size is also not in my consideration?

I believe you should always pay attention to the correct image size whether you are using lmdb dataset or not.

[roatienza]

There will be distortion when the image is resized to 224x224. Pls note that the dataset images are not all 32x100. They are of various sizes and orientations. So, regardless of input size, there will always be distortion. Distortion in 32x100 is less compared to 224x224. Since 224x224 is more of upsampling, there is minimal information loss from the pt of view of neural networks.

OK, how long have you been train your own dataset? I finetune it on my own dataset(resize img to 224*224) it work well, but I train it on a new size, pretrained model can't be use fully, It cost many days and can't get a good result.

As far as I can recall, on a single gpu (eg V100), the tiny model can be trained in a day using pre-trained DeiT weights. 2.5 days for the small. Our group attempted to train vitstr on 32x128 images from scratch and got improvement in scores.

[daeing]

There will be distortion when the image is resized to 224x224. Pls note that the dataset images are not all 32x100. They are of various sizes and orientations. So, regardless of input size, there will always be distortion. Distortion in 32x100 is less compared to 224x224. Since 224x224 is more of upsampling, there is minimal information loss from the pt of view of neural networks.

OK, how long have you been train your own dataset? I finetune it on my own dataset(resize img to 224*224) it work well, but I train it on a new size, pretrained model can't be use fully, It cost many days and can't get a good result.

As far as I can recall, on a single gpu (eg V100), the tiny model can be trained in a day using pre-trained DeiT weights. 2.5 days for the small. Our group attempted to train vitstr on 32x128 images from scratch and got improvement in scores.

OKOK, thank you for your reply. I tried (224224) in training, but got a terrible result when i do inference in 32480, but I tried to cut the picture into 32 150, 32 150, 32 * 180, the combine result same to got a good result. maybe it belong to resize problem. I also found many badcases belong to like this....... result ====== ressult, so many repetition. Have you ever meet this problem? Or Have you tried CTC instead to TokenLabelConverter? many thanks for your attention.

[roatienza]

In ViTSTR, the CTC and Attention have been replaced by the transformer encoder. So, there is no need for it.