search

gwxie / Dewarping-Document-Image-By-Displacement-Flow-Estimation

Dewarping Document Image By Displacement Flow Estimation with Fully Convolutional Network
MIT License
162 stars 36 forks source link

Training model unable to flat horizontal orientation of the image #6

Open AndyXW opened 3 years ago

AndyXW commented 3 years ago

Hello！ I used your Distorted Image code to generate my own training dataset, and I trained the model using this dataset. But it seems that the output model is not able to correct the horizontal orientation of the image, like the picture below shows. 101_4 copy You can see that the left side and the right side of the flated image is also as same as the distort image, keep tilt. But on the vertical direction, like the up side and bottom side have been corrected. The same problem happens on the all the test pictures. SO I check the output regress of the model, the displacement value of horizontal direction is very small, the max absolute value is only around 0.1. But the displacement value of vertical direction is relatively big, which seems reasonable. For detecting this problem, I have attempted to check the loss function, but it seems correct. And also I checked the generated label_regress which contains the flow of x and y. I just used the label_regess to flat the distorted image, then the distorted image can be flated in a totally correct way. So the generated training dataset is correct. Do you have any idea what the problem might be? Thanks for your help in advance.

AndyXW commented 3 years ago

And also I chose one image from training set, and check the output of model of different epoch. Maybe this will give you more information. image

gwxie commented 3 years ago

And also I chose one image from training set, and check the output of model of different epoch. Maybe this will give you more information. image

Hi, How many epochs have you trained? Did you start training from scratch?

AndyXW commented 3 years ago

I have trained 300 epochs and the model start training from scratch.

AndyXW commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

huxianer commented 3 years ago

I have trained 300 epochs and the model start training from scratch.

Yes,I meet the same problem start training from scratch,I think some thing should be fixed, such as network structure, some parameter . Could you give any advice @gwxie @AndyXW

gwxie commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

AndyXW commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

gwxie commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

AndyXW commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

gwxie commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

Hello, @AndyXW and @huxianer. Thanks for your feedback! I suggest you try to synthesize more data (about 10k-30k). Or, please simplify the structure of the model and use BN, etc.

huxianer commented 3 years ago

OK! I will try this way and sencerely grateful for your kind help!

AndyXW commented 3 years ago

Hi！I ran several experiments, that shows the second channel of output_regress won't update correctly whether it is yflow or xflow, as long as it is on the second channel, the output is small.

Hi @AndyXW , Have you tested on synthetic images?

Hi! Yes, I have tested on these images, but it is the same problem. Now I try to finetune the model you have trained on my dataset. Maybe it's gonna work.

Hi！How much images did you synthesize? Did the model converge?

4000 images, the model converge but the loss is high, the total loss is around 2.

Hello, @AndyXW and @huxianer. Thanks for your feedback! I suggest you try to synthesize more data (about 10k-30k). Or, please simplify the structure of the model and use BN, etc. Okay, thanks for your help!