ankush-me / SynthText

Code for generating synthetic text images as described in "Synthetic Data for Text Localisation in Natural Images", Ankush Gupta, Andrea Vedaldi, Andrew Zisserman, CVPR 2016.
http://www.robots.ox.ac.uk/~vgg/data/scenetext/
Apache License 2.0
2.02k stars 622 forks source link

The rotations of some word bounding boxes do not line up properly with the word itself. #26

Open cjnolet opened 7 years ago

cjnolet commented 7 years ago

It's strange and it happens quite often, maybe 20% of the time. It usually occurs when a word is rotated itself or has been projected onto a surface causing it to have a strange dimension. Thing is, though, it does not only happen on extreme projections. It also happens on only slightly projected text. It makes it hard for the fully convolutional network to converge well on the sin/cos pose params.

ankush-me commented 7 years ago

Can you please share some pictures? That'll help understand this problem better.

cjnolet commented 7 years ago

Unfortunately, I'm on a private network creating these images and they will not allow me to release them. They will allow me to help out fixing bugs and stuff, however.

Anyways, I've finally gotten some time to code up a visualization of each of the character bounding boxes and the final word bounding boxes that are generated. It looks like there is an empty character being drawn on the canvas. Literally it's blank and it ends up returning a 0-width bounding box which has a pretty large height. I believe that is throwing off the calculation of the final word bounding box dimensions.

I already know some of the fonts are having trouble drawing some of the characters in this language but that's so strange it's returning a 0-width bounding box. Many of the other fonts / bad characters just show up as an empty square.

IgorSondors commented 3 years ago

This problem occurs because of pygame render_to rotating character image within bounding box only but not rotating bounding box itself. Check this line https://github.com/ankush-me/SynthText/blob/e18aaf913b5849ae28bbb9700f52b7d64dd05e91/text_utils.py#L196 The is an example photo_2021-01-22_16-28-12

image

ankush-me commented 3 years ago

@IgorSondors --- thanks! If you can share a patch for this, that would be great!