search

bethgelab / stylize-datasets

A script that applies the AdaIN style transfer method to arbitrary datasets
Other
155 stars 37 forks source link

How did you ensure the bounding box location of stylized images not to be changed #8

Closed XCRobert closed 5 years ago

XCRobert commented 5 years ago

How did you ensure the bounding box location of stylized images not to be changed ? Does it need additional annotation bboxes for stylized images.

Hvitgar commented 5 years ago

You are right, stylization may change the bounding boxes by several pixels. We did not correct for that, but just used the original annotation files. Afaik, there is no algorithmic or general approach to solving that problem, since the bounding boxes vary depending on the style that is used and the style is randomly selected. As we applied stylization only to the training data (and used the concatenation of clean and stylized data most of the time) this didn't seem too much of an issue to us. Testing against clean aswell as corrupted data confirmed that performance was not hurt by the fact that the bounding boxes in the stylized dataset are not pixel perfect anymore.

michaelisc commented 5 years ago

As @Hvitgar already pointed out, potential alignment problems coming from the stylization were not an issue during training. We were worried though that it may cause problems when training for instance segmentation. We however found this not to be the case even when training only on stylized data.

For details compare Tables 2 & 4 in the paper: The performance drop going from Object Detection to Instance Segmentation is roughly the same for all models (~3% when evaluating on clean data) with no significant additional performance loss for the models trained only on stylized data.

XCRobert commented 5 years ago

One possible way is to stylize images with spatial control by using mask information. The performance may be improved.

Hvitgar commented 5 years ago

Can you provide any resources to that method? Might be very interesting to have a look at that

XCRobert commented 5 years ago

Can you provide any resources to that method? Might be very interesting to have a look at that

  1. https://github.com/xunhuang1995/AdaIN-style#spatial-control Or
  2. https://blog.csdn.net/qq_35586657/article/details/97103552 (results can be seen in the tiger-style cat pictures with and without mask ) https://github.com/cysmith/neural-style-tf/blob/a2c374f9ee2938f0022e1e0b720f4eb28cf7d0a8/neural_style.py#L388