CyberAgentAILab / layout-dm

LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [Inoue+, CVPR2023]
https://cyberagentailab.github.io/layout-dm
Apache License 2.0
225 stars 24 forks source link

Problem of ``AddNoiseToBBox`` transform strategy. #13

Closed ZetangForward closed 1 year ago

ZetangForward commented 1 year ago

I notice that when training FID model, this repo apply AddNoiseToBBox transform in fid/train.py #56. However, it may cause H or W smaller than 0, since H and W are decimal too.

I face the problem below which verify my guess.

Traceback (most recent call last):
  File "helper/train_fid.py", line 203, in <module>
    main()
  File "helper/train_fid.py", line 157, in main
    save_image(
  File "/workspace/zecheng/layout-generation/helper/visualization.py", line 108, in save_image
    img = convert_layout_to_image(boxes, labels, colors, canvas_size, **kwargs)
  File "/workspace/zecheng/layout-generation/helper/visualization.py", line 67, in convert_layout_to_image
    draw.rectangle([x1, y1, x2, y2], outline=colors[label], fill=c_fill)
  File "/opt/conda/envs/eval/lib/python3.8/site-packages/PIL/ImageDraw.py", line 292, in rectangle
    self.draw.draw_rectangle(xy, fill, 1)
ValueError: y1 must be greater than or equal to y0

Can you explain how to fix this problem, and why AddNoiseToBBox() is necessary ? Thank you

naoto0804 commented 1 year ago

Maybe a quick remedy is to clamp the range after adding noise? I'm curious why I did not happen to encounter the same bug. Did you change the std. of the noise?

why AddNoiseToBBox() is necessary ?

It is to make real and fake samples for layout classifier by applying AddNoiseToBBox() with p=0.5.

ZetangForward commented 1 year ago

Maybe a quick remedy is to clamp the range after adding noise? I'm curious why I did not happen to encounter the same bug. Did you change the std. of the noise?

why AddNoiseToBBox() is necessary ?

It is to make real and fake samples for layout classifier by applying AddNoiseToBBox() with p=0.5.

Thanks for your reply. Yes, I clamp the range after adding noise, and this problem is solved. Maybe this problem happens when the machine is different (I guess :)

hyer commented 1 year ago

I encounter the same problem with my custom dataset. It is caused by AddNoiseToBBox. I solved it by add data.x = torch.clamp(data.x, min=0.0, max=1.0) to AddNoiseToBBox. But I'm not sure if this will cause other problems.

naoto0804 commented 1 year ago

Thank you for reporting. Since the predicted values are usually bounded to the range from 0.0 to 1.0, it probably has no side-effect.