CyberAgentAILab / layout-dm

LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [Inoue+, CVPR2023]
https://cyberagentailab.github.io/layout-dm
Apache License 2.0
211 stars 23 forks source link

Testing on my own set of bounding boxes #3

Closed Ashwin-Pokharel closed 1 year ago

Ashwin-Pokharel commented 1 year ago

Hey , if i have my own set of bounding boxes how can i generate the dataset needed for testing. Would i have to create my own dataset class that mirrors the one being used by publaynet?

naoto0804 commented 1 year ago

Hi, thank you for your interest in our repo. The answer is generally yes. Please refer to a guide for custom dataset generation.

If you just want to test some own-made PubLayNet-like layouts quickly, another option might be to initialize a Batch instance directly. This instance is usually sampled from the dataset or dataloader instance in our framework, but it is possible to make it on your own.

Ashwin-Pokharel commented 1 year ago

oh thats perfect yea i will utilize the batch instance. i just wanted to still use the weights from publaynet but be able to apply it to my own custom bounding boxes. If you have any further suggestions for that please let me know.

Ashwin-Pokharel commented 1 year ago

Also one last thing , if i wanted to for example figure out what a resized layout might look like , how would i let the model predict only w , h or some other pieces

naoto0804 commented 1 year ago

how would i let the model predict only w , h or some other pieces

We've defined some common usecases as "task" here. If these cases do not meet your requirements, then you can define your task and implement get_cond for that task by hiding all the fields that you want to predict.

naoto0804 commented 1 year ago

If you have any further suggestions for that please let me know.

I'll make a small example to instantiate a Batch instance later since some other people might ask similar kinds of things. However, I'm busy in this and next week so I'm not sure when I can do it.

Ashwin-Pokharel commented 1 year ago

thank you so much , i was able to accomplish it by creating my own dataset and following most of the steps of the configuring custom dataset. Just had to remember pass in the custom dataset with the taget index during prediction.

Ashwin-Pokharel commented 1 year ago

sorry to reopen the issue thread but I had one last question , can these tasks take into account differing canvas sizes?

naoto0804 commented 1 year ago

No problem. Current layout generation approaches, including ours, cannot consider different canvas sizes since they do not feed the canvas size information to their model. We may provide some signals to consider the canvas sizes by logit adjustment during the iterative prediction of LayoutDM in principle, but I haven't tested it yet. It is the next challenge that the community should tackle.

Ashwin-Pokharel commented 1 year ago

ahh ok thank you!