Implementing PyTorch Lightning `predict` behavior

[mbendjilali]

Hello there,

Having successfully trained a custom dataset and evaluated it, I would like to uncomment the trainer.predict() line located in the eval.py script for visual inspection. Unfortunately, I've been battling to implement the predict_dataloader, predict_dataset and other new variables needed for this.

To be more specific, my only issue is that after a successful instantiation of the cfg.datamodule, the variable datamodule.predict_dataloader().dataset is set to None while it should should be set to MyCustomDataset(len(predict_dataset)).

Up to this point, I tried to mimick the implementation of test_dataloader, test_dataset etc. in various scripts:

• I added predict_transform and on_device_predict_transform to the configs/datamodule/MyCustomDataset.yaml script.
• I initialized self.predict_dataset, self.predict_transform and self.on_device_predict_transform to None, added lines to the prepare_data() and setup() functions, implemented predict_dataloader() in src/datamodules/base.py.
• I added a predict_dir to the process() function in src/datasets/base.py.

Could you please help me understand what is missing or located in the code where are loaded the datasets ?

[drprojects]

Hi @mbendjilali, thanks for your interest in the project ! Glad to hear you managed to adapt the codebase to a new dataset (feel free to open a PR if your dataset is open 😉).

As mentioned in the currently-open issue #9, we do not implement lightning's predict behavior yet. Unfortunately, I have not had time to work on this yet, but intend to at some point in the near future.

You seem to be on the correct path, mimicking the test behavior (though you will need to ignore all metrics computation at some point). I can't tell what is wrong just from that, you will need to share your code. Would you like to submit a PR (can be a draft that we iteratively update) to help implement the predict behavior ?

Best,

Damien

[mbendjilali]

Hello @drprojects, although I cannot share my dataset with the community, I will gladly share my advancement on the predict behavior ! For now, I am unable to identify/understand what the function outputs, but at least it runs... PR coming soon !

[drprojects]

Cool, looking forward to seeing what you built !

[mbendjilali]

Hello @drprojects, after much trouble understanding what was wrong with trainer.predict, I realized that the model was running inference at the 1st level of the super point graph, as stated in your article. This explains the shape of the output not matching the input's. Could you help with that ? What should I do to run inferences down to the points ?

[drprojects]

Hi @mbendjilali. Yes, as pointed out above, issue #9 refers to this, where you can find suggestions on how to compute the full-resolution output.

Also, in here I suggest a draft on how to properly and efficiently implement the full-resolution inference.

As mentioned in #9, implementing the full-resolution inference and the predict behavior are on my to-do list. Just haven't had time to work on this yet. This may be coming next month, as I am working on a new version of this repo that supports panoptic segmentation.

[mbendjilali]

Oh shoot, I should've read the other issue before writing my comment, sorry :upside_down_face: ! I'll try to update my PR with one of the solutions you proposed.

[drprojects]

I'll try to update my PR with one of the solutions you proposed.

Great ! If I may, I would suggest you first focus on implementing the predict behavior first, with a $P_1$ prediction output.

I would say the full-resolution inference is a different task.

[drprojects]

Hi, I finally released the v2 of this project, which should address this issue :partying_face:

Closing this issue for now, feel free to re-open if need be.