search

NVIDIAGameWorks / kaolin

A PyTorch Library for Accelerating 3D Deep Learning Research
Apache License 2.0
4.41k stars 540 forks source link

Parsing realistic 2d Images #423

Closed Medoalmasry closed 2 years ago

Medoalmasry commented 3 years ago

Is there a way to load a realistic 2D image, with it's corresponding mask into the kaolin library? As in without any camera properties, or the omniverse application?

Caenorst commented 3 years ago

Hi @Medoalmasry !

You can use a Deep Learning model to do the segmentation, in DIB-R paper, it is said that the mask in PASCAL3D+ is estimated from Mask R-CNN.

Medoalmasry commented 3 years ago

Would you please direct me towards a simple tutorial or at the very least reference the relevant api calls to load a single realistic image with it's mask and produce the corresponding point cloud or mesh?

Caenorst commented 2 years ago

There is is no such functionality directly implemented in Kaolin, there is a torchvision tutorial for Mask R-CNN here: https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html