NVIDIAGameWorks / kaolin

A PyTorch Library for Accelerating 3D Deep Learning Research
Apache License 2.0
4.41k stars 540 forks source link

Running tutorial on different Omniverse kitchenset data- does not fit objects #396

Open jaymefosa opened 3 years ago

jaymefosa commented 3 years ago

Swapping out the clock files for a kitchenware set of images (10 images) generated by the Kaolin datagenerator in Omniverse, following the Kaolin video tutorial, results in mesh generation that is roughly stuck in the shape of a sphere. Examples for the 'ladle' mesh completely failed.

Shown here is the pot mask where the handle can't be learned:

image

Should the tutorial code be runnable on other data generated using the kitchenset files? I had thought it could overfit to a single mesh example but it's having issues with shapes. My understanding was that the sphere template wasn't a barrier for these kinds of deformations?

Caenorst commented 3 years ago

The tutorial should work with other shapes, although I will admit that haven't tested others, there could be a little bit of hyperparameter tuning to do, will be looking at this one