search

facebookresearch / phosa

Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild
Other
177 stars 21 forks source link

Is it possible to make the Colab create a 3D model from a single image of a whale? #16

Closed hmatt843 closed 3 years ago

hmatt843 commented 3 years ago

Hi, I went through a Colab example (PHOSA Colab https://colab.research.google.com/drive/1QIoL2g0jdt5E-vYKCIojkIz21j3jyEvo?usp=sharing ) without having issues. But when I tried to change it to a whale jpg (https://bdmlr.org.uk/wp-content/uploads/2020/01/animal-humpback_whale.png) it failed out on several code blocks. Is what I'm trying to do (get a 3D model of a single whale image) possible with this?

jasonyzhang commented 3 years ago

Hi,

The default code is only intended to work for images with humans and bicycles. It should be possible to add any category that detectron2 can detect and segment. Unfortunately, detectron2 cannot segment whales out of the box since whales are not a COCO object category.

If you are looking to do 3D object reconstruction from images, I would recommend looking at approaches for deformable objects. Some examples of recent works include A-CSM and U-CMR. However, I don't think these approaches work on whales specifically.

Best, Jason