search

minosworld / minos

MINOS: Multimodal Indoor Simulator
MIT License
201 stars 33 forks source link

ObjectType is irelevant #34

Closed kojimano closed 6 years ago

kojimano commented 6 years ago

Also, I am getting a very random result for ObjectType channel of sensor input. I was expecting the ObjectType to be relevant to ObjectType.csv, but this does not likely to be the case. Also, I was observing the same object types (such as floor) are getting a very different assignment of semantic labels across different scenes.

kojimano commented 6 years ago

I am getting this as a not a bug since server/node_modules/sstk/client/js/lib/scene/SceneUtils.py is just assigning the index to the object in the order which exists in the scene, but I think it is more convenient for the uses to have the index constant over same object types. (For example, we can just set categoryIndex.indexOf('obj', true); for all object types no matter they exist in the scene or not in a constant order and export such constant order via csv file.) Thank you for all your effort on maintaining MINOS. I will close this issue once I get a response, since I have kind of solved the issue by modifying js script.

msavva commented 6 years ago

@kojimano the objectType sensor is designed to provide stable indices that correspond to category labels. The mapping from object category to index is specified through the --objectttypes_file argument (see https://github.com/minosworld/minos/issues/14 for a discussion on which predefined mapping files are available). Instance segmentation masks are provided by the objectId and roomId sensors -- the behavior of these id sensors is what you are describing (i.e. randomly assigned unique index for each object or room instance). I have summarized this information in the FAQ. Please let me know if anything is not clear.

Can you check if you observe inconsistent/unstable index values for the same object category by running python3 demo.py -s objectType --objecttypes_file ../minos/server/node_modules/sstk/server/static/data/suncg/objectTypes.csv --save_observations? (I have updated the gym wrapper demo.py with a --save_observations debug argument to dump all available observations to png files). If the output does not match the values in the csv file, please follow up with an example output so that we can track down any bugs.

kojimano commented 6 years ago

Thank you for all the clarifications!