search

jimmyyhwu / pose-interpreter-networks

Real-time robotic object pose estimation with deep learning
MIT License
122 stars 27 forks source link

Problem about training on LineMod dataset #4

Closed FionaFN closed 5 years ago

FionaFN commented 5 years ago

I want to train your network on linemod dataset, however, I have question writing the config file such as 20171103_OilChange.json. I cannot understand the messages in the json file like the following...

{"segmentation":{"counts":"mYVe08]Q14M4L3L4N1N3M3L3N3N1M4N2L3N3mLZOPUOh0jj0ElTO?Qk0HhTO;Tk0LfTO7Wk02aTO0\k08\TOKbk0?STODjk0j0hSOYOUl0W3O1N2O00001O00O100O1O1O1O1O1O1N2O1O1O1O1O1O100O1O1O1O1O1O100O1O1O1O1O1O1O1O1O1O1O1O1O1000000O10000000000O100000000000000001O000000001O000000000000001O0000000000000000O10000O100O1O1O1N2O1O1O1O1O1N2O1O1N2N2O1O1O1O1O1N2O1O1N2O1O1O1N2O1O1O1N2O1O1O1O1O1N2N2O1O1N200O100O100O100O100O100O1O1N3O2J6D<E`0Ca0\Oc0BQ[\T1","size":[1080,1920]},"area":21399,"pose":{"position":{"y":-0.3352892126408012,"x":-0.1564572835692801,"z":0.7772331158642385},"orientation":{"y":0.9502802730465292,"x":-0.303411278174998,"z":-0.01268932429862474,"w":0.06890558746337358}},"iscrowd":0,"image_id":0,"bbox":[643.0,0.0,173.0,179.0],"category_id":6,"id":0},

Could you please explain how did you create these .json files and what should I do to my own dataset such as LineMod to get these segmentation ground truth? Thank you !!!

jimmyyhwu commented 5 years ago

The .json annotation files follow the widely-used COCO format. The example you are showing is a segmentation mask encoded using RLE. You can encode/decode RLE using the mask tools in the COCO API.

See also usage of the COCO API in this codebase, such as in datasets.py.

Just a fair warning that as discussed in the paper, pose interpreter networks do not work with heavily occluded objects such as those in LINEMOD. The binary mask intermediate representation makes transferring synthetic to real very easy, but the trade-off is less robustness to occlusion.

FionaFN commented 5 years ago

Thank you so much for your kind reply to my stupid question!! I'm just looking into the COCO format...Maybe I should first read through dataset.py... Then how about if I train each object in LineMod seperately to avoid the occlusion problem?

jimmyyhwu commented 5 years ago

If objects are not occluded during training/testing then there should be no problem.

FionaFN commented 5 years ago

Okay! Thanks again!!

meghalD commented 4 years ago

I want to train your network on linemod dataset, however, I have question writing the config file such as 20171103_OilChange.json. I cannot understand the messages in the json file like the following...

{"segmentation":{"counts":"mYVe08]Q14M4L3L4N1N3M3L3N3N1M4N2L3N3mLZOPUOh0jj0ElTO?Qk0HhTO;Tk0LfTO7Wk02aTO0\k08\TOKbk0?STODjk0j0hSOYOUl0W3O1N2O00001O00O100O1O1O1O1O1O1N2O1O1O1O1O1O100O1O1O1O1O1O100O1O1O1O1O1O1O1O1O1O1O1O1O1000000O10000000000O100000000000000001O000000001O000000000000001O0000000000000000O10000O100O1O1O1N2O1O1O1O1O1N2O1O1N2N2O1O1O1O1O1N2O1O1N2O1O1O1N2O1O1O1N2O1O1O1O1O1N2N2O1O1N200O100O100O100O100O100O1O1N3O2J6D<E`0Ca0\Oc0BQ[\T1","size":[1080,1920]},"area":21399,"pose":{"position":{"y":-0.3352892126408012,"x":-0.1564572835692801,"z":0.7772331158642385},"orientation":{"y":0.9502802730465292,"x":-0.303411278174998,"z":-0.01268932429862474,"w":0.06890558746337358}},"iscrowd":0,"image_id":0,"bbox":[643.0,0.0,173.0,179.0],"category_id":6,"id":0},

Could you please explain how did you create these .json files and what should I do to my own dataset such as LineMod to get these segmentation ground truth? Thank you !!!

Can you please tell me how to generate data to train the segmentation and pose estimation network on LineMod dataset. currently I have .obj file for each object, cropped images and images without texture in LineMod data. How do I generate segmentation masks and .stl files, annotation files. Please help.