Rendering with my own object models.

[krish240574]

Hello, Thank you very much for the great work, much appreciated. I am trying to render and subsequently train the AAE using my own set of models. These are models of various cars. I intend to use the whole pipeline, object detection and pose estimation too.

Could you please list the steps I must follow, to achieve the same? The documentation in the Readme seems incomplete.

Edit: Here is what I have understood, please do correct me if I'm wrong : Step 1: Generate data for the detector.

• I can use a .ply file - 3D object model inside generate_syn_det_train.py to generate my training data
• this will generate a bunch of .png images and .xml files(bounding boxes) [I'm assuming these .png and .xml files are fed to the RetinaNet model for detection fine-tuning. ] Step 2: When I run the ae_train.py, it looks for a .npz file inside get_training_images - where do I find this .npz? Is the output of the RetinaNet fed into the ae_train.py, as a .npz?

Step 3: Train the AAE In the README, it mentions "Insert the paths to your 3D model and background images" - does it mean I use the same .ply used in the generate_syn_det_train.py, along with VOC images here?

Thank you very much, Kumar

[flugenheimer]

in your workspace there is a folder called tmp_datasets. Here is the .npz file. This file contains your renderings and masks.

What I did was to create a new script to generate data for the retinanet training using the .npz file of already rendered images: random background images, random scaling, location, noise etc etc, and automatically annotate boundary boxes etc in a csv file that retinanet can read directly.

[MartinSmeyer]

Step 1+2: You can do it as flugenheimer proposed or you can run the generate_syn_det_train.py to create training data for RetinaNet. In both cases you need to transform the labels into the .csv format needed by the RetinaNet repository. Step 3: Yes, use the same ply file here.