Understanding miscellaneous.py and analysis folder

[jiunyen-ching]

Hi, I successfully ran run_training.sh and run_test.sh. After run_test.sh, I have a model_iter149999.hdf5 file sized at 4.1 GB in the eval folder.

To evaluate the performance of the model, I looked into miscellaneous.py and wrote a short script: python miscellaneous.py --option fusion --benchmark nyucad

A few questions about the other parameters:

1. logdir Should this optional param, if I were to set it, be the same as the log directory I set in run_test.sh?

2. root_dir What does it mean by benchmark targets?

3. targets, target_model What does it mean by targets to compare each other? For target_model, should this be set to model_iter149999.hdf5?

4. In statistics.py, there is a def acquire_results( ) requiring a path to _fusionattributes.hdf5 which I seem to be missing. Are there steps that I skipped before I should run miscellaneous.py?

Edit: I ran analysis.sh and it required a main.py which is missing from the master copy. I edited from train.py to only require --phase and output-model-path but I was not successful.

[yuxiaoguo]

For fair comparison, the ground truth masks, to identity the empty/non-empty/surface grids, are copied from SSCNet. To make it work, you should configure the SSCNet and then run its download_data script. Then, modify the path of DEFAULT_GT in statistics.py to the SSCNet repo root. Finally, we could run the evaluation with command: python miscellaneous.py --option criterion --logdir eval --benchmark nyu

Some more tips:

• As we said in our article, for NYU dataset, the total training iterations is about 4000~ and pick the best results, evaluated from checkpoints saved per 200 iterations.
• All results reported in paper are pre-trained with SUNCG dataset. I suggest you ask for the permission of SUNCG dataset from its author.