Repoducing reported errors on 7 scenes dataset with pre-trained models

[hannastellmach]

Hi, I tried to reproduce the reported errors on the chess dataset using the pre-trained models. However, the performance does not match with the reported performance in your paper.

First, I created an lmdb dataset using the script caffe-posenet/posenet/scripts/create_posenet_lmdb_dataset.py. For the training set I used the dataset chess_train.txt and for the test set I used the dataset chess_val.txt from your 7Scenes_PoseNet_TrainVal folder. I downloaded the datasets from https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/.

Then, I created the dataset mean file using caffe-posenet/build/tools/compute_image_mean.

For testing I used your script caffe-posenet/posenet/scripts/test_posenet.py with your model (train_chess.prototxt, I changed the source and mean file to the appropriate directories) and your pre-trained weights (weights_chess.caffemodel) and set the iter to 2000 (as this is the size of the test set).

For testing on the chess dataset, I got: Median error 0.435215592384 m and 10.7102198545 degrees.

I would appreciate if you could give me some advises about possible mistakes or anything that I might missed.

Best, Hanna

[npiasco]

It seems that there was an error during the relative angle error computation before the ICCV paper submission: https://github.com/alexgkendall/caffe-posenet/commit/82de1236b5506f0b392fe20e10bae1e1d1960208 See this commit, a factor 2 was omitted