Closed DateBro closed 4 years ago
The report generated using retinanet is also weird.
Its config is almost the same as the former config, except estimate bbs = True
. During evaluating the AAE with retinanet, it printed a lot of 'no detection'. At first, I thought it was because some images don't contain obj_5. However, I found that the mAP of retinanet is lower than that in your paper, which is also mentioned in #60. I'd appreciate it if you could give some tips about the evaluation. 🙏
Translation Error Histogram:
Rotation Error Histogram:
Embedding PCA view sphere:
I found that I missed a configuration in train.config. After I added the configuration and retrained the AAE, the result is almost the as the former.
missing configuration:
ANTIALIASING: 8
CROP OFFSET SIGMA: 20
Translation and Rotation error is similar to the former. The PCA embedding view sphere is similar to your report in the docs directory.
I contacted with @qyz55 in issue #51. He told me he just followed the README and got a result similar to your paper. I redownloaded the repository and retrained the AAE but didn't get a satisfactory result. Could you share your training and evaluating configuration of obj5 or tell me something that may not be specifically listed in the readme? 🙏
I trained on obj_05 twice, with no changes to train_template.cfg except the model file, and got 60/66 vsd with gt bbox. Maybe you need to delete all the files in $AE_WORKSPACE_PATH/tmp_datasets to generate new images for training. Otherwise no differences can be seen by repetitive training.
I just glanced over it, but I think the configs seem ok and the translation errors without the bounding box detector also seem similar to the paper?
Sorry for not reading the paper of the arxiv version, which added some results such as translation error and rotation error. Besides, I forgot that the report in the docs directory is the result of ICP. The improvement of translation error which is generated by ICP is impressive. Thanks for your help!
Describe the bug I installed some libraries about latex and try to generate report.pdf of evaluating AAE with ground truth bounding boxes, but the result is weird. I retrained an Augmented Autoencoder, some of their results are similar, while others are a little different. Their translation error is far large than your report in doc directory. @MartinSmeyer Do you know where is the problem and what should I do?
System Info Describe the characteristic of your environment:
The configuration of the two reports is the same. Train config:
Evaluate config:
Additional context The translation error of the first report:![image](https://user-images.githubusercontent.com/32711044/70845546-05834d80-1e8b-11ea-8de0-f7bea74fc5f8.png)
The rotation error of the first report:![image](https://user-images.githubusercontent.com/32711044/70845587-8c382a80-1e8b-11ea-8d27-9444ca3dca7d.png)
The PCA embedding view sphere of the first report is:![image](https://user-images.githubusercontent.com/32711044/70845605-b2f66100-1e8b-11ea-9daf-221e359c2967.png)
The Recall vs translation error of the two reports are similar:![image](https://user-images.githubusercontent.com/32711044/70845621-dfaa7880-1e8b-11ea-9d90-25578e915bbb.png)
The translation error of the second report:![image](https://user-images.githubusercontent.com/32711044/70845549-1207a600-1e8b-11ea-9379-b7115b160747.png)
The rotation error of the second report:![image](https://user-images.githubusercontent.com/32711044/70845596-9c500a00-1e8b-11ea-9553-1547fd68fc1a.png)
The PCA embedding view sphere of the second report is:![image](https://user-images.githubusercontent.com/32711044/70845611-c30e4080-1e8b-11ea-8ad8-bc419f0a8643.png)