Closed byydzh closed 1 year ago
Hi,
Thank you for your testing, In the training, we perform diverse data augmentation in view points, grasping approaches and background to improve the generalizability, and performed uniform data preprocessing on two benchmarks. Hence the model should behaves similarly on the two benchmarks.
In the final version, we re-evaluated all metrics several times and tided the code to ensure consistency. It might be that the model are incorrected uploaded from the same experiments. We will re-examinate these checkpoints soon. However, the performance should not be differring too much based on pervious observations.
Please let me know if you have other concerns.
I see. Then I am thinking that your one pre-training model can achieve the results of the paper on two datasets at the same time, so will separate training get better results for separate datasets?
Hi,
Adding more data will in general improve the performance. However, since the augmented data are much more and diverse compared to the raw training data, from my recall we didnt observe significant performance difference in our first and final version.
What you are talking about in your paper is the results obtained by training separately. So you mean that there is no difference between the metrics of DexYCB obtained by training the DexYCB dataset alone and the metrics of DexYCB after training two data sets at the same time? which seems strange.
Hi,
I meant by "no significant performance" not "identical", the actual metrics can differ but the conclusion of comparing with related works in the paper remains unchanged.
thank you for the explanation, another question is do you have any plans to release the data augmentation code or related datasets?
Hi,
These parts come with the train code, which I'll try best finding time to tidy up and release should I am available.
Meanwhile, the detailed processes are described in the paper should you wish to re-implement. As they are generated online with the training, there are no data saved.
According to the two pre-trained models (snapshot_ho3d.pth.tar, snapshot_dexycb.pth.tar) you provided, the measured results are in line with the results mentioned in the paper. However, the two pre-trained checkpoints are identical except for the name, and cross-testing got the same results. This seems to indicate that this is the result of your single training, I am puzzled by this. Does that mean there's something special about its training?