How to generate that BBOX.json file ?

[Mingxiao-Li]

Hi, I am wondering how did you generate the BBOX.json file ? As I found this BBOX is different from the BBOX of both reverie data v1 and reverie data v2.

[cshizhe]

It is the same as the reverie data v1.

[Mingxiao-Li]

It is the same as the reverie data v1. Thanks for the answer. I have another problem that is I got different evaluation results when I use your eval code and the eval system in eval.ai , have you ever got this problem ?

[cshizhe]

There is a slight different in SPL calculation where I take the length of provided path as groudtruth path length while the official eval code uses the length of shortest path. But since most the provided paths are the shortest path, this doesn't have much influence on the results.

[Mingxiao-Li]

There is a slight different in SPL calculation where I take the length of provided path as groudtruth path length while the official eval code uses the length of shortest path. But since most the provided paths are the shortest path, this doesn't have much influence on the results.

The biggest difference I got is the success rate and oracle success rate, while the RGS are the same. Did you get the same success rate ?

[cshizhe]

I got the same evaluation results before. What do you get? I guess the eval.ai may update the evaluation codes and use bbox v2.

[Mingxiao-Li]

I got a quite 5~8% lower in sr and osr in eval.ai. That is possible, but then it is impossible to get the test results on v1 version , which is bad. ~~

[cshizhe]

You could check with the authors of the reverie dataset to see if it is possible to get test results with v1 bboxes.

[Mingxiao-Li]

ok, thanks.