search

anilbatra2185 / road_connectivity

Improved Road Connectivity by Joint Learning of Orientation and Segmentation (CVPR2019)
MIT License
116 stars 34 forks source link

How to evaluate DeepGlobe graph with APLS metric? #13

Closed tansor closed 5 years ago

tansor commented 5 years ago

Hi, thank you for your response. I wonder how to evaluate the inferred graph using APLS metric? What data structure do you use to evaluate, and what's the key attribution that the data structure must have? The reason why I ask so is that, according to https://github.com/CosmiQ/apls, the evaluation metric seems to be connected with lon/lat coordinate or utm coordinate. Source codes may help more! Thx!

anilbatra2185 commented 5 years ago

Hi @tansor, thanks for the query. I used the simple skeletonization of roads and convert to line-string formats. You can refer Spacenet-Road Challenge 2 winners source code. Evaluate the APLS metrics by APLS Visualizer tool with source code available at link, which is released as part of challenge.

tansor commented 5 years ago

Hi @anilbatra2185 , thanks for your answer. I'm still confused about how to obtain the DeepGlobe gt graph. Because the test method only allows gt graph to be networkx pkl or geoJson.

parser.add_argument('--test_method', default='gt_pkl_prop_pkl', type=str,
                        help="Method for creating ground truth and proposal "
                        + "graphs. Options:"
                        + " gt_pkl_prop_pkl   = ground truth pickle, proposal pickle"
                        + " gt_json_prop_pkl  = ground truth json, proposal pickle"
                        + "gt_json_prop_wkt  = ground truth json, proposal csv in wkt format"
                        + "gt_json_prop_json = ground truth json, proposal json"
                        )
anilbatra2185 commented 5 years ago

Hi @tansor, I used the same steps of skeletonization and conversion to line-strings using the DeepGlobe segmentation GT mask. And now it will act as GT graph for deepglobe and you might need to do small changes in APLS visualizer to compare deepglobe GT and predicted graph.

STEPS for DG

Let me know, if you see any issues/concerns.

tansor commented 5 years ago

Thank you for your patience! You have solved my confusion. This issue can be closed now.

heeji289 commented 5 years ago

I also wonder how you evaluate DeepGlobe dataset with APLS metric. I tried to do that via code provided by "https://github.com/CosmiQ/apls", but I failed to use it on DeepGlobe dataset.

Please give me some details.. I read above talks, but I still didn't understand.

tansor commented 5 years ago

Hi, @hjlim0314. The author did not use the code provided by "https://github.com/CosmiQ/apls", but a java implementation called APLS Visualizer tool in "https://drive.google.com/file/d/1rwbj_o-ELBfruPZuVkCnEQxAX2-Pz5DX/view?usp=sharing", which is released as part of the SpaceNet challenge.

heeji289 commented 5 years ago

@tansor Thank you for your tip! It will be very helpful

antran89 commented 4 years ago

Hi @hjlim0314, @tansor I just made some modifications for DeepGlobe APLS metric measurement. Please use if needed, https://github.com/antran89/road_visualizer

heeji289 commented 4 years ago

Hi @hjlim0314, @tansor I just made some modifications for DeepGlobe APLS metric measurement. Please use if needed, https://github.com/antran89/road_visualizer

Thank you for your code. It works on my deepglobe data set very well. I appreciate you notice me about that.