search

mitroadmaps / roadtracer

MIT License
235 stars 76 forks source link

Understanding Junction metric #24

Closed gpascualg closed 5 years ago

gpascualg commented 5 years ago

Hello,

While testing if my graph format was correct I decided to try the Junction metric of one graph with itself. I would expect, correct me if I'm wrong, for the metric to report 100% correct. Instead, this is what I'm getting:

343 total, 267 correct, 0 wrong, 79 extra, 76 missed ;;; 266 correct, 80 extra

With the following SVG outputs:

Ground Truth: truth

Inferred ground

Is there any reason for such behaviour? I attach my graph file (extension changed to .txt, otherwise github won't upload it):

region_3_3.txt

It is a 4096x4096 pixels region, data obtained form OSM and discretized (each node is at a 20 pixels distance of another).

uakfdotb commented 5 years ago

Sorry for slow response and thanks for linking the graph file that you're testing on.

If the two input graphs are the same, the junction metric should report error close to zero, but it may not be exactly zero because of the randomized junction clustering step.

However, it certainly should not be 23% as in your case.

The issue is your graph is directed. The junction metric code assumes the graph is bidirectional, an assumption we make because we do not believe road directionality can be inferred from satellite imagery.

After adding a "truth.MakeBidirectional()" and "inferred.MakeBidirectional()" call after reading the graphs in main function in junction_metric.go, I get this output:

477 clusters in a 481 clusters in b 409 total, 409 correct, 0 wrong, 3 extra, 0 missed ;;; 408 correct, 3 extra

As you can see the randomized clustering produced more clusters in one graph than the other, but the error is <1%.

gpascualg commented 5 years ago

Don't worry. I see, creating a directional graph was indeed not intended. That makes much more sense now, thank you very much! I am closing this issue, it's resolved