Investigate conflation output for Congo MGCP/OSM road segments

[mikejeffe]

Getting some wierd results when conflating the two input data sources in below issue. https://github.com/DigitalGlobe/dgis-devops/files/172679/NotConflatingRoad_clippedData.zip

To reproduce: 1) Ingest files in zip 2) Conflate Ref (MGCP) with Secondary (OSM) 3) View merged output.

It looks like the review threshold was not reached when conflating these but there were some merged segments where there was enough overlap between the two input layers. In general it doesn't look like a very good job when conflating the two inputs. Greedy and Average produce the same results because we haven't implemented an Average conflation strategy within the Unify code to support reviews.

Perhaps this is something @bwitham or @jasonsurratt can dig into when devops tasks subside.

[curranMapper]

Thanks @mjeffe98

Also wanted to add @njackson01 as this issue was pointed out by one of the VGI analyst working the current MGCP/OSM conflation project.

[curranMapper]

Wanted to add the attached pdf to this ticket. I think this helps confirm @mjeffe98 suspicion that it is a at least partly a thresholding issue. If that is the case, in order to mitigate issues like this for users in the future wanted to throw out some possible options and I can also get user feedback on:

1.If user does encounter this issue (road data not conflating bc the distance is to great), maybe data is significantly far off from each other geometrically that user should re-examine data accuracy or a red-flag that user may want to not conflate this data. (This would be leaving UI capability feature “as-is” and can be worked in on the pre-data processing slide and mitigated through training and workflow process papers)

2.Would tweaking “greedy” or “unify” in the Hoot UI be possible to allow user to extend threshold in instances such as this? Would doing so open up other issues user needs to consider? (ie Use at own risk) Is there a an existing tweak that is already in Hoot UI to accomplish this?

Conflation_ImprovedOutput.pdf

[jasonsurratt]

You can specify the circular error (CE95) of the features with the error:circular tag. This value is in meters. The larger the value the further out hoot will search. I haven't dug into the data, but in this case setting the CE on one input to 80m or more may have solved the problem.

[curranMapper]

nice, I am going to try this thur am, will update then

[curranMapper]

Re-ran on small sample set and this resolved issue. Applying this tag to entire cell and testing what happens (ie imagine intersections get more convoluted?) I will then test by applying to specific ways only which is probably the cleaner approach anyway.

[drew-bower]

Looking forward to hearing how it goes.

[curranMapper]

When applying tag to entire cell (with the threshold of 80m applied to entire dataset) I believe the double line issue is greatly mitigated when using “reference” conflation type.

Some issues may arise in urban areas when applying “average” conflation type surrounding intersections in dense areas. So if user were to apply an increased threshold in the future, it seems like a good idea to only apply to rural areas only (leave urban at default) if running "average" conflation type.

This is based on small sample with unique distance, AO, and conflation type. These 3 variables will always have to be considered by users when applying this manually applied extended threshold.

CongoRoadConflation_ManualThresholding.pdf

[curranMapper]

Customer is interested in having a dial to allow user to adjust the road threshold manually. Need to explore scenarios where user would not want to apply it, and how to then communicate current and future users as well. Interested if there are already known "gotchyas" that may occur when applying this and doing additional tests to better understand.

[jasonsurratt]

You can specify the highway search radius with search.radius.highway. Exposing that knob is a matter of specifying the correct value (in meters) from the UI. Similar settings are described below.

=== search.radius.default

* Data Type: double
* Default Value: `-1.0`

The default search radius to use when conflating features. If two features are within the search
radius then they will be considered for conflation.  If the value is -1 then the circular error will
be used to calculate an appropriate search radius.  Not all feature matching routines will honor
the default value.

=== search.radius.generic.line

* Data Type: double
* Default Value: `${search.radius.default}`

The search radius to use when conflating generic lines.  See `search.radius.default`.

=== search.radius.generic.polygon

* Data Type: double
* Default Value: `${search.radius.default}`

The search radius to use when conflating generic polygons.  See `search.radius.default`.

=== search.radius.highway

* Data Type: double
* Default Value: `${search.radius.default}`

The search radius to use when conflating highways.  See `search.radius.default`.

=== search.radius.waterway

* Data Type: double
* Default Value: `${search.radius.default}`

The search radius to use when conflating waterways.  Not used if
waterway.auto.calc.search.radius = true.  See `search.radius.default`.

[curranMapper]

Thanks @jasonsurratt. In the UI, under advanced conflation type/road options/ is changing the "Search Radius Highway" value to = 80 accomplish the same as error:circular tag = 80?

[jasonsurratt]

@sisskind, does this set search.radius.highway? If so, that gives the same effect as setting the error:circular tag on all input highways from both data sets.

[bwitham]

@curranMapper afaik, changing "Search Radius Highway" in the UI advanced options should set the corresponding internal hoot setting and, therefore, do what you expect it to in this case.

[curranMapper]

feature is already cooked in to UI then, awesome. Will discuss more with you Monday, and see if doing a short use case write up is beneficial. thanks @bwitham

[curranMapper]

When setting the conflation type to Average, the Road Options, Search Radius is not available to user in advanced setting (graphic below). However, when applying an 'error:circular' tag to the data directly, and then ingesting and run Average Conflation, it accomplishes the same thing as user would expect by adjusting "Search Radius" in the advanced options. Is there a reason why this advanced option is not available to user when selecting Average Conflation? Can it be?

[jasonsurratt]

Moved this into its own ticket.

https://github.com/ngageoint/hootenanny/issues/661

Is there anything left to be done with the original request for #461?

[bwitham]

661 looks to handle what remains. closing.