search

twdb / sdi

Reader for Speciality Devices Incorporated (SDI) depth sounder binary format
BSD 3-Clause "New" or "Revised" License
0 stars 1 forks source link

change easting northing filter to use line fit instead of median #26

Open bklappauf opened 10 years ago

bklappauf commented 10 years ago

there are still bad navigation points getting through the current filter and I think it is difficult to choose an appropriate threshold when comparing to the median because it will depend on the size of the lake and the length of the line. One option would be to fit a line to the northing easting data for each survey line and filter by distance from the line. Then the threshold is much easier to select since it will represent some presumed maximum variation of the sensor from the survey line during the survey. Also it would be possible to estimate a replacement point on the line if desired instead of Nan.

wilsaj commented 10 years ago

In https://github.com/twdb/sdi/pull/29 , I used a simple approach of replacing the fixed threshold with 5 * std and it seems to work pretty well. I spent a little bit of thresholding based on a linear fit but the boat tracks are sometimes non-linear near the ends or in shallows, so that assumption throws out a lot of good data, and picking an appropriate neighborhood to sub-select for outliers was tricky. I couldn't get a linear fit to perform better than the 5 stds approach, but I only tried fiddling with various nth order polynomial fits and thresholds on residuals. I'll leave this ticket open in case someone wants to explore it further.