seafloor-geodesy / gnatss

Community Seafloor Global Navigation Satellite Systems - Acoustic (GNSS-A) Transponder Surveying Software
https://gnatss.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
9 stars 12 forks source link

fix: Fix bug when running NDP1 example #252

Closed lsetiawan closed 2 months ago

lsetiawan commented 2 months ago

This PR fixes a bug that @johnbdesanto found as we're running the program for testing:

1) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 500 --from-cache --remove-outliers config.yaml
2) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 30 --from-cache --remove-outliers config.yaml
          -This should run w/out issue, remove 500 threshold outliers and generate new outliers.csv file w/ 30 threshold outliers
3) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 30 --from-cache config.yaml
          -This should run w/out issue, outliers not removed since --remove-outliers flag not called, 30-threshold outliers saved to outlier.csv
4) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --from-cache config.yaml
          -This run claims 0 new outliers, but outliers.csv still has many 30-threshold outliers from previous run (maybe unintended, bug here?)
5) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --from-cache --remove-outliers config.yaml
          -This run removes outliers from 2 calls ago, but not every 30-threshold outlier removed
6) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 30 --from-cache config.yaml
          -This run does not remove outliers b/c --remove-outliers not set, generates new outliers.csv file w/ many outliers
7) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 100 --from-cache config.yaml
          -This run flags fewer outliers b/c of lax threshold (100 vs 30), 57 outliers flagged and outliers.csv file overwritten
8) gnatss run --extract-dist-center --extract-process-dataset --qc --distance-limit 150 --residual-limit 100 --from-cache --remove-outliers config.yaml
          -Black magic happens, points removed from data set, but same 57 outliers from previous run remain and are flagged again. Further repetitions continue to remove presumably wrong data.

Additionally, the handling of outliers is refined.