murrayds
commented
4 years ago In reference to the discussion in #13. Possible solutions listed at the bottom of this issue.
you may want to check out the points at an extremely small distance. Under -3, you're talking about literally the scale of meters.
Some geographic distances are also incredibly small, as in 2.22E-04km.
Sometimes, these seem to be centers or hospitals that are part of the university and that have roughly similar coordinates
1917 | Oxford University Hospitals NHS | 51.763873 | -1.219806 1921 | John Radcliffe Hospital | 51.763871 | -1.219806 111 | Technical University of Denmark | LYNGBY | 55.7856 | 12.5214 30807 | National Institute of Aquatic Resources | LYNGBY | 55.785574 | 12.521381
Options:
- Leave as is
- Chop off the decimal points on lat/long coordinates—less accurate but will mask small distances 3 decimal places seems to get us within ~100 meter accuracy.
- Threshold a certain distance, say 1km. Impute small distances with this minimum value.
- Remove comparisons at this small level
- Group together points that have the same city, and a small geographic distance (say 1km)—set their distance to 0.
@yy thoughts?
yy
After looking through the institution lookup file, not all geographic coordinates should be trusted. Some are flipped (i.e, longitude appearing in the latitude slot), and some just don't make sense at all.
We should validate these coordinates, and fix what we can. One way to do this efficiently/systematically would to group institutions by country, and using geographic distance, identify anomalous coordinates and follow up on them. After a quick scan of the data, this should be only the order of only about 100-300.