GPX import simplification

[Firefishy]

Currently we import all trackpoints in any GPX traces uploaded to the OpenStreetMap.org website. There has been a large increase in GPX traces uploaded to OSM in the last user. Particularly by sunnypilot and dragonpilot. This is causing the OpenStreetMap postgresql database to grow faster then planned.

The GPX traces are often overly dense or noisy.

The importer should be enhanced to simplify or be selective in choosing which points to import into the OpenStreetmap database.

The import code runs as a rails job. The importer logic is here.

[tomhughes]

Actually the main loop is in the model: https://github.com/openstreetmap/openstreetmap-website/blob/aceae23c8eca7a7f1b593bc1cbe15f100e022fcb/app/models/trace.rb#L214

It's a bit tricky because currently the points are iterated multiple times I think - once to create the points in the database and once to create each image which to be fair is already a bit rubbish.

I guess we only have to simplify the database objects so we could leave them all in for the images.

[mmd-osm]

I'm posting a link to the GPSbabel GPS track simplification for reference purposes: https://github.com/GPSBabel/gpsbabel/blob/cd3cd130541f7a72706120511156c6a61e3bbcc8/smplrout.cc ... That's the implementation I recommended in https://github.com/sunnyhaibin/sunnypilot/issues/201

Since porting this stuff to Ruby is probably a lot of effort, maybe it's worthwhile exploring some alternatives, like https://github.com/odlp/simplify_rb

I would also try to run the simplification first, and then feed the results into both image generation and db update. This way it's a bit easier for our users to spot really weird results due to the simplification.

Maybe the import confirmation email should also include a hint about the simplification, such as the original and final number of GPS points.