mozilla / ichnaea

Mozilla Ichnaea
http://location.services.mozilla.com
Apache License 2.0
575 stars 139 forks source link

Remove old datamap rows #492

Closed hannosch closed 7 years ago

hannosch commented 8 years ago

The map has changed a while back to only show dots for areas, which have been updated in the past year.

The underlying datamap_* tables still contain the rows for older times. And the datamaps script is pulling out all rows, independent of their modified date and putting them onto disk, as there is no database index on the modified column.

To make this more efficient, we should:

I did a manual clean-up of the tables to remove any rows with modified < '2015-11-01' and optimized the tables afterwards, so there isn't a big one-time backlog to process.

hannosch commented 7 years ago

Did one more manual cleanup, deleting all datamap_* rows with created < '2016-03-01` and modified < '2016-03-01'.