search

mozilla-services / autopush

Python Web Push Server used by Mozilla
https://autopush.readthedocs.io/
Mozilla Public License 2.0
215 stars 34 forks source link

Purge old `webpush` records from Router table #1382

Open jrconlin opened 4 years ago

jrconlin commented 4 years ago

As of April 2020, the Webpush router table is 1.33 TB with 6.3B rows.

There is a lot of old data in there.

Purging the old data would require a scan to kill all records that have a last_connect earlier than a given threshold (180d?) and have a router_type of webpush. The operation itself would not be cheap, but it would save the cost of a lot of no longer needed records.

"garbage collection"

jrconlin commented 4 years ago

Doing some back of the envelope cost calculations on this, it might be more expensive to purge the records than leave them. If we presume a 60% garbage rate, we'll recoup the costs of doing a delete after about 20 months. One problem, however is that the most recent burst of registration records, due to the churn from client rotating UAID on reconnect, would not be picked up by this purge.