Sotera / watchman

Watchman: An open-source social-media event-detection system
GNU General Public License v2.0
20 stars 7 forks source link

Feature: prune db after X hours #95

Closed lukewendling closed 7 years ago

lukewendling commented 7 years ago

am i crazy or could we simply use mongo 3's TTL index... for all collections, keyed on the 'created' field, which we currently mix into all models. (the Timestamp mixin)

set all ttl indexes for ~48 hours so we'd be assured of a complete 1 day of prior data, with enough buffer, so we'd know all related records exist for previous day, even when events are created nearly a full day after smposts ... and we don't have to do anything else in code. just let mongo auto remove data, which it does in a background process with this feature.

https://docs.mongodb.com/v3.2/core/index-ttl/

@justinlueders @drJAGartner

lukewendling commented 7 years ago

superseded by #130