search

runabol / tork

A distributed workflow engine
https://tork.run
MIT License
599 stars 40 forks source link

Prune old nodes #267

Open runabol opened 1 year ago

runabol commented 1 year ago

The nodes table contain entries for both current online nodes and those that were online in the past. Over time the nodes table will grow indefinitely. A process should be put in place to prune old nodes -- say over 24 hours --- to prevent the table from getting too large.

AKARSHITJOSHI commented 1 year ago

Can we remove nodes where LastHeartbeatAt was 24 hrs ago ??

runabol commented 1 year ago

Yea, node IDs are generated on every startup of the Tork process and never reused. After 5 minutes a node is considered offline. 24 hours seems quite safe. If for some random reason a worker sends a hearbeat more than 24 hours later it will simply create the record again in the DB.

runabol commented 1 year ago

That being said, the TTL should probably be configurable in case certain administrators want to retain them longer for analysis purposes or some such thing.