search

thoth-station / document-sync-job

Sync Thoth documents to an S3 API compatible remote
GNU General Public License v3.0
0 stars 5 forks source link

Perform document sync only for documents that are recent #15

Closed fridex closed 2 years ago

fridex commented 2 years ago

This introduces a breaking change

Description

With this change, we will be able to instrument the document sync job to sync only documents that are no older than X days. This is an optimization for the cronjob. Ex. we can have the cronjob running @daily and reduce CPU time occupied to tell it to sync documents that were created past 7 days (as an example). This will reduce the need for iterating over all the documents each time the cronjob runs; eventual cronjob misses will be handled properly by the overlaping dates (@daily vs. past X days sync).

fridex commented 2 years ago

/approve

sesheta commented 2 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fridex, mayaCostantini

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/thoth-station/document-sync-job/blob/master/OWNERS)~~ [fridex] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment