When looking for cache results to be pruned, we execute this SQL query:
SELECT id FROM builds WHERE rc = 0 AND used < ? ORDER BY used ASC LIMIT ?
Here rc is the number of children a particular cache object has, and used is the timestamp of when that object was used. As soon as the next cache object starts building, it becomes dependent on the parent, but we only update rc when the build has finished and the object is saved to the database.
There is a map in_progress which holds the current builds. This PR adds a check to the prune function to query the in_progress operations to prevent the removal of these parents.
Depending upon the backend store, the delete operation may take a fraction of a second or many minutes; therefore, the query results are refreshed between each deletion. The query must return many results (not just one) as the in_progress check may filter some results. This prevents the prune from being blocked by a single active build with the oldest parent.
~before could be removed from the code in the future.
When looking for cache results to be pruned, we execute this SQL query:
Here
rcis the number of children a particular cache object has, andusedis the timestamp of when that object was used. As soon as the next cache object starts building, it becomes dependent on the parent, but we only updatercwhen the build has finished and the object is saved to the database.There is a map
in_progresswhich holds the current builds. This PR adds a check to the prune function to query thein_progressoperations to prevent the removal of these parents.Depending upon the backend store, the delete operation may take a fraction of a second or many minutes; therefore, the query results are refreshed between each deletion. The query must return many results (not just one) as the
in_progresscheck may filter some results. This prevents the prune from being blocked by a single active build with the oldest parent.~beforecould be removed from the code in the future.Addresses https://github.com/ocurrent/ocaml-ci/issues/873