danielealbano / cachegrand

cachegrand - a modern data ingestion, processing and serving platform built for today's hardware
BSD 3-Clause "New" or "Revised" License
975 stars 34 forks source link

Implement a queue to track the epoch of the requested and completed operations #257

Closed danielealbano closed 1 year ago

danielealbano commented 1 year ago

The PR implements an operation queue to track the epoch of the operations and when they are completed.

This queue will be used by the workers to identify up to which point the epoch can be moved forward for the single workers and will be used in combination with the epoch gc.

Each time an operation is started an operation will be registered in the queue and when the operation will finish it will be marked as completed.

The worker then, randomly, will process the entities at the beginning of this queue to determine how many operations have completed and how much the queue can be moved forward.

The epoch operation queue uses a specialized 64bit ring unbounded queue using the MSB bit (bit n. 64) to indicate the completition and the remaining 63 bits to store the epoch, a 63 bits epoch is able to count about 100 years of runtime on a 3ghz cpu, should be plenty for the forseeable future.

codecov[bot] commented 1 year ago

Codecov Report

Base: 82.22% // Head: 82.28% // Increases project coverage by +0.06% :tada:

Coverage data is based on head (8ed9a4d) compared to base (4743eee). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #257 +/- ## ========================================== + Coverage 82.22% 82.28% +0.06% ========================================== Files 155 156 +1 Lines 9623 9656 +33 ========================================== + Hits 7912 7945 +33 Misses 1711 1711 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `82.28% <100.00%> (+0.06%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/danielealbano/cachegrand/pull/257?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano) | Coverage Δ | | |---|---|---| | [...unded\_queue\_spsc/ring\_bounded\_queue\_spsc\_generic.h](https://codecov.io/gh/danielealbano/cachegrand/pull/257/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#diff-c3JjL2RhdGFfc3RydWN0dXJlcy9yaW5nX2JvdW5kZWRfcXVldWVfc3BzYy9yaW5nX2JvdW5kZWRfcXVldWVfc3BzY19nZW5lcmljLmg=) | `98.39% <100.00%> (+0.31%)` | :arrow_up: | | [src/epoch\_operation\_queue.c](https://codecov.io/gh/danielealbano/cachegrand/pull/257/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#diff-c3JjL2Vwb2NoX29wZXJhdGlvbl9xdWV1ZS5j) | `100.00% <100.00%> (ø)` | | | [src/transaction\_spinlock.h](https://codecov.io/gh/danielealbano/cachegrand/pull/257/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#diff-c3JjL3RyYW5zYWN0aW9uX3NwaW5sb2NrLmg=) | `90.00% <0.00%> (-5.00%)` | :arrow_down: | | [src/intrinsics.h](https://codecov.io/gh/danielealbano/cachegrand/pull/257/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#diff-c3JjL2ludHJpbnNpY3MuaA==) | `100.00% <0.00%> (ø)` | | | [src/xalloc.c](https://codecov.io/gh/danielealbano/cachegrand/pull/257/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano#diff-c3JjL3hhbGxvYy5j) | `97.14% <0.00%> (+1.43%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Daniele+Salvatore+Albano)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.