p2panda / aquadoggo

Node for the p2panda network handling validation, storage, aggregation and replication
GNU Affero General Public License v3.0
69 stars 5 forks source link

Reduce when `blob` task is issued and check for duplicate work #578

Closed sandreae closed 11 months ago

sandreae commented 11 months ago

This PR fixes a couple of logic errors in the issuing of blob tasks which meant already materialized blobs were being written to the filesystem multiple times.

All in all simpler logic :+1:

I think this may also be an issue in schema task. The reason being (actually for both cases, cos I copied the logic from there :rofl: ) that we used to not check and issue tasks for reverse relations, so to account for that we had to handle cases when a "child" of a system schema dependency came in and completed the dependency tree. Now that we do issue check for and issue reverse relation tasks we are in effect handling this case twice, which causes extra work.

closes: #574

πŸ“‹ Checklist

codecov[bot] commented 11 months ago

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (94bc8e9) 92.48% compared to head (acd517d) 92.66%. Report is 1 commits behind head on main.

:exclamation: Current head acd517d differs from pull request most recent head a3ececb. Consider uploading reports for the commit a3ececb to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #578 +/- ## ========================================== + Coverage 92.48% 92.66% +0.18% ========================================== Files 106 106 Lines 18396 18402 +6 ========================================== + Hits 17013 17052 +39 + Misses 1383 1350 -33 ``` | [Files](https://app.codecov.io/gh/p2panda/aquadoggo/pull/578?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda) | Coverage Ξ” | | |---|---|---| | [aquadoggo/src/materializer/tasks/dependency.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/578?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9tYXRlcmlhbGl6ZXIvdGFza3MvZGVwZW5kZW5jeS5ycw==) | `97.53% <100.00%> (ΓΈ)` | | | [aquadoggo/src/materializer/tasks/blob.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/578?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9tYXRlcmlhbGl6ZXIvdGFza3MvYmxvYi5ycw==) | `84.25% <84.61%> (+28.05%)` | :arrow_up: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/p2panda/aquadoggo/pull/578/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.