p2panda / aquadoggo

Node for the p2panda network handling validation, storage, aggregation and replication
GNU Affero General Public License v3.0
69 stars 5 forks source link

HTTP routes to serve files with correct content type and etag headers #544

Closed adzialocha closed 1 year ago

adzialocha commented 1 year ago

This PR switches from using symlinks and the ServeDir struct from tower to HTTP routes which resolve the blobs manually and respond with the correct content types.

Notable changes:

Closes #541 and #501

📋 Checklist

adzialocha commented 1 year ago

Observing some weird race-conditions (locally and in CI) where files do not exist yet when running fs methods. After changing this to tokio::fs (making it async) it became more stable .. I'm not exactly sure but tokio seems to bring different guarantees with its methods.

By the way, this is an interesting method I've stumbled upon: https://docs.rs/tokio/latest/tokio/fs/struct.File.html#method.sync_data - don't think we need it (we're fine with the OS managing files however it wants, as long as we can read from them), just good to keep in mind that file systems are weird.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 93.84% and project coverage change: +0.08% :tada:

Comparison is base (6c5d477) 92.42% compared to head (edc0b30) 92.50%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## development #544 +/- ## =============================================== + Coverage 92.42% 92.50% +0.08% =============================================== Files 105 105 Lines 17663 17932 +269 =============================================== + Hits 16325 16588 +263 - Misses 1338 1344 +6 ``` | [Files Changed](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda) | Coverage Δ | | |---|---|---| | [aquadoggo/src/config.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9jb25maWcucnM=) | `81.25% <ø> (ø)` | | | [aquadoggo/src/materializer/tasks/blob.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9tYXRlcmlhbGl6ZXIvdGFza3MvYmxvYi5ycw==) | `53.33% <52.17%> (-22.55%)` | :arrow_down: | | [aquadoggo/src/http/api.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9odHRwL2FwaS5ycw==) | `92.70% <93.58%> (+26.03%)` | :arrow_up: | | [aquadoggo/src/db/stores/blob.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9zdG9yZXMvYmxvYi5ycw==) | `97.38% <100.00%> (ø)` | | | [aquadoggo/src/graphql/mutations/publish.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL211dGF0aW9ucy9wdWJsaXNoLnJz) | `98.71% <100.00%> (+0.03%)` | :arrow_up: | | [aquadoggo/src/graphql/queries/collection.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3F1ZXJpZXMvY29sbGVjdGlvbi5ycw==) | `99.60% <100.00%> (ø)` | | | [aquadoggo/src/graphql/queries/document.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3F1ZXJpZXMvZG9jdW1lbnQucnM=) | `99.06% <100.00%> (ø)` | | | [aquadoggo/src/graphql/queries/next\_args.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3F1ZXJpZXMvbmV4dF9hcmdzLnJz) | `97.00% <100.00%> (ø)` | | | [aquadoggo/src/graphql/schema.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3NjaGVtYS5ycw==) | `92.79% <100.00%> (ø)` | | | [aquadoggo/src/graphql/tests.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3Rlc3RzLnJz) | `100.00% <100.00%> (ø)` | | | ... and [7 more](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda) | | ... and [2 files with indirect coverage changes](https://app.codecov.io/gh/p2panda/aquadoggo/pull/544/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.