search

p2panda / aquadoggo

Node for the p2panda network handling validation, storage, aggregation and replication
GNU Affero General Public License v3.0
70 stars 5 forks source link

Integrate `Bytes` operation value #552

Closed sandreae closed 1 year ago

sandreae commented 1 year ago

MIGRATED TO: https://github.com/p2panda/aquadoggo/pull/554

📋 Checklist

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 87.79% and project coverage change: -0.17% :warning:

Comparison is base (a404d0c) 92.50% compared to head (47c902f) 92.34%. Report is 3 commits behind head on development.

:exclamation: Current head 47c902f differs from pull request most recent head b55ebcd. Consider uploading reports for the commit b55ebcd to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## development #552 +/- ## =============================================== - Coverage 92.50% 92.34% -0.17% =============================================== Files 105 106 +1 Lines 17932 18027 +95 =============================================== + Hits 16588 16647 +59 - Misses 1344 1380 +36 ``` | [Files Changed](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda) | Coverage Δ | | |---|---|---| | [aquadoggo/src/db/models/document.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9tb2RlbHMvZG9jdW1lbnQucnM=) | `100.00% <ø> (ø)` | | | [aquadoggo/src/db/models/operation.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9tb2RlbHMvb3BlcmF0aW9uLnJz) | `33.33% <ø> (ø)` | | | [aquadoggo/src/db/models/query.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9tb2RlbHMvcXVlcnkucnM=) | `100.00% <ø> (ø)` | | | [aquadoggo/src/db/stores/entry.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9zdG9yZXMvZW50cnkucnM=) | `99.40% <ø> (ø)` | | | [aquadoggo/src/db/stores/schema.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi9zdG9yZXMvc2NoZW1hLnJz) | `98.37% <ø> (ø)` | | | [aquadoggo/src/db/types/entry.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi90eXBlcy9lbnRyeS5ycw==) | `90.00% <ø> (ø)` | | | [aquadoggo/src/db/types/operation.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9kYi90eXBlcy9vcGVyYXRpb24ucnM=) | `88.00% <ø> (ø)` | | | [aquadoggo/src/graphql/queries/collection.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3F1ZXJpZXMvY29sbGVjdGlvbi5ycw==) | `99.60% <ø> (ø)` | | | [...ggo/src/graphql/scalars/document\_view\_id\_scalar.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3NjYWxhcnMvZG9jdW1lbnRfdmlld19pZF9zY2FsYXIucnM=) | `78.57% <ø> (ø)` | | | [aquadoggo/src/graphql/scalars/hex\_bytes\_scalar.rs](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda#diff-YXF1YWRvZ2dvL3NyYy9ncmFwaHFsL3NjYWxhcnMvaGV4X2J5dGVzX3NjYWxhci5ycw==) | `0.00% <0.00%> (ø)` | | | ... and [29 more](https://app.codecov.io/gh/p2panda/aquadoggo/pull/552?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=p2panda) | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sandreae commented 1 year ago

SQLite has BLOB type for binary data, Postgres has Large Object or bytea....

adzialocha commented 1 year ago

SQLite has BLOB type for binary data, Postgres has Large Object or bytea....

Time to drop support? 😥 Or we use hexadecimal strings for now until we find a better solution anyhow (see my comment below)?

adzialocha commented 1 year ago

Did you consider not storing blob pieces values at all in the database and retrieve the bytes from the file instead during replication?

https://doc.rust-lang.org/std/io/trait.Seek.html 😎 .. and interesting for writing: https://doc.rust-lang.org/std/io/struct.Cursor.html

I don't think it is too hard but also definitely not required for this development period, I've made a ticket here: https://github.com/p2panda/aquadoggo/issues/553

sandreae commented 1 year ago

Did you consider not storing blob pieces values at all in the database and retrieve the bytes from the file instead during replication?

https://doc.rust-lang.org/std/io/trait.Seek.html 😎 .. and interesting for writing: https://doc.rust-lang.org/std/io/struct.Cursor.html

I don't think it is too hard but also definitely not required for this development period, I've made a ticket here: #553

Yeh, this would be ideal. We still store the encoded operation in the db, so we wouldn't actually need to grab the bytes during replication either. Don't know if I want to tackle it right now though but it could be soon :+1: .