ArunaStorage / aruna

The data orchestration engine
https://aruna-storage.org
Apache License 2.0
16 stars 1 forks source link

Feat/nats improvements #171

Closed lfbrehm closed 5 months ago

lfbrehm commented 5 months ago

Summary

Adds missing event messages for server to server sync, additional bug fixes and improvements.

Changes

codecov[bot] commented 5 months ago

Codecov Report

Attention: Patch coverage is 18.84615% with 422 lines in your changes are missing coverage. Please review.

Project coverage is 31.08%. Comparing base (7861bc2) to head (165955a). Report is 60 commits behind head on main.

:exclamation: Current head 165955a differs from pull request most recent head f2704f5. Consider uploading reports for the commit f2704f5 to get more accurate results

Files Patch % Lines
...ts/server/src/middlelayer/presigned_url_handler.rs 0.00% 73 Missing :warning:
...nents/data_proxy/src/caching/grpc_query_handler.rs 0.00% 67 Missing :warning:
...onents/server/src/caching/notifications_handler.rs 0.00% 40 Missing :warning:
components/server/src/hooks/hook_handler.rs 0.00% 35 Missing :warning:
...ver/src/middlelayer/service_accounts_db_handler.rs 0.00% 34 Missing :warning:
...mponents/server/src/notification/natsio_handler.rs 47.36% 20 Missing :warning:
...omponents/data_proxy/src/grpc_api/proxy_service.rs 0.00% 19 Missing :warning:
components/data_proxy/src/database/persistence.rs 0.00% 16 Missing :warning:
components/data_proxy/src/structs.rs 0.00% 13 Missing :warning:
components/server/src/database/dsls/rule_dsl.rs 7.14% 13 Missing :warning:
... and 22 more
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #171 +/- ## ========================================== + Coverage 30.72% 31.08% +0.35% ========================================== Files 127 127 Lines 24139 23854 -285 ========================================== - Hits 7416 7414 -2 + Misses 16723 16440 -283 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

lfbrehm commented 5 months ago

I tried to address the failing tests situation. Object::batch_create was the culprit, but I do not know why. Maybe this is related to this issue

lfbrehm commented 5 months ago

I refactored the slightly ugly solution to skip the inner loops in Object::batch_create and InternalRelation::batch_create, as suggested by @St4NNi. I'm not quite happy with removing the COPY FROM queries, because they are suggested by the tokio_postgres maintainer as a performance optimized solution. In the long run, this should be debugged, replacing this temporary solution.