search

nodestream-proj / nodestream

A Declarative framework for Building, Maintaining, and Analyzing Graph Data
https://nodestream-proj.github.io/docs/
Apache License 2.0
36 stars 11 forks source link

Fix `Target` and `Schema` bugs #284

Closed zprobst closed 5 months ago

zprobst commented 5 months ago

This PR Fixes two issues:

  1. If you have two pipelines; one that says a property is a key and another that says it only knows the property as a property, the order of operations takes precedence. This occurs when using a match only clause. If we know something is a key, it needs to be defined as such. Same thing with indexes.

  2. Currently we are passing writer configurations (such as batch_size) through the target. Currently these values are getting passed as part of the the connector configuration directly to the writer that is able to sort through these arguments and use them. However, with the addition of the Migrator, these configuration values are treated as invalid. This PR shuffles the writer configuration values as well the connector specific values away from one another and only sends the writer arguments when appropriate.

codecov[bot] commented 5 months ago

Codecov Report

Attention: Patch coverage is 94.73684% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 97.58%. Comparing base (5e099b0) to head (12a65b3).

Files Patch % Lines
nodestream/databases/writer.py 66.66% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #284 +/- ## ======================================= Coverage 97.58% 97.58% ======================================= Files 139 139 Lines 4588 4601 +13 ======================================= + Hits 4477 4490 +13 Misses 111 111 ``` | [Flag](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | Coverage Δ | | |---|---|---| | [3.10-macos-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.58% <94.73%> (+<0.01%)` | :arrow_up: | | [3.10-ubuntu-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.58% <94.73%> (+<0.01%)` | :arrow_up: | | [3.10-windows-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.53% <94.73%> (+<0.01%)` | :arrow_up: | | [3.11-macos-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.58% <94.73%> (+<0.01%)` | :arrow_up: | | [3.11-ubuntu-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.58% <94.73%> (+<0.01%)` | :arrow_up: | | [3.11-windows-latest](https://app.codecov.io/gh/nodestream-proj/nodestream/pull/284/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj) | `97.53% <94.73%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=nodestream-proj#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.