zazuko / barnard59

An intuitive and flexible RDF pipeline solution designed to simplify and automate ETL processes for efficient data management.
26 stars 2 forks source link

sparql-http-client v3 #282

Closed tpluscode closed 5 months ago

changeset-bot[bot] commented 6 months ago

🦋 Changeset detected

Latest commit: d8c0c06a562d7f4afa5ca163efbf2072dcd072b0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages | Name | Type | | --------------------- | ----- | | barnard59-graph-store | Major | | barnard59-sparql | Minor | | barnard59-base | Patch | | barnard59-shacl | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 95.95960% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 85.43%. Comparing base (19dfb85) to head (d8c0c06). Report is 1 commits behind head on master.

Files Patch % Lines
packages/graph-store/post.js 84.61% 2 Missing :warning:
packages/graph-store/put.js 84.61% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #282 +/- ## ========================================== + Coverage 85.34% 85.43% +0.09% ========================================== Files 173 173 Lines 6801 6859 +58 Branches 102 102 ========================================== + Hits 5804 5860 +56 - Misses 995 997 +2 Partials 2 2 ``` | [Flag](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | Coverage Δ | | |---|---|---| | [barnard59](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `81.20% <ø> (ø)` | | | [barnard59-base](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `48.85% <100.00%> (+0.15%)` | :arrow_up: | | [barnard59-core](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `90.92% <ø> (ø)` | | | [barnard59-csvw](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `80.21% <ø> (ø)` | | | [barnard59-cube](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `96.39% <ø> (ø)` | | | [barnard59-formats](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `32.63% <ø> (ø)` | | | [barnard59-ftp](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `97.60% <ø> (ø)` | | | [barnard59-graph-store](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `95.10% <93.22%> (-0.58%)` | :arrow_down: | | [barnard59-http](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `87.50% <ø> (ø)` | | | [barnard59-rdf](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `94.95% <ø> (ø)` | | | [barnard59-s3](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `100.00% <ø> (ø)` | | | [barnard59-shacl](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `51.96% <ø> (ø)` | | | [barnard59-sparql](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `94.93% <100.00%> (+3.82%)` | :arrow_up: | | [barnard59-validation](https://app.codecov.io/gh/zazuko/barnard59/pull/282/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko) | `97.53% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=zazuko#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

tpluscode commented 6 months ago

@giacomociti the last failing tests are put/post should do nothing if the stream was closed and no quads have been written

Something must have changed in the library that a request is being sent anyway. I don't know if this OK and we can safely remove these tests or if it is an important regression 🤔

giacomociti commented 6 months ago

the tests were introduced in commit https://github.com/zazuko/barnard59/commit/c49bea0fc17a532b15c1cc7d56edabaa7fe27b2c but I'm not able to relate them to the commit message "fix: close writable if no quads have been written"

tpluscode commented 6 months ago

Looks like the test was added to simply match the implemented behaviour.

Guess it's a consequence of the stream splitter being removed. Previous version created a new request for subsequent graphs in data stream. No request if you had not a single quad to decide where to write to. Now it simply sends whatever comes its way.

On second thought, an empty stream is something that could be a legitimate output of a pipeline. POST would do nothing and PUT would clear the target graph?

graph-store is already a breaking release in this PR so might just go with what we have? Add a warning to the log if POST/PUT happens without any data?

giacomociti commented 6 months ago

in pipeline put.ttl the step op:graph-store\/put needs one more input argument:

[
      code:name "graph" ;
      code:value "graph"^^p:VariableName ;
],
tpluscode commented 6 months ago

in pipeline put.ttl the step op:graph-store\/put needs one more input argument:

will add an e2e test pipeline!