SolidBench / rdf-dataset-fragmenter.js

Fragments an RDF dataset into multiple parts
MIT License
3 stars 8 forks source link

Add fragmentation strategies for dataset summary generation #18

Closed surilindur closed 2 months ago

surilindur commented 4 months ago

This is an initial draft pull request for adding fragmentation strategies that will generate dataset summaries using the VoID vocabulary and the custom membership filter vocabulary for Bloom filters. The PR includes the following changes:

The following other changes are also included, but I can try to move them to another PR if they feel unrelated:

Any feedback is welcome! I will keep this as a draft until everything has been fixed.

rubensworks commented 4 months ago

At first glance, this looks pretty good, nice work @surilindur!

However, as you suggested, it would be better to have the repo cleanup stuff (like linting) moved to a separate PR. It makes it quite difficult to review the changes otherwise. Could you look into that? We could either first have the cleanup PR, and then the additions, or the other way around, that's up to you.

In any case, I would wait with merging the summary additions PR until we know for certain that @constraintAutomaton's shapes will fit into this framework as well.

surilindur commented 4 months ago

The PR has been split now, but I noticed there is an error in the way the vocabulary is used for the Bloom filters. I will fix it before converting this into not-a-draft PR.

Edit: Also, the other PRs could be merged earlier.

surilindur commented 4 months ago

This is now rebased on the work in #20 because it helps make development easier

coveralls commented 4 months ago

Pull Request Test Coverage Report for Build 8924226253

Details


Totals Coverage Status
Change from base Build 8813392341: 0.0%
Covered Lines: 887
Relevant Lines: 887

💛 - Coveralls
surilindur commented 4 months ago

This still needs some fixes for the use of the membership vocabulary, I just noticed... :sob: I will also keep this as a draft until the shape generation has been tested to work with the same idea.

rubensworks commented 4 months ago

Ok, no rush! :-)

surilindur commented 4 months ago

This should be ready for review whenever there is time for it. I managed to set up the benchmarks using the branch from the fork, so there is no hurry.

surilindur commented 2 months ago

This is now ready for review again! I have tested that it works with SolidBench.

rubensworks commented 2 months ago

Thanks! Released as 2.7.0.