weso / wdsub

Wikidata Subsetting
MIT License
15 stars 5 forks source link

Combine this project with SparkWDSub #13

Open labra opened 3 years ago

labra commented 3 years ago

There is a sister project called SparkWDSub which generates the wikidata subsets using Apache Spark GraphX to represent the Wikidata graph and allows the generation of Wikidata subsets following ShEx-based graph traversal.

We were initially using WShEx as defined in this project but later created a different ShEx dialect called simpleShEx for that project. In principle those dialects should be the same and we should join them.

Another possibility would include the SparkWDSub code in this project and have a single project. We are keeping them separate by now because it is easy to spot errors. Once SparkWDSub is more stable, we could embed it here.