New library: LDkit - Githubissues

karelklima commented 6 months ago

Hi there!

First of all - thank you for building rdfjs.dev, it is a great resource for developers starting with semantic web technologies, and a good way to showcase existing projects and libraries.

Second - I might have an interesting addition to the existing mix of the libraries (disclaimer - I am the author and maintainer). I would like to suggest to add LDkit, an object graph mapping toolkit for TypeScript developers. It is a library that lets you read and write RDF using simplified TypeScript schemas, providing end to end type safety and pretty good developer experience. Effectively it translates RDF nodes to JavaScript/TypeScript primitives (even dates, which is pretty cool!) You can think of it as an ORM library, but for RDF. I created the library to make it easier for web app developers to work with RDF data, which is a long standing problem in the semantic web area.

Website / docs: https://ldkit.io/ GitHub: https://github.com/karelklima/ldkit

LDkit supports any SPARQL endpoint data source out of the box, and it is fully compatible with Comunica, making it compatible with any kind of data source that Comunica supports (including in-memory database).

To showcase the capabilities of LDkit, I have created several working examples that are available on GitHub, the most advanced being an IMDb-like movie database that uses DBpedia as the data source, and supports searching and browsing movies, actors, directors, etc.: https://github.com/karelklima/ldkit/tree/main/examples/next-movie-database

I have recently released a 2.0 version of the library that came up with new features like filtering and working with arrays - two things that are notoriously challenging for anyone who works wit RDF.

Thank you for your consideration to add LDkit to rdfjs.dev, I will be happy to help with anything that is required to make the addition.

megoth commented 5 months ago

Thank you for your kind words, and thank you for letting me know about your library. It sounds like it's a good match with rdfjs.dev ^_^

I will need some time to review it and get some demos working, but from skimming the website I believe it should have what I need to make it work.

karelklima commented 5 months ago

Sounds great, let me know if you have any questions or need any assistance :)

megoth commented 5 months ago

Hey, just FYI: I've started work on the library page at: https://github.com/megoth/rdfjs/tree/library/ldkit

I'm having problems retrieving the profile data by using Persons.findByIri though (as seen in L32 in the local demo and L21 in the Solid demo) - maybe you can see what I'm doing wrong? I'll revisit the branch in the near future, but wanted to give you a heads up.

Some context to the branch; I'm creating a local demo and a Solid demo:

The local demo loads a local turtle-string into memory and saves it to the browser's local storage.
The Solid demo requires the user to authenticate with their Solid profile, after which I retrieve the WebId graph, and stores it to the online resource when updating.

I'm planning to use Comunica's QueryEngine for the local demo (since it needs to work with N3.Store) and the internal QueryEngine for Solid, but for the in-progress work I'm only using Comunica.

karelklima commented 5 months ago

Hi, thank you for looking into LDkit! I have fixed the local demo - basically just changed the PersonSchema a bit: https://github.com/megoth/rdfjs/compare/library/ldkit...karelklima:rdfjs:library/ldkit

I also added updating of data back to the N3 store through LDkit and logging of SPARQL queries that LDkit performs agains the data store.

I have ran into one issue though when I was implementing Persons.update query. If I tried to update the name of the profile to the same value that already existed in the store, the store got deleted completely.

Example SPARQL that LDkit generates:

DELETE {
<https://megothcapgemini.solidcommunity.net/profile/card#me> <http://xmlns.com/foaf/0.1/name> ?v1 .
}
INSERT {
<https://megothcapgemini.solidcommunity.net/profile/card#me> <http://xmlns.com/foaf/0.1/name> "New Name" .
}
WHERE {
<https://megothcapgemini.solidcommunity.net/profile/card#me> <http://xmlns.com/foaf/0.1/name> ?v1 .
}

Seems like when the "New Name" is equal to ?v1, the triple to insert does not get inserted. I believe that this is a valid query though, and possibly a bug in Comunica, I will dive into that further.

As a simple workaround, I have added a simple check that runs Person.insert query if there are no persons in the store.

Regarding the Solid demo - I will look into that tomorrow.

karelklima commented 5 months ago

Update: looks like the problem I mentioned is indeed a bug in Comunica, I will try to fix that: https://github.com/comunica/comunica/issues/1301

karelklima commented 5 months ago

I have improved the local demo and have fixed the Solid demo.

The local demo has a workaround because of the aforementioned bug in Comunica (that I have fixed, waiting for merge and release).

The Solid demo has a workaround as well for something else that also seems to be a bug in Comunica - will deep dive into that next.

https://github.com/megoth/rdfjs/compare/library/ldkit...karelklima:rdfjs:library/ldkit

karelklima commented 5 months ago

Seems like the Solid issue was a problem with my profile card and mismatched URLs.

I was using solidcommunity.net as my Solid pod host. I have created an account there in 2018 and have apparently missed the migration from the original solid.community domain to solidcommunity.net domain. During the migration my profile card was apparently not updated, so it is essentially invalid. So much for persistent URLs :-)

My current profile is: https://karelklima.solidcommunity.net/profile/card#me But the IRIs in the document are: https://karelklima.solid.community/profile/card#me

Hence, I have created a fresh testing Solid pod and everything works well there - e. g. I was able to retrieve my name from the pod.

Regarding Comunica - as far as I am aware you need to use that with LDkit in order to query Solid pods. LDkit creates SPARQL queries that Comunica can apply on various data sources. The build in query engine in LDkit only supports SPARQL endpoints.

I will create a PR with my changes.

megoth commented 3 months ago

Thank you for your work, and apologies for my late response. I haven't been feeling great for a while, and haven't had time to work on any of my open source projects. But now I want to get into it a bit again (before I leave for vacation at least).

I've merged your PR, and continuing the work to write documentation. I've also fixed so that the Solid profile gets updated when the user submits the form. (Check the latest update on library/ldkit.)

I'm unsure about emptyStore in the local demo, could you explain the need for that? I've tried removing it, and it doesn't seem to produce any bugs. I'll leave it for now, but I want to remove it if it's not strictly needed.

karelklima commented 2 weeks ago

@megoth Hello, I hope you are doing better now! This time it's me who need to apologize for the late response, I haven't been feeling great as well.

Thank you for the merge and for the changes that you've made.

Regarding the emptyStore in the local demo:

The store is only created once when the component module is being loaded.
On render, there is an effect that runs each time the turtle content in the local storage changes (as it is a dependency on the React.useEffect).
When the new turtle is received, it needs to be added to the store, which may include outdated data (e.g. when a name changes). For example, there may be an existing triple <$PROFILE_URI> foaf:name "megoth" and a new incoming triple <$PROFILE_URI> foaf:name "new megoth". If the store wasn't emptied, it would contain both triples after the operation. The application may seem to work even after that, because LDkit picks only one value based on the way the Persons schema is defined, and by chance it can be the right updated value.
Therefore, unless I'm missing something, the store needs to be emptied every time before loading the new turtle content, which in itself represents the whole dataset.

Let me know how can I help further!

karelklima commented 2 weeks ago

In addition, I have fixed an error in Comunica that required workaround in the LDkit local example, so we can now remove that workaround: https://github.com/comunica/comunica/issues/1301

megoth / rdfjs

New library: LDkit #8