search

fermyon / spin

Spin is the open source developer tool for building and running serverless applications powered by WebAssembly.
https://developer.fermyon.com/spin
Apache License 2.0
5.02k stars 242 forks source link

Support graph-DBs #2561

Open hpvd opened 2 weeks ago

hpvd commented 2 weeks ago

Since more data in graph shape is being used, it would be handy to make this easily possible also with spin 1) similar to sqlite: an embedded property graph database e.g. kuzudb https://github.com/kuzudb/kuzu 2) BYO Graph DB e.g. falkorDB (the successor of redisgraph, same team...) https://github.com/falkordb/falkordb or good old neo4j

imho the most important thing for a broad adaption should be choosing the ones supporting opencypher / GQL (Graph Query Language) as query language. This has just (April 2024) become the new ISO standard for graph querying (graph peer to sql) https://jtc1info.org/slug/gql-database-language/

itowlson commented 2 weeks ago

Thanks for sharing this idea - very intriguing!

I can definitely see the attraction to having an embedded graph database in the same way as we have SQLite. The challenge would be portability to the cloud: either we would need to partner with a cloud provider running a hosted form of the same database (perhaps a Redis host with Falkor support?), or we'd have to allow backing it onto a BYO database (which could have compatibility implications).

BYO databases avoid this problem, but my understanding is that - as with SQL - standardisation applies only to the query language, not the protocol. (Although perhaps I am wrong. I see Falkor works with the Redis client: is that usual?) That would mean we would need to pick one or two, a la Postgres and MySQL, and provide a host service interface for those, until Wasm network suppoirt is pervasive enough for people to use the standard clients.

One thing I noticed: since Falkor works with Redis clients, it seems like it should already be usable via Spin's Redis API, using the execute API (e.g. https://docs.rs/spin-sdk/latest/spin_sdk/redis/struct.Connection.html#method.execute, https://fermyon.github.io/spin-python-sdk/v3/wit/imports/redis.html#spin_sdk.wit.imports.redis.Connection.execute). Not friendly, for sure, as you'd need to parse the results of execute yourself, but perhaps a starting point?

hpvd commented 2 weeks ago

Thanks for sharing this idea - very intriguing!

thanks for getting back on this!

I can definitely see the attraction to having an embedded graph database in the same way as we have SQLite. The challenge would be portability to the cloud: either we would need to partner with a cloud provider running a hosted form of the same database (perhaps a Redis host with Falkor support?), or we'd have to allow backing it onto a BYO database (which could have compatibility implications).

falkordb and kuzudb are fresh approaches (teams) and could be a good candidates to show interest in being part of this.

kuzu seems already having (very) first contacts to the wasm world, e.g.

BYO databases avoid this problem, but my understanding is that - as with SQL - standardisation applies only to the query language, not the protocol. (Although perhaps I am wrong. I see Falkor works with the Redis client: is that usual?) That would mean we would need to pick one or two, a la Postgres and MySQL, and provide a host service interface for those, until Wasm network suppoirt is pervasive enough for people to use the standard clients.

yep you are absolutely right. Opencypher / GQL (Graph Query Language) is the query language. I was referring to this because one may choose DBs for first implementations/Demo (embedded and BYO) supporting this, since it has the widest adaption in graph world. And its probably the easiest to start with.

For the connection there is the bolt standard which is used by many graphdbs:

One thing I noticed: since Falkor works with Redis clients, it seems like it should already be usable via Spin's Redis API, using the execute API (e.g. https://docs.rs/spin-sdk/latest/spin_sdk/redis/struct.Connection.html#method.execute, https://fermyon.github.io/spin-python-sdk/v3/wit/imports/redis.html#spin_sdk.wit.imports.redis.Connection.execute). Not friendly, for sure, as you'd need to parse the results of execute yourself, but perhaps a starting point?

interesting point!

hpvd commented 2 weeks ago

maybe also helpful as starting point: there is a rust bolt client https://docs.rs/bolt-client/latest/bolt_client/

hpvd commented 2 weeks ago

better:

This project aims to provide a comprehensive set of libraries that allow for interaction with graph database servers that support the Bolt protocol... This set of libraries allows interacting with servers supporting versions 1 through 4.4 of the protocol

https://github.com/0xSiO/bolt-rs