search

oxigraph / oxigraph

SPARQL graph database
Apache License 2.0
1.06k stars 68 forks source link

Add full text search support to Oxigraph #48

Open Tpt opened 4 years ago

Tpt commented 4 years ago

It would be nice to have some kind of full text search support in Oxigraph. On the query level, it should probably be exposed as a special SERVICE to avoid having to tweak SPARQL grammar similarly to blazegraph.

On the implementation side, I would be more in inclined to write our own search system using existing components like the Snowball stemmer in order to keep having a single storage system (Sled/RocksDB). It would make things like transactions much simpler to implement.

ktk commented 4 years ago

There is no consensus yet but it's one of the topics discussed in the SPARQL 1.2 CWG: https://github.com/w3c/sparql-12/issues/40

Pretty much all implementations I've seen so far are using Lucene but that would drag the whole JVM stack with it I suppose.

Tpt commented 4 years ago

@ktk Yes, pulling a JVM does not seem a good way to go. The "search in Rust" area is quite active with Tantivy and MeiliSearch.

epoz commented 2 years ago

How about using the FTS support of sqlite? https://www.sqlite.org/fts5.html

Tpt commented 2 years ago

@epoz It would mean add a dependency on sqlite. That's definitely possible but maybe not the nicest way to do it. It also mean inheriting some sqlite constraints (single writer...). I am more considering integrating a Rust library (there are a few of them) or write our own implementation that would share Oxigraph main storage.

ktk commented 1 year ago

Just stumbled over this today. Will play with it the next weeks as we need a generic text search solution for facet stuff, SPARQL is not that great for it right now (paging, order, etc...): https://blog.meilisearch.com/v1-enterprise-ready-stable/

According to the Github "Languages" information, the search-part is pretty much all Rust, might be something to check out: https://github.com/meilisearch/meilisearch.

Not that this is priority now but just to make sure it's known here :)

ktk commented 1 year ago

Eh never mind, I just see you did mention this @Tpt , sorry about ;)

nikokaoja commented 10 months ago

What is the status of free text search in oxi?

Tpt commented 10 months ago

@nikokaoja Sadly no progress until now. Help welcome!