search

lnx-search / lnx

⚡ Insanely fast, 🌟 Feature-rich searching. lnx is the adaptable, typo tollerant deployment of the tantivy search engine.
https://lnx.rs
MIT License
1.25k stars 46 forks source link

Enable JSON type #155

Open mpr1255 opened 2 months ago

mpr1255 commented 2 months ago

Tantivy enables the json type as an indexable field. It seems LNX does not. Anyone got any appetite to do that? Would unlock a huge number of use cases.

ChillFish8 commented 2 months ago

Hey, it is unlikely for lnx to support JSON fields, in fact in future versions it is likely that dynamic schemas will be explicitly disallowed with a focus on strict schemas with the ability to update the schema on the fly at your convenience.

mpr1255 commented 2 months ago

Ah. Thanks for such a speedy response and being clear on the plan. Yes, my use case is a bit cruddish -- search backend for an LLM app. So users will do transformations on a field and save it back to the db in another field that then needs to be searchable. So the second scenario you mention would work. However I guess neither is implemented now anyway. We should probably look at ParadeDB or quickwit then I suppose. Too bad because I loved the yt vids. Such a good vibe from this software.

ChillFish8 commented 2 months ago

For that sort of application if you are mutating and changing document data to some unknown/arbitrary set of fields you may be better off using Meilisearch if you need typo-tolerance or strict low-latency. Otherwise yes Quickwit probably would be a good choice :slightly_smiling_face:

mpr1255 commented 2 months ago

Thanks SO much for the rec to Meilisearch. It had been vaguely on my radar but I hadn't done a deep dive. It is almost exactly what we're looking for in terms of features and data model. Fantastic. Most appreciated.