Now I don't really like semantic search as a "search" method if it's the only possible option. A lot of the time, I know exactly what I'm looking for by keyword and I want keywords to behave exactly how I expect: if I search "webdev" I want posts with "webdev" to appear. Not necessarily posts that have "Website development" in the content but no explicit "webdev" token present.
On the other hand, semantic search can be quite useful for searching broad vibes, topics, ideas and even asking questions (all of which Weaviate supports quite nicely)
So, I think it's worth separating "search" from "semantic search" in a logical way for those two use-cases. On the API I don't think it really matters (two separate endpoints, or a simple query parameter that enables/disables semantic search) but the UI is the main challenge.
This search bar in the top should default to bog-standard, boring, normal, regular, keyword-search. Nothing fancy, no TF-IDF or LLM features. Sane defaults.
But the actual search page can have a larger search bar which could cover more options, enable semdex, use written questions, maybe even shudder generative chat-style responses?
That's a much bigger piece of work though, so I'd prefer to ensure search and semdex is separated as a concept and the current Search feature is implemented and polished to behave exactly like any user would expect it to behave in any other (non-AI) application.
Then, it's worth exploring what an AI version of that could look like as a logically separate feature.
Currently, search operates in two modes:
where col like %query%style queriesNow I don't really like semantic search as a "search" method if it's the only possible option. A lot of the time, I know exactly what I'm looking for by keyword and I want keywords to behave exactly how I expect: if I search "webdev" I want posts with "webdev" to appear. Not necessarily posts that have "Website development" in the content but no explicit "webdev" token present.
On the other hand, semantic search can be quite useful for searching broad vibes, topics, ideas and even asking questions (all of which Weaviate supports quite nicely)
So, I think it's worth separating "search" from "semantic search" in a logical way for those two use-cases. On the API I don't think it really matters (two separate endpoints, or a simple query parameter that enables/disables semantic search) but the UI is the main challenge.
This search bar in the top should default to bog-standard, boring, normal, regular, keyword-search. Nothing fancy, no TF-IDF or LLM features. Sane defaults.
But the actual search page can have a larger search bar which could cover more options, enable semdex, use written questions, maybe even shudder generative chat-style responses?
That's a much bigger piece of work though, so I'd prefer to ensure search and semdex is separated as a concept and the current Search feature is implemented and polished to behave exactly like any user would expect it to behave in any other (non-AI) application.
Then, it's worth exploring what an AI version of that could look like as a logically separate feature.