Gastove
commented
9 years ago Right. After chatting about this in Slack, I think we've come to consensus that we should attempt to use Postgres Full Text Search for this. This will:
- Give us a much simpler infrastructure (a server, a DB, and maybe a client, instead of a server, db, client, and search service)
- Keep debugging "in house" -- that is, while we have to write a lot more code, there are, in all, way fewer moving pieces.
- Maybe eventually let us contribute an open-source Clojure wrapper for PG Full Text Search -- there currently aren't any.
Doing this will be a hell of a lot easier if we stop using the permanent filesystem backend, and instead switch to storing everything in Postgres. Then, the FTS index can just be an index on the documents table, and queries can be done against it directly (potentially even in Korma). This will 100% require #26 to be done, also.
So, I'ma try and do this like so, in this order (though gh suppresses enumerations when you use tick boxes):
- [ ] Build a Postgres storage backend
- [ ] Implement dynamic routes on top of that backend
- [ ] Implement FTS on top of the new PG storage backend.
Oh god.
We need this
Yes, we sure do. RTD does search reasonably well.
But how
My current theory is Elastic Search. There's a Clojure wrapper for it; it can, by all accounts, handle HTML. Getting it configured will be.... somewhat of an adventure. Running this in-process would be really great, but it might also be impossible.