search

fragmenta / query

An SQL query builder for golang
MIT License
41 stars 10 forks source link

pure golang based db #4

Open joeblew99 opened 8 years ago

joeblew99 commented 8 years ago

I think that integration of a pure golang DB and Indexer would be very nice. Bleve allows ALL content to be indexed fully and then easily found in the DB when needed. Its very nice for a CMS, because you can find everything. It could also be linked to Modules too.

Bleve: https://github.com/blevesearch/bleve/tree/master/index/store/boltdb http://www.blevesearch.com/

BoltDB: https://github.com/boltdb/bolt

Video tour of all this: https://www.youtube.com/watch?v=OynPw4aOlV0 Jump to 6 minutes to get into the stuff fast... :)

Bolt DB isa KV store that is 100% golang and runs on Windows too. Bleve is built on top of BoltDB.

So the idea is to put your data into BoltDB, and also put all the query meta data 7 indexing in Bleve. When doing any type of search (in the web ui) your hitting the Bleve system, which returns documentID's. A bit liek google returns web page Urls. Then the documents can be called for fro the BoltDB KV store.

BoltDB is not of course like a SQL DB, and its takes soem getting used to, but once you have a full query engine like Bleve, you tend to not need a full SQL DB.

Its more like how a CQRS Architecture works.

Anyway, please consider this. It would be a great marry up with your CMS.

thanks in advance, and happy to discuss this further if needed. I am gmt +1 Berlin

kennygrant commented 8 years ago

I agree boltdb would be a great addition, and would get rid of lots of issues with bootstrapping (the main issues people seem to have getting started are related to database setup). I'll definitely be looking into this and thanks for the pointers.

The only issues would be around relations and use of sql, but this would also be an interesting exercise - to either support boltdb in query, or have an example which doesn't use query but uses a simpler storage model instead. For a basic CMS the relations could be stored in a different way no doubt.

In principle though I think this would be really interesting to add and would welcome contributions on it.

joeblew99 commented 8 years ago

there is an ORM for boltDB called Storm. Its still a bit rough, but they are working on the query engine now. i am also working on it.

also with boltDB if you use atoms, then you can sync data Master to Master, by iintegrating Raft. Consul, as i guess you know is built this way.

food for thought as they say. This is part of my attraction to a KV store. You can do some things with it that are hard with a sql db

kennygrant commented 6 years ago

I think what I'm going to look at for this is the ql library, because it is pure go, and supports the sql driver interface, so it could be more easily switched out for a more performant relational database later on with (hopefully) no code changes, and an easy migration of data.

It now looks mature enough for production use.