Initial BoltDB support for #48.

[fluffle]

Initial BoltDB support for #48.

• Introduces a set of interfaces for DB access: Database, Collection and Key.
• Provides MongoDB and BoltDB implementations for Database and Collection.
• Only enough done to allow migration of config collection.
• Starts mapping MongoDB data model into BoltDB:
  • MongoDB "Collection" == BoltDB "Bucket".
  • Key is represented by an ordered list of string key:value tuples.
  • MongoDB reads/writes turn these into bson.M{key: value, ...} for Find/Upsert.
  • BoltDB reads/writes turn these into []byte{key + "\x1f" + value} nested bucket keys for Get/Put.
• Migration triggered manually via notice, logs any diffs as errors.
• Stolen patience diff algorithm from Bram Cohen \o/

[gundalow]

I don't know mongo, nor boltdb

None core bits look ok (documentation and .travis are inline)

Need to do backups

---

Review status: 0 of 24 files reviewed at latest revision, 1 unresolved discussion.

---

_main.go, line 36 [r1] (raw file):_ New boltdb needs adding to backup.sh

---

_Comments from the review on Reviewable.io_

[fluffle]

Oh, backups are an excellent point. Bolt maintains a lock on the db while sp0rkle is running so it's not possible to write an external tool to back up the DB. Their suggested approach is to simply write out a copy of the db from a transaction:

https://github.com/boltdb/bolt#database-backups

I should be able to do this relatively easily using a poller. It's also an incentive to fix some of the poller problems at some point, so they can be stopped/started manually if necessary.

---

Comments from the review on Reviewable.io

[gundalow]

Review status: 0 of 24 files reviewed at latest revision, 2 unresolved discussions.

---

README.md, line 37 [r2] (raw file): Is this note worthy?

Also maybe it should be rewritten as:

boltdb includes the 'bolt' cmdline

In case the order changes

---

Comments from the review on Reviewable.io

[fluffle]

Gonna merge this with master now the migration stuff is looking a bit less shonky.

Next up: stats package and figuring out how to abuse bolt buckets for indexes \o/

---

Review status: 0 of 25 files reviewed at latest revision, 2 unresolved discussions.

---

main.go, line 36 [r1] (raw file): Done.

---

_README.md, line 37 [r2] (raw file):_ Done.

---

Comments from the review on Reviewable.io

[gundalow]

Review status: 0 of 25 files reviewed at latest revision, 3 unresolved discussions.

---

.travis.yml, line 22 [r6] (raw file): Should this be

go get github.com/boltdb/bolt/...

---

Comments from the review on Reviewable.io

[fluffle]

Nah, the bolt tool is only vaguely useful for manual debugging, it's not used in the code anywhere... On 6 Mar 2016 13:22, "John R Barker" notifications@github.com wrote:

Review status: 0 of 25 files reviewed at latest revision, 3 unresolved

discussions.

.travis.yml, line 22 [r6] https://reviewable.io:443/reviews/fluffle/sp0rkle/62#-KCB4xE2KAurAo5Zzrha:-KCB4xE3Sur2S1j7X8yR:1006440590 (raw file https://github.com/fluffle/sp0rkle/blob/d3f8add11eb207da133d3132f596364ea9b8069e/.travis.yml#L22): Should this be

go get github.com/boltdb/bolt/...

---

Comments from the review on Reviewable.io https://reviewable.io:443/reviews/fluffle/sp0rkle/62

— Reply to this email directly or view it on GitHub https://github.com/fluffle/sp0rkle/pull/62#issuecomment-192887777.

[gundalow]

Reviewed 13 of 24 files at r1, 1 of 4 files at r3, 8 of 9 files at r5, 3 of 3 files at r6. Review status: all files reviewed at latest revision, 2 unresolved discussions.

---

Comments from the review on Reviewable.io