jacobobryant / biff

A Clojure web framework for solo developers.
https://biffweb.com
MIT License
878 stars 43 forks source link

Run Biff without RocksDB? #113

Closed seancorfield closed 2 years ago

seancorfield commented 2 years ago

One of my systems is a very old Mac and RocksDB won't run on it:

Execution error (UnsatisfiedLinkError) at jdk.internal.loader.NativeLibraries/load (NativeLibraries.java:-2).
/private/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/xtdb_rocksdb-6.12.7/librocksdbjni-osx-x86_64.jnilib: dlopen(/private/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/xtdb_rocksdb-6.12.7/librocksdbjni-osx-x86_64.jnilib, 1): Symbol not found: __ZdlPvSt11align_val_t
  Referenced from: /private/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/xtdb_rocksdb-6.12.7/librocksdbjni-osx-x86_64.jnilib
  Expected in: /usr/lib/libc++.1.dylib
 in /private/var/folders/p1/30gnjddx6p193frh670pl8nh0000gn/T/xtdb_rocksdb-6.12.7/librocksdbjni-osx-x86_64.jnilib

Is there a way to run Biff locally for development without using RocksDB?

jacobobryant commented 2 years ago

Yes, you can override the default XT config by setting :biff.xtdb/opts in config.edn and have it use LMDB instead:

{...
 :dev {...
       :biff.xtdb/opts {:xtdb/index-store {:kv-store {:xtdb/module xtdb.lmdb/->kv-store
                                                      :db-dir "storage/xtdb/index-lmdb"}}
                        :xtdb/document-store {:kv-store {:xtdb/module xtdb.lmdb/->kv-store
                                                         :db-dir "storage/xtdb/docs-lmdb"}}
                        :xtdb/tx-log {:kv-store {:xtdb/module xtdb.lmdb/->kv-store
                                                 :db-dir "storage/xtdb/tx-log-lmdb"}}}}}

(:biff.xtdb/opts gets passed to biff/use-xt and then biff/start-node)

And update deps.edn:

{...
 :deps {com.biffweb/biff {...
                          :exclusions [com.xtdb/xtdb-rocksdb]}
        com.xtdb/xtdb-lmdb {:mvn/version "1.21.0-beta2"}}}

(I guess you'd want to leave com.xtdb/xtdb-rocksdb in if you use it in production though, which would be the case in my config example above since the lmdb changes have been put under the :dev section)

seancorfield commented 2 years ago

Works perfectly! Thank you. Yes, setting up deps.edn so it works differently for dev/prod via aliases would be the next step but this at least lets me play with the framework.

Is LMDB not recommended for production then?

jacobobryant commented 2 years ago

Great!

Is LMDB not recommended for production then?

In general I don't have any opinion on it beyond what the XT docs say about rocksdb vs lmdb. But now that you mention it, if you're using lmdb in dev, I probably would recommend also using it in prod to keep things the same.

seancorfield commented 2 years ago

Perhaps you could add a note to the Biff docs showing how to use lmdb instead of rocksdb?

jacobobryant commented 2 years ago

Just made a release which adds a :biff.xtdb/kv-store config option which can be set to :lmdb, to make this a bit easier. That also avoids a problem that I missed earlier: if you use the value of :biff.xtdb/opts I gave above in the :prod section and you set :biff.xtdb/topology :jdbc, the jdbc options will get clobbered. I added a Troubleshooting section to the docs with instructions for LMDB using the new option: https://biffweb.com/docs/#unsatisfiedlinkerror. And there's a new release here with upgrade instructions: https://github.com/jacobobryant/biff/releases/tag/v0.4.2-beta

seancorfield commented 2 years ago

Cool. Thanks. Will take another look at Biff in a week or two when I get another block of spare time.