juxt / joplin

Flexible datastore migration and seeding for Clojure projects
Eclipse Public License 1.0
316 stars 49 forks source link

Cassandra Configuration Does Not Work With Given Readers #108

Open briprowe opened 5 years ago

briprowe commented 5 years ago

The cassandra configuration requires there be a vector of hosts:

{:migrators {:cass "migrations/migrators/cass"}
 :seeds {:cass "migrations.seeds.cass/run"}
 :databases {:cass-dev {:type :cass :hosts ["localhost"] :keyspace "foo"}
             :cass-stage {:type :cass :hosts ["10.10.10.10", "10.10.10.11", "10.10.10.12"]
                          :keyspace "foo"
                          :credentials {:user "something"
                                        :password "maybe-something-else?"}}}
 :environments {:dev [{:db :cass-dev :migrator :cass :seed :cass}]
                :stage [{:db :cass-stage :migrator :cass :seed :cass}]}}

But the readers for the tagged literals #env and #envf can only return strings.

This makes it hard to define migrations that need to discover the IPs dynamically (eg. when cassandra's running in a kubernetes cluster).

A possible solution would be to create a reader that lets client code supply a function that can fill in the data necessary. Suppose that reader's tag is #foo. Then the configuration data might look like:

{:migrators {:cass "migrations/migrators/cass"}
 :seeds {:cass "migrations.seeds.cass/run"}
 :databases {:cass-dev {:type :cass :hosts #foo my.dev.namespace/minikube-config :keyspace "foo"}
             :cass-stage {:type :cass :hosts #foo my.stage.namespace/lookup-cassandra-endpoints
                          :keyspace "foo"
                          :credentials {:user "something"
                                        :password "maybe-something-else?"}}}
 :environments {:dev [{:db :cass-dev :migrator :cass :seed :cass}]
                :stage [{:db :cass-stage :migrator :cass :seed :cass}]}}

The joplin reader function would then look something like:

(defn load-config [r]
  (edn/read {:readers {'env (fn [x] (System/getenv (str x)))
                       'envf (fn [[fmt & args]]
                               (apply format fmt
                                      (map #(System/getenv (str %)) args)))
                       'foo (fn [fn-symbol]
                              (require (symbol (namespace fn-symbol)))
                              ((ns-resolve *ns* fn-symbol)))}}
(PushbackReader. (io/reader r))))

Of course, #foo's not the best tag for this reader, but I can't think of a great one at the moment.

Thoughts?