robertluo / waterfall

The Unlicense
16 stars 1 forks source link

::consumer-config schema key is missing in kafka-cluster #25

Closed skydread1 closed 1 year ago

skydread1 commented 1 year ago

Problem

Evaluating a cluster which contains ::consumer-config triggers a malli error

If we evaluate the cluster in the waterfall ns comment, we can reproduce the error:

(require '[malli.dev]) 
 (malli.dev/start!)

  (def clu
    (kafka-cluster 
     {::nodes "localhost:9092"
      ::shapes [(shape/topic (constantly "sentence")) (shape/edn) (shape/value-only)]
      ::consumer-config {:position :beginning} ;; malli.core/extra-key
      ::group-id "tester1"
      ::topics ["sentence"]
      ::source-xform (map identity)}))
;=>
...
Errors:

  {:in [0 :robertluo.waterfall/consumer-config],
   :message "disallowed key",
   :path [0 :robertluo.waterfall/consumer-config],
   :schema [:map
            {:closed true}
            [:robertluo.waterfall/nodes :robertluo.waterfall/nodes]
            [:robertluo.waterfall/shapes
             [:vector [:fn #object["malli.core$_instrument$fn__21038@cd4fc92"]]]]
            [:robertluo.waterfall/producer-config
             {:optional true}
             :robertluo.waterfall/producer-config]
            [:robertluo.waterfall/group-id {:optional true} :string]
            [:robertluo.waterfall/topics {:optional true} [:vector :string]]
            [:robertluo.waterfall/source-xform {:optional true} fn?]],
   :type :malli.core/extra-key,
   :value {:position :beginning}}

Suggestion

Add the registry key to waterfall/kafka-cluster mall schema

robertluo commented 1 year ago

Thanks for the reporting and the fix.