FundingCircle / jackdaw

A Clojure library for the Apache Kafka distributed streaming platform.
https://fundingcircle.github.io/jackdaw/
BSD 3-Clause "New" or "Revised" License
369 stars 80 forks source link

Word Count Modernization #191

Closed creese closed 5 years ago

creese commented 5 years ago

This PR updates the "Word Count" example.

Highlights:

When writing code, you can treat the topic metadata fake just like a map. When used with a "getter", it returns the topic metadata for the name supplied, EDN serdes for the key and value, and a partition count of one.

(def topic-metadata (new-fake-topic-metadata))
(:foo topic-metadata)
;; => {:topic-name "foo", 
       :partition-count 1,
       :replication-factor 1,
       :key-serde  #object[org.apache.kafka.common.serialization.Serdes ... ],
       :value-serde #object[org.apache.kafka.common.serialization.Serdes ... ]}

When in development, the topics are inferred from the topology. The dev workflow is largely unchanged. Simply jack in and type:

(reset)
;; => :reloading (word-count user)
;; => :resumed
(publish (:input topic-metadata) nil "all streams lead to kafka")
(publish (:input topic-metadata) nil "hello kafka streams")
(get-keyvals (:output topic-metadata))
;; => (["all" 1] ["streams" 1] ["lead" 1] ["to" 1] ["kafka" 1] ["hello" 1] ["kafka" 2] ["streams" 2])

The user namespace is part of the dev alias and is not loaded automatically. To apply the dev alias, add the following to your Emacs config:

(setq cider-clojure-cli-global-options "-A:server:client:dev")

or type C-u -A:server:client:dev when jacking in.

codecov[bot] commented 5 years ago

Codecov Report

Merging #191 into master will decrease coverage by 2.68%. The diff coverage is 18.69%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #191      +/-   ##
==========================================
- Coverage   81.43%   78.74%   -2.69%     
==========================================
  Files          40       41       +1     
  Lines        2391     2498     +107     
  Branches      149      149              
==========================================
+ Hits         1947     1967      +20     
- Misses        295      382      +87     
  Partials      149      149
Impacted Files Coverage Δ
src/jackdaw/streams/describe.clj 18.69% <18.69%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f404699...8c9e685. Read the comment docs.