search

FundingCircle / jackdaw

A Clojure library for the Apache Kafka distributed streaming platform.
https://fundingcircle.github.io/jackdaw/
BSD 3-Clause "New" or "Revised" License
369 stars 80 forks source link

Consider splitting up test namespaces into another project? #197

Open vincentjames501 opened 5 years ago

vincentjames501 commented 5 years ago

When we included jackdaw in our project our uberjar size grew about ~20-30mb (most of that is Kafka libs of course). jackdaw is bringing in aleph and transitively manifold which appears to only really be used in the test related namespaces (the admin namespace uses manifold in a few spots which seem easy to remove) increasing our jar ~5MB. It would be nice to bring in a dependency like [fundingcircle/jackdaw-test "0.6.8"] instead? I can of course exclude these dependencies which I have, just a thought!

cddr commented 5 years ago

It was suggested when we began the project that the test component could be housed elsewhere. However, there was quite a strong desire for jackdaw to include tools that could be used for testing as well as implementing a kafka based system.

As you've noted, manifold is also used by the admin ns which fuzzes the issue slightly as there are now 2 top-level namespaces that would break if you exclude aleph when depending on jackdaw (i.e. jackdaw.admin and jackdaw.test). Not sure it would be that easy to remove though for someone who doesn't know the underlying Java primitives that well :-)

I think if anything, there's some useful async code under the test namespace that should probably be slightly refactored and moved into a non-test namespace and made available for folks that want to use the kafka client in an asynchronous fashion.

AndreaCrotti commented 1 year ago

+1 on this, but manifold is not a massive issue, the problem is more that aleph brings in netty as well. This could be particularly annoying if you already use a different version of Netty in your application.

Aleph could still be an optional dependency and the test machine could maybe only work if that's available? Moving it out might be better but it's not really the only possibility.

AndreaCrotti commented 1 year ago

From this simple attempt https://github.com/FundingCircle/jackdaw/pull/360 just doing that change makes a 4MB difference in the produced jar, but well more importantly it doesn't include netty anymore.