Code & Tests Refactoring
Some parts of the code could be easier to understand. Let's clean them up.
Add mocking capabilities between implementation parts to be able to test code without the need for setting up Kafka
Add more detailed descriptions to modules
Add named structs to handle internal implementations
API implementation
I've used protocols to mimic Factory patterns that can build & request & respond depending on input parameters.
Add new describe-groups request & response
Add list-offsets request & response templater
Add use protocols to handle multiple versions of each request & response
Dependencies
Bump min version of elixir to v1.7. Recently v1.14 has been released, and more & more apps have been deprecated. In release v1.0.0, I'm thinking about bumping the min version to v1.10.
Add hammox deps to make mocking for unit tests easier & type safe.
Github Actions Improvements
Add zookeeper health checks
Checklist
[ ] Test implementation on personal env with six brokers & ten consumer groups. MSK-based with Kafka v2.8 for a week.
What's Next
Migrate more requests to Kayrock in batches, two requests per PR, each time test it on env.
Split unit tests & integration tests into to commands & two actions in GitHub
To reviewers
Ok, this one is huge. Sorry about that, but I wanted to show you my idea, and I needed to show a bigger picture. Please review, comment & discuss. I currently have enough free time to push kafka_ex to v1.0.0. My idea is to create a branch
release-v1 in the kafka_ex repo and create PR there. Then I'll be able to make smaller PRs for you to review.
And once all is migrated, we can do one big PR with cleanup and merge it to the master.
To Discuss
Do you like the idea of protocols? I see it as over-engineering, and it looks weird for requests. But on the other hand, I'd like to have this app protocol agnostic at the end of migration.
Do you like the idea of the v1 branch? This will be a long process to migrate, and there could be a few breaking changes.
Can we bump the elixir version to at least 1.7, preferably 1.10? v1.6 is waaaay behind. Mor and more libs are stopping supporting that, and we have plenty of deprecation issues.
DO NOT MERGE
The first part of moving KafkaEx closer to v1.0.0
PR Content
Overview
Code & Tests Refactoring Some parts of the code could be easier to understand. Let's clean them up.
API implementation I've used protocols to mimic Factory patterns that can build & request & respond depending on input parameters.
describe-groups
request & responselist-offsets
request & response templaterDependencies
hammox
deps to make mocking for unit tests easier & type safe.Github Actions Improvements
Checklist
What's Next
To reviewers
Ok, this one is huge. Sorry about that, but I wanted to show you my idea, and I needed to show a bigger picture. Please review, comment & discuss. I currently have enough free time to push
kafka_ex
to v1.0.0. My idea is to create a branchrelease-v1
in thekafka_ex
repo and create PR there. Then I'll be able to make smaller PRs for you to review.And once all is migrated, we can do one big PR with cleanup and merge it to the master.
To Discuss