search

CallistoLabsNYC / samsa

Rust-native Kafka/Redpanda protocol and client implementation.
Apache License 2.0
11 stars 2 forks source link

Add integration tests for Compression, large reads and writes, and multiple partitions #80

Closed hgm-king closed 1 day ago

hgm-king commented 2 weeks ago

We want to add 5 new integration tests:

  1. Test the writing and reading using the compression setup
  2. Test reading and writing to a topic with 5-10 partitions
  3. Test writing and then reading 1M messages to a topic
  4. Test All 3 prior functionalities combined
  5. Test writing to a topic and then reading the responses (when the producer required_acks is 1)
morukele commented 2 weeks ago

Hey @hgm-king, I would love to work on this.

What do I need to get the test environment setup?

hgm-king commented 2 weeks ago

Hey @morukele you got it! The test environment is easy, it requires docker-compose and we use the make utility on unix for easy commands (see Makefile)

To setup, you simply run docker-compose up to spin up the redpanda cluster. Then you can run our examples and tests. Tests are ran with KAFKA_BROKERS=localhost:9092 KAFKA_TOPIC=test_topic cargo test --tests --all-features -- --show-output --test-threads=1.

I notice that 2 integration tests are failing so I will try and fix those now :)

morukele commented 2 weeks ago

@hgm-king Thanks for the explanation. I couldn't get it up and running when I tried it alone, but it works perfectly now.

It is late in my timezone, but I will do the test in the morning. This should give you enough time to fix the failing test.

dhonig commented 2 weeks ago

@morukele let us know how you get on with the tests and we can help you out.