search

tabular-io / iceberg-kafka-connect

Apache License 2.0
192 stars 41 forks source link

Refactor integration tests to adopt other catalogs #91

Closed ajantha-bhat closed 11 months ago

ajantha-bhat commented 11 months ago

TestContext was a singleton class coupled with REST catalog. This is an attempt to make it more generic to plugin other catalog for test.

bryanck commented 11 months ago

Thanks for doing this. One change I'd like to see is to reuse the shared containers, i.e. MinIO, Kafka, Kafka Connect, so we only have the additional overhead of the extra catalog container per catalog type.

ajantha-bhat commented 11 months ago

Thanks for doing this. One change I'd like to see is to reuse the shared containers, i.e. MinIO, Kafka, Kafka Connect, so we only have the additional overhead of the extra catalog container per catalog type.

Done. I also thought of using UUID for table names if the tests are executed in parallel. But didn't do as the tests are passing.

bryanck commented 11 months ago

I think this looks good, thanks again @ajantha-bhat . It might need to be refactored a bit when we submit to the Iceberg but we can deal with that then.

ajantha-bhat commented 11 months ago

It might need to be refactored a bit when we submit to the Iceberg but we can deal with that then.

I didn't know contributing this project as a module to Iceberg is in the roadmaps. nice. Looking forward to it. This can definitely get better attention if it lives in Iceberg repo.