test: add integration test with minio

[abyssnlp]

Description

Adds integration tests with s3-compatible MinIO.

• docker/docker-compose.yaml spins up the minio container
• docker/copy_tables.sh creates the bucket; unzips, copies and creates the tables in the minio store
• The environment variables for the integration test are supplied via integration_test/env.toml
• Adds running integration tests on pull requests via ci.yml

Notes:

• We can extract the logic to run the integration tests in the Makefile or as a separate script
• The integration test takes around ~3 minutes to complete
• The integration test is only for the hudi-rs rust support via datafusion

This closes #81 .

How are the changes test-covered

• [ ] N/A
• [x] Automated tests (unit and/or integration tests)
• [ ] Manual tests
  • [ ] Details are described below

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 0% with 8 lines in your changes missing coverage. Please review.

Project coverage is 86.82%. Comparing base (3359e10) to head (5a747fd). Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/tests/src/lib.rs	0.00%	4 Missing :warning:
integration_test/tests/hudi_test.rs	0.00%	4 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #112 +/- ## ========================================== - Coverage 87.42% 86.82% -0.61% ========================================== Files 14 15 +1 Lines 700 736 +36 ========================================== + Hits 612 639 +27 - Misses 88 97 +9 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[xushiyan]

@abyssnlp don't think the ci job is triggered. probably format issue.

[abyssnlp]

@xushiyan The CI is failing due to this error: dtolnay/rust-toolchain@1.75.0 is not allowed to be used in apache/hudi-rs. I used this action to setup the rust toolchain. Is there a way to whitelist this action or should I use an alternative?

[abyssnlp]

I've removed the external action since GHA runner has rustup already.

[abyssnlp]

@xushiyan All tests (including integration) are green. The labeler is failing due to permissions. I've made the small fix but let me know if this should be done in a different PR.

[abyssnlp]

@xushiyan Addressed all comments in a28debc.

I've reused the TestTable enum but I had to add an extra method to get the s3 path as we unzip and load the tables into minio for the integration tests so we don't need to extract them.

[xushiyan]

@abyssnlp are you planning to resume this work?

[abyssnlp]

Hi @xushiyan,

I've been busier than usual so couldn't get around to this earlier. I've addressed your comments in https://github.com/apache/hudi-rs/pull/112/commits/5a747fd1dc45fccf85cfdcad6fd8ae5068fc8993.

• The integration tests now run in a container
• I'll add a test for python as well later

Let me know what you think.

[xushiyan]

Ok, due to time constraint, I'll push this to the next release.