search

apache / iceberg-rust

Apache Iceberg
https://rust.iceberg.apache.org/
Apache License 2.0
469 stars 95 forks source link

refactor: Remove dependency of tokio. #418

Open liurenjie1024 opened 1 week ago

liurenjie1024 commented 1 week ago

With the coming runtime module pr: https://github.com/apache/iceberg-rust/pull/233 we should carefully add runtime dependency.

We have only one part depending on tokio: https://github.com/apache/iceberg-rust/blob/3b8121eaa9e9628536093836dcc41119716afd9e/crates/iceberg/src/writer/file_writer/parquet_writer.rs#L278

The dependency of tokio in test code should be fine since it will not affect downstream users.

Xuanwo commented 1 week ago

Let's me do this. I feel like it's a good win for this to remove direct dep on tokio.

vivek378521 commented 15 hours ago

Is this issue open?

I saw the code, we can replace tokio async write with futures async write. I think poll_shutdown will be replaced with poll_close, if we have any plans in place already then let me know?

Xuanwo commented 14 hours ago

I saw the code, we can replace tokio async write with futures async write. I think poll_shutdown will be replaced with poll_close, if we have any plans in place already then let me know?

Hi, thanks a lot for the research. Feel free to take it.

vivek378521 commented 13 hours ago

Ah, the function: AsyncArrowWriter accepts a tokio async writer object, replacing it with futures won't work. I think I am trying to punch above my weight. Lol.

Xuanwo commented 12 hours ago

Ah, the function: AsyncArrowWriter accepts a tokio async writer object, replacing it with futures won't work. I think I am trying to punch above my weight. Lol.

Thanks for you try! I will take it then.