search

apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.31k stars 679 forks source link

Impl AsyncFileWriter by object_store #5766

Open Xuanwo opened 1 month ago

Xuanwo commented 1 month ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Follow up of https://github.com/apache/arrow-rs/issues/5738, add AsyncFileWriter support for object_store.

Describe the solution you'd like

I plan to add a ParquetObjectWriter like we do for ParquetObjectReader

Describe alternatives you've considered

None

Additional context

I'm willing to help implementing this.

Xuanwo commented 1 month ago

cc @alamb @tustvold, feel free to assign this task to me if you think it's a good idea.

alamb commented 1 month ago

Was this closed in https://github.com/apache/arrow-rs/pull/5753 ?

Xuanwo commented 1 month ago

Was this closed in #5753 ?

https://github.com/apache/arrow-rs/pull/5753 adds the trait for parquet, and this issue will impl trait for object_store.

tustvold commented 1 month ago

How are you getting on with this, is there any way I can assist? It would be good for this to make the release early next week

Xuanwo commented 1 month ago

How are you getting on with this, is there any way I can assist? It would be good for this to make the release early next week

Hi, apologies for the delay. So far, there are no known blockers; I'll provide updates here if any arise.

I aim to resolve this issue by the end of the week. If I'm unable to do so, feel free to take over next week.

Xuanwo commented 4 weeks ago

Hi @tustvold, I've reported a block issue here: https://github.com/apache/arrow-rs/issues/5834.

Xuanwo commented 2 weeks ago

Hi @tustvold, I've reported a block issue here: #5834.

https://github.com/apache/arrow-rs/issues/5834 has been fixed, I'm will come back to this issue now. I believe this one should be much simpler.