search

datafusion-contrib / datafusion-objectstore-s3

S3 as an ObjectStore for DataFusion
Apache License 2.0
59 stars 13 forks source link

Update to DataFusion 7.0.0. #43

Closed matthewmturner closed 2 years ago

matthewmturner commented 2 years ago

Currently based on the hash for the release candidate https://github.com/apache/arrow-datafusion/commit/ca765d54dda6114da55ece8d876c042eca3ea870. Once 7.0.0 is released will update to point to that.

@seddonm1 FYI

seddonm1 commented 2 years ago

Nice. Much more succinct. Unlucky the 7 release doesn't have the objectstore changes. But still ready for release 👍

matthewmturner commented 2 years ago

Agree...given how early this project still is do you think it would be terrible to update to using the hash as dependency again before the 8.0.0 release?

Or maybe we can temporarily create and maintain new branch like datafusion-latest that would use a datafusion hash while leaving main to be based on 7.0.0?

Just as some additional context, I am planning on adding writing capabilities to ObjectStore in the near future and of course would want to extend that here as well.

seddonm1 commented 2 years ago

I think as long as the crates release points to a stable Datafusion release (7.0.0) and we have an appropriate release (and tag) on here then main can easily point to the next release as we know upstream APIs are still in flux.

matthewmturner commented 2 years ago

I think as long as the crates release points to a stable Datafusion release (7.0.0) and we have an appropriate release (and tag) on here then main can easily point to the next release as we know upstream APIs are still in flux.

okay, that sounds good!

matthewmturner commented 2 years ago

With datafusion 7.0.0 now released I'm going to go through final review and publish. Let me know if any more feedback!