search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
13.88k stars 3.38k forks source link

[CI][Packaging] Re-synchronize conda_recipes with arrow-cpp-feedstock #42111

Open amoeba opened 3 weeks ago

amoeba commented 3 weeks ago

Describe the enhancement requested

According to https://github.com/apache/arrow/issues/41856#issuecomment-2133932916, we need to re-sync the recipes from https://github.com/conda-forge/arrow-cpp-feedstock into this dev/tasks/conda_recipes in order to get the nightlies building and publishing to the arrow-nightlies channel.

Component(s)

Continuous Integration, Python

amoeba commented 3 weeks ago

It looks like I could base the changes I make off of https://github.com/apache/arrow/commit/c9674bcc16411d7ecfd3b5587f544758b9fc7062.

Edit: https://github.com/apache/arrow/pull/37624 is a better link

kou commented 3 weeks ago

@h-vetinari Can we create a script to automate this? For example, we have https://github.com/apache/arrow/blob/main/ci/conan/merge_upstream.sh for ci/conan/.

h-vetinari commented 3 weeks ago

The custom adaptations in the recipe should be doable to script, the adaptations in the build scripts are harder, unless there's a good way to factor out the arrow-specific bits into a hook that can just be inserted somewhere (ideally at the end).

Possible definitely, but perhaps not easy (and progressively less easy the more past 80% you want to automate).

kou commented 3 weeks ago

Thanks. It makes sense. We don't need to automate all tasks if it's not easy. If there are easy parts, we can automate them to reduce maintenance cost.