search

conda-forge / pyarrow-feedstock

A conda-smithy repository for pyarrow.
BSD 3-Clause "New" or "Revised" License
6 stars 26 forks source link

Nightly builds #96

Closed mrocklin closed 4 years ago

mrocklin commented 4 years ago

It would be useful to have nightly (or weekly) dev builds of Arrow. It is used by many projects these days, releases at a 3-month cadence, and is challenging for many of its users to build (at least the Python folks).

What would need to happen to have nightly builds here? cc'ing @jakirkham , and at his suggestion (I think), @conda-forge/core

mrocklin commented 4 years ago

Also, presumably this would also require nightly builds of arrow-cpp

kszucs commented 4 years ago

If conda-forge doesn't have a solution yet for publishing nightly packages then we could reuse our feedstock testing suite which are trying to follow the CF setup.

I suppose it would be the best to remain compatible with the CF channel, but for example our pinnings are not always up to date.

cc @xhochy

xhochy commented 4 years ago

We should upload the builds from our nightly crossbow jobs to a custom channel. These builds include all three/four of the packages we built based out of Arrow. We should be able to generate the pinnings on the fly instead of committing them to the repo.

I simply fail to setup the conda crossbow jobs on my fork, @kszucs I'll reach out to you when I find the time to retry this.

kszucs commented 4 years ago

Yeah, it requires a couple CI services. You can also use ursalabs/crossbow.

xhochy commented 4 years ago

Nightly builds are now available via conda install -c ursa-labs/label/nightly pyarrow arrow-cpp