search

conda-forge / faiss-split-feedstock

A conda-smithy repository for faiss-split.
BSD 3-Clause "New" or "Revised" License
4 stars 15 forks source link

Arch Migrator #58

Closed regro-cf-autotick-bot closed 1 year ago

regro-cf-autotick-bot commented 2 years ago

This feedstock is being rebuilt as part of the aarch64/ppc64le migration.

Feel free to merge the PR if CI is all green, but please don't close it without reaching out the the ARM migrators first at @conda-forge/arm-arch.

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/3495928881, please use this URL for debugging.

conda-forge-linter commented 2 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham commented 1 year ago

@conda-forge-admin, please re-render

conda-forge-linter commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

conda-forge-linter commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham commented 1 year ago

@conda-forge-admin, please re-render

github-actions[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/faiss-split-feedstock/actions/runs/3842104858.

jakirkham commented 1 year ago

@conda-forge-admin, please re-render

jakirkham commented 1 year ago

cc @Ethyling

h-vetinari commented 1 year ago

@jakirkham, you might find some useful (if somewhat dated) commits for this in https://github.com/conda-forge/faiss-split-feedstock/pull/45

jakirkham commented 1 year ago

Thanks Axel 🙏

Have looked through that PR and it looks like this includes similar changes. The approach differs slightly (though the differences are largely cosmetic)

jakirkham commented 1 year ago

@conda-forge-admin, please restart CI

(Travis CI is having some issues cloning the PR for some reason. Hoping this clears that out)

jakirkham commented 1 year ago

The CUDA builds are taking longer than the CI limits. Had briefly explored other ways to improve this (using Ninja), but ran into some bugs (likely due to the CMake config from FAISS).

As the other archs (CPU only) already build ok, have reverted to just the changes to build those for now. That way we can hopefully get those in sooner.

There will be another bot PR for the CUDA arch migrator. We can follow up on other changes needed for that one.

jakirkham commented 1 year ago

It looks like all the jobs passed except one Linux x86_64 CUDA job that ran into the CI time limit. Guessing this just gets close and in this one case went over (based on looking at other similar jobs).

If the changes here seem reasonable, would suggest we merge and then I can babysit CI on main tomorrow to get the packages out.

h-vetinari commented 1 year ago

If the changes here seem reasonable, would suggest we merge and then I can babysit CI on main tomorrow to get the packages out.

Yeah, that's what's unfortunately necessary on this feedstock. There were some specific CUDA builds that always timed out, which have consequently been skipped. It's a pity, with an hour or so more we'd be fine here. Or if we made some progress on archspec, we wouldn't have to compile the code twice for x86 to have fat binaries for sse/avx2...

jakirkham commented 1 year ago

Thanks Axel! 🙏

Yeah no worries.

Interesting about the archspec and repeated compilation point. Maybe this is an angle worth looking at further. Have there been existing attempts here?

h-vetinari commented 1 year ago

Interesting about the archspec and repeated compilation point. Maybe this is an angle worth looking at further. Have there been existing attempts here?

Have a look in #23 and the links therein. This is definitely worth pursuing, but conda-forge infra isn't ready yet, and I've given up pinging on the respective issue (which isn't on this feedstock).

jakirkham commented 1 year ago

Thanks Axel! 🙏

Will take a look. Think one of the issues was we lacked the kind of plugin infrastructure we would want to start customizing Conda in the ways we would like. This was very recently introduced late last year. Please see this blogpost.

It may be more doable in the future, but would require more work in the tooling as well before conda-forge would see direct benefits.

On a different note, only a couple builds failed. Restarted them somewhat recently.

Have also refreshed the macOS ARM builds to use the changes from here simplifying that PR ( https://github.com/conda-forge/faiss-split-feedstock/pull/39 ).

In terms of CUDA arch builds, PR ( https://github.com/conda-forge/faiss-split-feedstock/pull/62 ) was recently opened by the migrator.