conda-forge / faiss-split-feedstock

A conda-smithy repository for faiss-split.
BSD 3-Clause "New" or "Revised" License
4 stars 15 forks source link

Rebuild for CUDA 11.8 support #70

Closed regro-cf-autotick-bot closed 3 months ago

regro-cf-autotick-bot commented 1 year ago

This PR has been triggered in an effort to update cuda118.

Notes and instructions for merging this PR:

  1. Please merge the PR only after the tests have passed.
  2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.


If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/6057259647, please use this URL for debugging.

Closes https://github.com/conda-forge/faiss-split-feedstock/pull/64

conda-forge-webservices[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham commented 1 year ago

@conda-forge-admin , please re-render

jakirkham commented 1 year ago

@conda-forge-admin , please re-render

jakirkham commented 1 year ago

Think we will want this logic as well ( https://github.com/conda-forge/faiss-split-feedstock/pull/64/commits/c2edc96077222ab030a1e3b93e15e8db079be96e )

Edit: Done in commit ( https://github.com/conda-forge/faiss-split-feedstock/pull/70/commits/805825f24ecafd1d4906175441420ca52ecde960 ) below

jakirkham commented 1 year ago

@conda-forge-admin , please re-render

github-actions[bot] commented 1 year ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/faiss-split-feedstock/actions/runs/7014307041.

jakirkham commented 1 year ago

It looks like TestClustering.test_ivf_train_2level failed on the CPU only build. Not sure what is going on there

Punting on the arch builds where we are running into CI limits. Likely need to trim GPU architectures there. Had tried pushing on cross-compiling as an alternative in the past without much luck ( https://github.com/conda-forge/faiss-split-feedstock/pull/62 )

h-vetinari commented 1 year ago

Something we can do (finally) is split off the AVX2 builds into separate jobs. It's on my backlog.

jakirkham commented 11 months ago

@conda-forge-admin , please re-render

jakirkham commented 11 months ago

@conda-forge-admin , please re-render

jakirkham commented 11 months ago

@conda-forge-admin , please re-render

jakirkham commented 11 months ago

@conda-forge-admin , please re-render

jakirkham commented 11 months ago

It looks like TestClustering.test_ivf_train_2level failed on the CPU only build. Not sure what is going on there

Looks like upstream relaxed the threshold of this test ( https://github.com/facebookresearch/faiss/pull/2927 ). Have added a patch doing the same

Punting on the arch builds where we are running into CI limits. Likely need to trim GPU architectures there. Had tried pushing on cross-compiling as an alternative in the past without much luck ( #62 )

Have disabled these above

jakirkham commented 11 months ago

On Windows CUDA 11.8, it appears the job hung waiting for input in the midst of running the tests

```python tests\test_fast_scan_ivf.py ............................................ [ 34%] Traceback (most recent call last): File "C:\Miniforge\Scripts\conda-mambabuild-script.py", line 9, in sys.exit(main()) File "C:\Miniforge\lib\site-packages\boa\cli\mambabuild.py", line 256, in main call_conda_build(action, config) File "C:\Miniforge\lib\site-packages\boa\cli\mambabuild.py", line 228, in call_conda_build result = api.build( File "C:\Miniforge\lib\site-packages\conda_build\api.py", line 253, in build return build_tree( File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3819, in build_tree test(pkg, config=metadata.config.copy(), stats=stats) File "C:\Miniforge\lib\site-packages\conda_build\build.py", line 3625, in test utils.check_call_env( File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 445, in check_call_env return _func_defaulting_env_to_os_environ("call", *popenargs, **kwargs) File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 416, in _func_defaulting_env_to_os_environ proc = PopenWrapper(_args, **kwargs) File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 286, in __init__ self.out, self.err = self._execute(*args, **kwargs) File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 360, in _execute self.disk = max(directory_size(disk_usage_dir), self.disk) File "C:\Miniforge\lib\site-packages\conda_build\utils.py", line 186, in directory_size out = subprocess.check_output(command.format(path), shell=True) File "C:\Miniforge\lib\subprocess.py", line 421, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "C:\Miniforge\lib\subprocess.py", line 505, in run stdout, stderr = process.communicate(input, timeout=timeout) File "C:\Miniforge\lib\subprocess.py", line 1141, in communicate stdout = self.stdout.read() KeyboardInterrupt ............................................................. !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! %PREFIX%\lib\site-packages\faiss\swigfaiss.py:5811: KeyboardInterrupt (to show a full traceback on KeyboardInterrupt use --full-trace) ========== 334 passed, 1 skipped, 4 deselected in 116.91s (0:01:56) =========== Terminate batch job (Y/N)? Entering debug mode. Use h or ? for help. At D:\a\_tasks\CmdLine_d9bafed4-0b18-4f58-968d-86655b4d2ce9\2.231.0\ps_modules\VstsTaskSdk\ToolFunctions.ps1:113 char:13 + if ($originalEncoding) { + ~~~~~~~~~~~~~~~~~ ^C [DBG]: PS D:\a\1\s>> ```

Are we running pytest with pdb enabled? If so, that might explain this behavior (and we should disable using pdb in CI to fix this)

jakirkham commented 11 months ago

It looks like TestClustering.test_ivf_train_2level failed on the CPU only build. Not sure what is going on there

Looks like upstream relaxed the threshold of this test ( https://github.com/facebookresearch/faiss/pull/2927 ). Have added a patch doing the same

Appears that was not enough

>       self.assertLess(ndiff, 51)
E       AssertionError: 52 not less than 51

On Windows CUDA 11.8, it appears the job hung waiting for input in the midst of running the tests

Sorry think I misread. It looks like CUDA 11.2 jobs for Windows & Linux exceed the CI time limit ( 6hrs )

jakirkham commented 9 months ago

@conda-forge-admin , please re-render

jakirkham commented 8 months ago

JFYI we are planning to drop CUDA 11.2 support in conda-forge soon

Please see this announcement and this issue ( https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/5339 ) for more details

The smoothest upgrade path would be to add CUDA 11.8. IOW completing this migration PR

Please let us know if you have any questions on next steps

jakirkham commented 7 months ago

JFYI CUDA 11.2 is now officially dropped

jakirkham commented 7 months ago

@conda-forge-admin , please re-render

conda-forge-webservices[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

conda-forge-webservices[bot] commented 7 months ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

jakirkham commented 7 months ago

@conda-forge-admin , please re-render