rgommers / pixi-dev-scipystack

Experimental out-of-tree dev setup for NumPy, SciPy and related projects with Pixi
Other
6 stars 1 forks source link

Unable to find `blas` in array-api envs #11

Closed lucascolley closed 3 weeks ago

lucascolley commented 4 weeks ago

If I try to python dev.py build -C-Dblas=blas -C-Dlapack=lapack -C-Duse-g77-abi=true from an array-api or array-api-cuda env, I get the error:

Run-time dependency blas found: NO (tried pkgconfig)

scipy/meson.build:222:9: ERROR: Dependency "blas" not found, tried pkgconfig

If I instead run the same in a default env, the build passes. I am then able to switch to an array-api-cuda env and run CUDA tests (as long as I don't git clean -xdf in between).

rgommers commented 3 weeks ago

I'm not sure exactly what's missing, but I cleaned up the task organization in commit e7a21a8097 so that build and other commands wouldn't show up everywhere. It now looks like:

% pixi r -e array-api build

build: command not found

Available tasks:
    ipython
    test-cpu
    test-jax
    test-strict
    test-torch

% pixi task list -s

Tasks per environment:
----------------------
array-api-cuda:
doc: doc, ipython
blis:
default: bench, build, check-missingdeps, ipython, mypy, profile-build, test, wheel
pytorch: ipython, test-torch
jax: ipython, test-jax
mlx: ipython
array-api-strict: ipython, test-strict
lint: ipython, lint
netlib: ipython, test
array-api: ipython, test-cpu, test-jax, test-strict, test-torch
cupy:
mkl: ipython, test
bldcheck: ipython

The extra dependencies in the other envs should be runtime switchable without a rebuild, hence build exists only in the default env.

Running a test command in those other envs after a git clean -xdf (e.g., pixi r test-cpu) works as well for me.

What are you actually doing, a pixi shell -e array-api and then manually building?

lucascolley commented 3 weeks ago

Right, thanks for the explanation. The problem with my workflow was that I went into an array-api shell immediately then tried to do everything from there. In the future, I should build in the default env (which pixi run build will do as long as I'm not in a non-default env) then go from there. Got it, thanks!

rgommers commented 3 weeks ago

The problem with my workflow was that I went into an array-api shell immediately then tried to do everything from there.

Trying to turn it back into "now I have an activated env" I guess? (I tried that too) It doesn't exactly work like that, although I can't yet explain what is different (maybe activation scripts or some such thing). Using the task system is better.

Note that global envs that are more conda/mamba like, which sometimes one needs, will be coming in the future: https://pixi.sh/latest/design_proposals/pixi_global_manifest/.