mark-thm
commented
11 months ago Minimal repro in #42
Open mark-thm opened 11 months ago
mark-thm
commented
11 months ago Minimal repro in #42
mark-thm
commented
11 months ago It looks like what's happening here is:
Also verified that pytensor already declares the numpy dependency https://github.com/mark-thm/rules_pycross/blob/64e9cfa8651f4d63607c1507a27122a6b115e615/examples/crossenv/example_lock.bzl#L501-L519 -- and the sandbox above shows the dependency is available
codebreach
commented
10 months ago based on the tree shared it seems that py3.9 was used but your error says using py3.10
i had a similar issue with grpc which was fixed by passing the python interpreter to the grpc repo. GRPC is a bazel compatible repo so that might not apply here but just wanted to point that out in case its relevant or helps!
http_archive(
name = "com_github_grpc_grpc",
# Due to https://github.com/grpc/grpc/issues/24665
# See https://github.com/grpc/grpc/issues/24665#issuecomment-1373577865
# This makes sure we use the right python version and not the local python version
repo_mapping = {"@local_config_python": "@python3_9"},
sha256 = "7f42363711eb483a0501239fd5522467b31d8fe98d70d7867c6ca7b52440d828",
strip_prefix = "grpc-1.51.0",
urls = [
"https://github.com/grpc/grpc/archive/refs/tags/v1.51.0.tar.gz",
],
)another thing to check is if you are using zig toolchain and if its correctly configured for linux arm64
jvolkman
commented
7 months ago It looks like what's happening here is:
- pytensor's setup.py loads numpy: https://github.com/pymc-devs/pytensor/blob/9ada9455f37479831b488b99efdba37b204d709a/setup.py#L2
- pycross is running the wheel build using exec-Python (Mac, arm64 or x86_64) (what else could it run?)
- numpy has been built for linux aarch64, the object files are present, but exec-Python doesn't recognize those files
Also verified that pytensor already declares the numpy dependency https://github.com/mark-thm/rules_pycross/blob/64e9cfa8651f4d63607c1507a27122a6b115e615/examples/crossenv/example_lock.bzl#L501-L519 -- and the sandbox above shows the dependency is available
Sorry for the way late reply. I agree that this is what is happening, and I'm not sure that there is a great answer. There are now hooks in pycross_wheel_build that would allow you to tweak the build environment before the build occurs. In the case of pytensor, numpy is only imported so that numpy.get_include() can be used. So one option would be to just insert a mock numpy module in PYTHONPATH that provides this method with the appropriate path.
You can see some examples of using hooks in the external_linking examples.
brians-neptune
commented
2 weeks ago cc_library/etc rules instead of the Python setup.py. I haven't looked at pytensor so I'm not sure if it makes sense here, I'm using depedencies that were already Bazel-ified.
I'm running into a pretty cryptic problem where building pytensor@2.12.3 fails with missing numpy C libraries even though building numpy directly succeeds. I have the pretty much same setup as the crossenv example but with a linux_aarch64 platform and toolchains defined, and have added pytensor to pyproject.toml/poetry.lock, re-generated the bzl lockfile. Then, building:
Fails with error:
Even though this succeeds:
If it helps, here's the contents of numpy from the sandbox
❯ tree ./bazel-out/darwin-fastbuild/bin/deps/numpy_1.25.2I'd add that pytensor has a declared dependency on numpy.