SomeoneSerge
commented
5 months ago Seems like we either need to find a way to disable tests at the cmake level, or we'll have to move gtest from nativeCheckInputs to buildInputs?
Open Cryolitia opened 5 months ago
SomeoneSerge
commented
5 months ago Seems like we either need to find a way to disable tests at the cmake level, or we'll have to move gtest from nativeCheckInputs to buildInputs?
autrimpo
commented
5 months ago We already disable tests at cmake level with onnxruntime_BUILD_UNIT_TESTS for CUDA builds. Changing it from doCheck to false continues with the cross-compile for me on the current master (80368c5). It fails at the end:
onnxruntime-aarch64-unknown-linux-gnu> [100%] Linking CXX shared library libonnxruntime.so
onnxruntime-aarch64-unknown-linux-gnu> /build/source/onnxruntime/python/onnxruntime_pybind_ortvalue.cc:11:10: fatal error: numpy/arrayobject.h: No such file or directory
onnxruntime-aarch64-unknown-linux-gnu> 11 | #include <numpy/arrayobject.h>
onnxruntime-aarch64-unknown-linux-gnu> | ^~~~~~~~~~~~~~~~~~~~~
onnxruntime-aarch64-unknown-linux-gnu> compilation terminated.
onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:132: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_ortvalue.cc.o] Error 1onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** Waiting for unfinished jobs....
onnxruntime-aarch64-unknown-linux-gnu> /build/source/onnxruntime/python/onnxruntime_pybind_mlvalue.cc:11:10: fatal error: numpy/arrayobject.h: No such file or directory
onnxruntime-aarch64-unknown-linux-gnu> 11 | #include <numpy/arrayobject.h>
onnxruntime-aarch64-unknown-linux-gnu> | ^~~~~~~~~~~~~~~~~~~~~
onnxruntime-aarch64-unknown-linux-gnu> compilation terminated.
onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:104: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_mlvalue.cc.o] Error 1
onnxruntime-aarch64-unknown-linux-gnu> /build/source/onnxruntime/python/onnxruntime_pybind_iobinding.cc:11:10: fatal error: numpy/arrayobject.h: No such file or directory
onnxruntime-aarch64-unknown-linux-gnu> 11 | #include <numpy/arrayobject.h>
onnxruntime-aarch64-unknown-linux-gnu> | ^~~~~~~~~~~~~~~~~~~~~
onnxruntime-aarch64-unknown-linux-gnu> compilation terminated.
onnxruntime-aarch64-unknown-linux-gnu> /build/source/onnxruntime/python/onnxruntime_pybind_state.cc:10:10: fatal error: numpy/arrayobject.h: No such file or directory
onnxruntime-aarch64-unknown-linux-gnu> 10 | #include <numpy/arrayobject.h>
onnxruntime-aarch64-unknown-linux-gnu> | ^~~~~~~~~~~~~~~~~~~~~
onnxruntime-aarch64-unknown-linux-gnu> compilation terminated.
onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:90: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_iobinding.cc.o] Error 1onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:188: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_state.cc.o] Error 1
onnxruntime-aarch64-unknown-linux-gnu> /build/source/onnxruntime/python/onnxruntime_pybind_sparse_tensor.cc:11:10: fatal error: numpy/arrayobject.h: No such file or directory
onnxruntime-aarch64-unknown-linux-gnu> 11 | #include <numpy/arrayobject.h>
onnxruntime-aarch64-unknown-linux-gnu> | ^~~~~~~~~~~~~~~~~~~~~
onnxruntime-aarch64-unknown-linux-gnu> compilation terminated.
onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:174: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_sparse_tensor.cc.o] Error 1
onnxruntime-aarch64-unknown-linux-gnu> /nix/store/hmjkyxmnr2nc6mbq3r8cwwg552j1vqaf-aarch64-unknown-linux-gnu-binutils-2.41/bin/aarch64-unknown-linux-gnu-ld: /nix/store/pl0jivzg5w84j8pbkpg9dmc3gg4rz7rd-protobuf-21.12/lib/libprotobuf-lite.so.3.21.12.0: error adding symbols: file in wrong format
onnxruntime-aarch64-unknown-linux-gnu> collect2: error: ld returned 1 exit status
onnxruntime-aarch64-unknown-linux-gnu> make[2]: *** [CMakeFiles/onnxruntime.dir/build.make:168: libonnxruntime.so.1.16.3] Error 1
onnxruntime-aarch64-unknown-linux-gnu> make[1]: *** [CMakeFiles/Makefile2:1277: CMakeFiles/onnxruntime.dir/all] Error 2
onnxruntime-aarch64-unknown-linux-gnu> make[1]: *** Waiting for unfinished jobs....
onnxruntime-aarch64-unknown-linux-gnu> make[1]: *** [CMakeFiles/Makefile2:1365: CMakeFiles/onnxruntime_pybind11_state.dir/all] Error 2
onnxruntime-aarch64-unknown-linux-gnu> make: *** [Makefile:166: all] Error 2
error: builder for '/nix/store/91h4j7fz7m8qknsfh841hrc9g2m3vcg7-onnxruntime-aarch64-unknown-linux-gnu-1.16.3.drv' failed with exit code 2;
last 10 log lines:
> | ^~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> make[2]: *** [CMakeFiles/onnxruntime_pybind11_state.dir/build.make:174: CMakeFiles/onnxruntime_pybind11_state.dir/build/source/onnxruntime/python/onnxruntime_pybind_sparse_tensor.cc.o] Error 1
> /nix/store/hmjkyxmnr2nc6mbq3r8cwwg552j1vqaf-aarch64-unknown-linux-gnu-binutils-2.41/bin/aarch64-unknown-linux-gnu-ld: /nix/store/pl0jivzg5w84j8pbkpg9dmc3gg4rz7rd-protobuf-21.12/lib/libprotobuf-lite.so.3.21.12.0: error adding symbols: file in wrong format
> collect2: error: ld returned 1 exit status
> make[2]: *** [CMakeFiles/onnxruntime.dir/build.make:168: libonnxruntime.so.1.16.3] Error 1
> make[1]: *** [CMakeFiles/Makefile2:1277: CMakeFiles/onnxruntime.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make[1]: *** [CMakeFiles/Makefile2:1365: CMakeFiles/onnxruntime_pybind11_state.dir/all] Error 2
> make: *** [Makefile:166: all] Error 2
For full logs, run 'nix log /nix/store/91h4j7fz7m8qknsfh841hrc9g2m3vcg7-onnxruntime-aarch64-unknown-linux-gnu-1.16.3.drv'.I'll try playing around with the inputs a bit.
autrimpo
commented
5 months ago Just a quick update on this - i tried moving numpy and protobuf around to no avail, and their current location is probably where they should be according to the docs.
SomeoneSerge
commented
5 months ago How about explicitly adding host's -I${numpy}/${python.sitePackages}/numpy/core/include/ (et cetera) to NIX_CFLAGS_COMPILE
jonringer
commented
3 months ago Sorry, I was banned for 6 weeks, unable to respond.
This is a bit tricky, as python packages have two feet: one in the depsBuildHost and the other in the depsHostHost world. Not sure how python packages should look when both compiling against them and using them for c-style linking
jonringer
commented
3 months ago One initial thought, would be to add an additional dev output, which then should get picked up by the default build to add them as a NIX_CLFAGS_COMPILE, and then the headers should at least be available. Linking of pre-built artifacts is bit trickery (e.g. libnumpy.so if it exists).
Steps To Reproduce
nix build nixpkgs#pkgsCross.aarch64-multiplatform.onnxruntime -v -Lon x86_64-linuxBuild log
Additional context
Add any other context about the problem here.
Notify maintainers
@jonringer @puffnfresh @ck3d @cbourjau
Additionally CC recently committer and reviewer(sorry for bother): @autrimpo @SomeoneSerge
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"and paste the result.Add a :+1: reaction to issues you find important.