Open jwknaup opened 1 month ago
Seeing the same issue on a Jetson Orin AGX with JP6. I think it should be fixable with the patch from https://github.com/tensorflow/tensorflow/issues/62490 (applied to xla -> tsl -> absl
). I'm building with the patch now, I'll comment again if this fixes the issue.
FYI the above workaround works as a one-off; it would be nice to get the fix upstreamed once https://github.com/abseil/abseil-cpp/issues/1665 is resolved.
Thanks Kyle! Worked for me as well
@kylestach Hi, I am facing the same issue here with the Jetson Orin Nano. Did you figure out ?
Note that this has been fixed upstream in absl: https://github.com/abseil/abseil-cpp/pull/1732. Maybe we can bump the vendored copy of absl before the next release?
There is already a PR to add this patch in XLA: https://github.com/openxla/xla/pull/15687 So it should be fixed soon.
Did you link the wrong PR?
Updated my previous comment.
In that case can we bump the vendored copy of XLA? Or just stop vendoring things so it's no longer a JAX bug?
JAX seem to update the XLA commit everyday or so. I looked, current JAX upstream point to an XLA that has the patch. So taking today/tomorrow JAX nightly or building upstream should get you the fix.
If you try, can you confirm the fix works for you?
Description
When I attempt to build jaxlib v0.4.30 on a Jetson Orin with the command
python build/build.py --enable_cuda --cuda_path /usr/local/cuda-12.2 --cudnn_path /usr/lib/aarch64-linux-gnu --cuda_version 12.2 --cudnn_version 8
I get the errorERROR: /home/nvidia/.cache/bazel/_bazel_nvidia/a5643b5cc286b9b13a96818003a4a7dd/external/xla/xla/stream_executor/gpu/BUILD:349:19: Compiling xla/stream_executor/gpu/gpu_timer_kernel_cuda.cu.cc failed: (Exit 4): crosstool_wrapper_driver_is_not_gcc failed: error executing command (from target @xla//xla/stream_executor/gpu:gpu_timer_kernel_cuda)
The full output is below
System info (python version, jaxlib version, accelerator, etc.)
I am using a Jetson Orin Nano with Tegra GPU running Jetpack 6
Cuda 12.2
cuDNN 8