ROCM build fails when the ROCM_PATH is a symlinked location

Description

The jaxlib build with ROCm will fail if the ROCM_PATH is directed to a symlinked location. This issue is particularly critical for NixOS, as the ROCM_PATH is invariably a symlinked location joining each individual library together, thereby making ROCM build on NixOS unfeasible. While this may be less significant for other distributions, it is still a common issue, given that /opt/rocm is likely to be a symlink.

To reproduce this behavior, start from a ROCM container rocm/dev-ubuntu-22.04:5.7.1-complete and run the following commands to prepare the build environment.

apt update
apt install g++ python3-dev python-is-python3 git
pip install numpy wheel build
git clone --branch jaxlib-v0.4.22 https://github.com/google/jax.git
cd jax

In this environment the realpath for ROCM library is /opt/rocm-5.7.1 and there is also a symlink /opt/rocm -> /opt/rocm-5.7.1

Now build jaxlib with python build/build.py --enable_rocm --rocm_path=/opt/rocm will result in a failure, butpython build/build.py --enable_rocm --rocm_path=/opt/rocm-5.7.1 won't.

Build log

``` Bazel binary path: ./bazel-6.1.2-linux-x86_64 Bazel version: 6.1.2 Python binary path: /usr/bin/python3 Python version: 3.10 NumPy version: 1.26.2 MKL-DNN enabled: yes Target CPU: x86_64 Target CPU features: release CUDA enabled: no ROCm enabled: yes ROCm toolkit path: /opt/rocm ROCm amdgpu targets: gfx900,gfx906,gfx908,gfx90a,gfx1030 Building XLA and installing it in the jaxlib source tree... ./bazel-6.1.2-linux-x86_64 run --verbose_failures=true //jaxlib/tools:build_wheel -- --output_path=/jax/dist --cpu=x86_64 INFO: Options provided by the client: Inherited 'common' options: --isatty=0 --terminal_columns=80 INFO: Reading rc options for 'run' from /jax/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'run' from /jax/.bazelrc: Inherited 'build' options: --nocheck_visibility --apple_platform_type=macos --macos_minimum_os=10.14 --announce_rc --define open_source_build=true --spawn_strategy=standalone --enable_platform_specific_config --experimental_cc_shared_library --define=no_aws_support=true --define=no_gcp_support=true --define=no_hdfs_support=true --define=no_kafka_support=true --define=no_ignite_support=true --define=grpc_no_ares=true --define=tsl_link_protobuf=true -c opt --config=short_logs --copt=-DMLIR_PYTHON_PACKAGE_PREFIX=jaxlib.mlir. --@xla//xla/python:enable_gpu=false INFO: Reading rc options for 'run' from /jax/.jax_configure.bazelrc: Inherited 'build' options: --strategy=Genrule=standalone --repo_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env=PYENV_ROOT --python_path=/usr/bin/python3 --action_env ROCM_PATH=/opt/rocm --config=avx_posix --config=mkl_open_source_only --config=rocm INFO: Found applicable config definition build:short_logs in file /jax/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:avx_posix in file /jax/.bazelrc: --copt=-mavx --host_copt=-mavx INFO: Found applicable config definition build:mkl_open_source_only in file /jax/.bazelrc: --define=tensorflow_mkldnn_contraction_kernel=1 INFO: Found applicable config definition build:rocm in file /jax/.bazelrc: --crosstool_top=@local_config_rocm//crosstool:toolchain --define=using_rocm=true --define=using_rocm_hipcc=true --@xla//xla/python:enable_gpu=true --define=xla_python_enable_gpu=true --repo_env TF_NEED_ROCM=1 --action_env TF_ROCM_AMDGPU_TARGETS=gfx900,gfx906,gfx908,gfx90a,gfx1030 INFO: Found applicable config definition build:rocm in file /jax/.jax_configure.bazelrc: --action_env TF_ROCM_AMDGPU_TARGETS=gfx900,gfx906,gfx908,gfx90a,gfx1030 INFO: Found applicable config definition build:linux in file /jax/.bazelrc: --config=posix --copt=-Wno-unknown-warning-option --copt=-Wno-stringop-truncation --copt=-Wno-array-parameter INFO: Found applicable config definition build:posix in file /jax/.bazelrc: --copt=-fvisibility=hidden --copt=-Wno-sign-compare --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 Loading: Loading: Loading: 0 packages loaded Analyzing: target //jaxlib/tools:build_wheel (0 packages loaded, 0 targets configured) INFO: Analyzed target //jaxlib/tools:build_wheel (0 packages loaded, 0 targets configured). INFO: Found 1 target... [0 / 1] [Prepa] BazelWorkspaceStatusAction stable-status.txt [34 / 291] Compiling llvm/lib/Support/DebugCounter.cpp [for tool]; 0s local ... (16 actions, 15 running) [50 / 291] Compiling llvm/lib/Support/Timer.cpp [for tool]; 1s local ... (16 actions, 15 running) ... [5,531 / 7,541] Compiling lib/Dialect/NVGPU/IR/Dialect.cpp; 20s local ... (16 actions running) ERROR: /root/.cache/bazel/_bazel_root/bee4ad1fd43279be7a03b33426e824d5/external/xla/xla/service/gpu/BUILD:1196:23: Compiling xla/service/gpu/cub_sort_kernel.cu.cc failed: undeclared inclusion(s) in rule '@xla//xla/service/gpu:cub_sort_kernel_f64': this rule is missing dependency declarations for the following files included by 'xla/service/gpu/cub_sort_kernel.cu.cc': '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_hip_runtime_wrapper.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/cuda_wrappers/cmath' '/opt/rocm/llvm/lib/clang/17.0.0/include/stddef.h' '/opt/rocm-5.7.1/include/hip/hip_version.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_hip_libdevice_declares.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_hip_math.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/cuda_wrappers/algorithm' '/opt/rocm/llvm/lib/clang/17.0.0/include/cuda_wrappers/new' '/opt/rocm/llvm/lib/clang/17.0.0/include/limits.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/stdint.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_hip_stdlib.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_cuda_math_forward_declares.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_hip_cmath.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/__clang_cuda_complex_builtins.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/cuda_wrappers/complex' '/opt/rocm/llvm/lib/clang/17.0.0/include/__stddef_max_align_t.h' '/opt/rocm/llvm/lib/clang/17.0.0/include/stdarg.h' '/opt/rocm-5.7.1/include/hip/hip_runtime.h' '/opt/rocm-5.7.1/include/hip/hip_common.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_runtime.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_common.h' '/opt/rocm-5.7.1/include/hip/hip_runtime_api.h' '/opt/rocm-5.7.1/include/hip/amd_detail/host_defines.h' '/opt/rocm-5.7.1/include/hip/driver_types.h' '/opt/rocm-5.7.1/include/hip/texture_types.h' '/opt/rocm-5.7.1/include/hip/channel_descriptor.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_channel_descriptor.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_vector_types.h' '/opt/rocm-5.7.1/include/hip/surface_types.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_runtime_pt_api.h' '/opt/rocm-5.7.1/include/hip/amd_detail/hip_ldg.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_atomic.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_device_functions.h' '/opt/rocm-5.7.1/include/hip/amd_detail/math_fwd.h' '/opt/rocm-5.7.1/include/hip/hip_vector_types.h' '/opt/rocm-5.7.1/include/hip/amd_detail/device_library_decls.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_warp_functions.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_unsafe_atomics.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_surface_functions.h' '/opt/rocm-5.7.1/include/hip/amd_detail/texture_fetch_functions.h' '/opt/rocm-5.7.1/include/hip/hip_texture_types.h' '/opt/rocm-5.7.1/include/hip/amd_detail/ockl_image.h' '/opt/rocm-5.7.1/include/hip/amd_detail/texture_indirect_functions.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_math_functions.h' '/opt/rocm-5.7.1/include/hip/amd_detail/hip_fp16_math_fwd.h' '/opt/rocm-5.7.1/include/hip/library_types.h' '/opt/rocm-5.7.1/include/hip/hip_fp16.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_fp16.h' '/opt/rocm-5.7.1/include/hip/hip_bfloat16.h' '/opt/rocm-5.7.1/include/hip/amd_detail/amd_hip_bfloat16.h' '/opt/rocm-5.7.1/include/rocprim/intrinsics/thread.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/types/future_value.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/types/double_buffer.hpp' '/opt/rocm-5.7.1/include/rocprim/types/integer_sequence.hpp' '/opt/rocm-5.7.1/include/rocprim/types/key_value_pair.hpp' '/opt/rocm-5.7.1/include/rocprim/types/tuple.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/all_true.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_adjacent_difference.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_adjacent_difference_impl.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/thread.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_discontinuity.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_exchange.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/atomic.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/bit.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/thread.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/warp.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/warp_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/warp_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_histogram.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_histogram_atomic.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_histogram_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_crosslane.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_dpp.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_radix_rank.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_scan_warp_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_crosslane.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_shared_mem.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_scan_reduce_then_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_radix_rank_basic.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_radix_rank_match.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_discontinuity.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load_func.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/types/future_value.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_reduce_warp_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_reduce_crosslane.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_reduce_dpp.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_reduce_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_segment_bounds.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_reduce_shared_mem.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_reduce_raking_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store_func.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store.hpp' '/opt/rocm-5.7.1/include/hipcub/thread/thread_operators.hpp' '/opt/rocm-5.7.1/include/hipcub/backend/rocprim/thread/thread_operators.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_adjacent_difference.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_adjacent_difference.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_adjacent_difference.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_adjacent_difference_config.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/device/config_types.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/thread.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_transform.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/match_result_type.hpp' '/opt/rocm-5.7.1/include/rocprim/types/tuple.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/zip_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/config.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/types/tuple.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_transform_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_transform.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/match_result_type.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/temp_storage.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/counting_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/transform_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/match_result_type.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_histogram.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_histogram.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/uint_fast_div.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_histogram_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_histogram.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_config_helper.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/device/config_types.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_sort_bitonic.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_sort_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/block/detail/block_sort_merge.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/merge_path.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_sort_stable.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_merge_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_merge.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/merge_path.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_merge_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load_func.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_merge_sort_mergepath.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_merge_sort_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_merge_sort_block_merge.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_merge_sort_block_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_config_helper.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_partition.hpp' '/opt/rocm-5.7.1/include/rocprim/type_traits.hpp' '/opt/rocm-5.7.1/include/rocprim/types.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_select_config.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_scan_common.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/thread.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/lookback_scan_state.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_reduce_crosslane.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/detail/warp_scan_crosslane.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/binary_op_wrappers.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/functional.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/various.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/temp_storage.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/ordered_block_id.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_partition.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_discontinuity.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_radix_sort_onesweep.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_exchange.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_radix_rank.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store_func.hpp' '/opt/rocm-5.7.1/include/rocprim/device/specialization/device_radix_block_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_radix_sort_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_radix_sort_block_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/specialization/device_radix_merge_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_merge_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/arg_index_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/types/key_value_pair.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_reduce_config.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_reduce_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_reduce_by_key.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_reduce_by_key_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_reduce_by_key.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/constant_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_run_length_encode.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/discard_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_run_length_encode_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_select.hpp' '/opt/rocm-5.7.1/include/rocprim/detail/binary_op_wrappers.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/types/future_value.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_scan.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_scan_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_scan_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_scan_by_key.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/config/device_scan_by_key.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_scan_by_key.hpp' '/opt/rocm-5.7.1/include/rocprim/types/tuple.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_scan_by_key_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_segmented_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/reverse_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_segmented_radix_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_load.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_exchange.hpp' '/opt/rocm-5.7.1/include/rocprim/intrinsics/warp_shuffle.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_load_func.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_sort.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_store.hpp' '/opt/rocm-5.7.1/include/rocprim/block/block_store_func.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_segmented_radix_sort_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_segmented_radix_sort_config.hpp' '/opt/rocm-5.7.1/include/rocprim/device/device_segmented_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/device/detail/device_segmented_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/texture_cache_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/constant_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/counting_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/iterator/transform_iterator.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_exchange.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_reduce.hpp' '/opt/rocm-5.7.1/include/rocprim/warp/warp_scan.hpp' clang++: warning: argument unused during compilation: '-fcuda-flush-denormals-to-zero' [-Wunused-command-line-argument] Target //jaxlib/tools:build_wheel failed to build INFO: Elapsed time: 1016.443s, Critical Path: 144.18s INFO: 5549 processes: 1076 internal, 4473 local. FAILED: Build did NOT complete successfully ERROR: Build failed. Not running target Traceback (most recent call last): File "/jax/build/build.py", line 602, in main() File "/jax/build/build.py", line 573, in main shell(command) File "/jax/build/build.py", line 44, in shell output = subprocess.check_output(cmd) File "/usr/lib/python3.10/subprocess.py", line 421, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['./bazel-6.1.2-linux-x86_64', 'run', '--verbose_failures=true', '//jaxlib/tools:build_wheel', '--', '--output_path=/jax/dist', '--cpu=x86_64']' returned non-zero exit status 1. ```

It seems that the symlink location is included in the dependency, however the realpath is used during the build, resulting in a "undeclared inclusion(s) in rule" error.

What jax/jaxlib version are you using?

v0.4.22 (v0.4.21 also has this bug)

Which accelerator(s) are you using?

GPU

Additional system info?

1.26.2 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] uname_result(system='Linux', node='968a744453c3', release='6.6.6', version='SMP PREEMPT_DYNAMIC', machine='x86_64')

NVIDIA GPU info

No response

google / jax