search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
22.99k stars 4.03k forks source link

Visual Studio 2019 should be preferred over 2017 #10772

Closed gjasny closed 4 years ago

gjasny commented 4 years ago

Description of the problem:

On my developer machine where Visual Studio 2017 and 2019 is co-installed Bazel 2.1.0 picks 2017. I would expect the newer (and of cause better) 2019 is used.

I believe it's related to this code comment: https://github.com/bazelbuild/bazel/blob/2d1f3b905178663cfa2f1b973ff6c642c8789e74/tools/cpp/windows_cc_configure.bzl#L172

For VS2019 the registry key does not exist, therefore the loop and function returns once 2017 has been found.

IMHO the proper fix would be to check for the existence of vswhere and use it to query for Visual Studio installations with the C++ payload installed.

Do you think that would make sense and should I give it a try?

Thanks, Gregor

CC: @laszlocsomor

laszlocsomor commented 4 years ago

Thanks for volunteering! Sounds great, let's do it! /cc @meteorcloudy Do I understand correctly that vswhere is installed with VS 2017 and newer?

laszlocsomor commented 4 years ago

For the record: to select a specific VS version for Bazel, use the BAZEL_VC and BAZEL_VC_FULL_VERSION envvars like so: https://docs.bazel.build/versions/2.1.0/windows.html#build-c-with-msvc

shelllet commented 2 years ago

BAZEL awaly use Visual Studio 2022, even if i set BAZEL_VC BAZEL_VC_FULL_VERSION to vs 2019。

laszlocsomor commented 2 years ago

@shelllet , try this in cmd.exe:

set CC_CONFIGURE_DEBUG=1

bazel --batch query @local_config_cc//...

What's the output?

laszlocsomor commented 2 years ago

Also, what does this print in cmd.exe?

set BAZEL_VC
shelllet commented 2 years ago

Also, what does this print in cmd.exe?

set BAZEL_VC

BAZEL_VS image

now i add bazel_vc and bazel_vs image

shelllet commented 2 years ago

@shelllet , try this in cmd.exe:

set CC_CONFIGURE_DEBUG=1

bazel --batch query @local_config_cc//...

What's the output?

so many output messages

FATAL: Command line too long (37102 > 32768): -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=c:\users\admin\_bazel_admin\exlkphug --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED -Xverify:none -Djava.util.logging.config.file=c:\users\admin\_bazel_admin\exlkphug\javalog.properties -Dcom.google.devtools.build.lib.util.LogHandlerQuerier.class=com.google.devtools.build.lib.util.SimpleLogHandler$HandlerQuerier -XX:-MaxFDLimit -Djava.library.path=C:\Users\admin\_bazel_admin\install\7e447298f5ed9555a00a8dcb11768f3e;C:\Users\admin\_bazel_admin\install\7e447298f5ed9555a00a8dcb11768f3e\embedded_tools\jdk\bin;C:\Users\admin\_bazel_admin\install\7e447298f5ed9555a00a8dcb11768f3e\embedded_tools\jdk\bin\server -Dfile.encoding=ISO-8859-1 -Dbazel.windows_unix_root=C:\Windows\system32\bash.exe -jar C:\Users\admin\_bazel_admin\install\7e447298f5ed9555a00a8dcb11768f3e\A-server.jar --batch --connect_timeout_secs=30 --output_user_root=c:\users\admin_bazel_admin --install_base=c:\users\admin_bazel_admin\install\7e447298f5ed9555a00a8dcb11768f3e --install_md5=7e447298f5ed9555a00a8dcb11768f3e --output_base=c:\users\admin_bazel_admin\exlkphug --workspace_directory=c:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg --default_system_javabase= --failure_detail_out=c:\users\admin_bazel_admin\exlkphug\failure_detail.rawproto --expand_configs_in_place --idle_server_tasks --write_command_log --nowatchfs --nofatal_event_bus_exceptions --nowindows_enable_symlinks --client_debug=false --host_jvm_args=-Dbazel.windows_unix_root=C:\Windows\system32\bash.exe --product_name=Bazel --noincompatible_enable_execution_transition --option_sources=batch: query --startup_time=2115 --command_wait_time=0 --binary_path=D:\vcpkg\downloads\tools\bazel\4.1.0-windows\bazel.exe --rc_source=client --default_override=0:common=--isatty=1 --default_override=0:common=--terminal_columns=120 --default_override=0:build=--python_path=C:/Users/admin/AppData/Local/Programs/Python/Python39/python.exe --rc_source=c:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg.bazelrc --rc_source=c:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg.tf_configure.bazelrc --default_override=1:build:linux=--copt=-w --default_override=1:build:linux=--host_copt=-w --default_override=1:build:linux=--define=PREFIX=/usr --default_override=1:build:linux=--define=LIBDIR=$(PREFIX)/lib --default_override=1:build:linux=--define=INCLUDEDIR=$(PREFIX)/include --default_override=1:build:linux=--define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --default_override=1:build:linux=--cxxopt=-std=c++14 --default_override=1:build:linux=--host_cxxopt=-std=c++14 --default_override=1:build:linux=--config=dynamic_kernels --default_override=1:build:linux=--distinct_host_configuration=false --default_override=1:test:rbe_linux_cuda_base=--test_env=LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64 --default_override=1:build:avx2_linux=--copt=-mavx2 --default_override=1:build:android_x86=--config=android --default_override=1:build:android_x86=--cpu=x86 --default_override=1:build:android_x86=--fat_apk_cpu=x86 --default_override=1:build=--define --default_override=1:build=framework_shared_object=true --default_override=1:build=--java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --default_override=1:build=--host_java_toolchain=@tf_toolchains//toolchains/java:tf_java_toolchain --default_override=1:build=--define=use_fast_cpp_protos=true --default_override=1:build=--define=allow_oversize_protos=true --default_override=1:build=--spawn_strategy=standalone --default_override=1:build=-c --default_override=1:build=opt --default_override=1:build=--announce_rc --default_override=1:build=--define=grpc_no_ares=true --default_override=1:build=--noincompatible_remove_legacy_whole_archive --default_override=1:build=--enable_platform_specific_config --default_override=1:build=--define=with_xla_support=true --default_override=1:build=--config=short_logs --default_override=1:build=--config=v2 --default_override=1:build=--define=no_aws_support=true --default_override=1:build=--define=no_hdfs_support=true --default_override=2:build=--action_env --default_override=2:build=PYTHON_BIN_PATH=D:/vcpkg/downloads/tools/msys2/5ac3933426d8437e/mingw64/bin/python3.exe --default_override=2:build=--action_env --default_override=2:build=PYTHON_LIB_PATH=D:/vcpkg/downloads/tools/msys2/5ac3933426d8437e/mingw64/lib/python3.8/site-packages --default_override=2:build=--python_path=D:/vcpkg/downloads/tools/msys2/5ac3933426d8437e/mingw64/bin/python3.exe --default_override=2:build=--define=with_xla_support=false --default_override=2:build=--copt=/d2ReducedOptimizeHugeFunctions --default_override=2:build=--host_copt=/d2ReducedOptimizeHugeFunctions --default_override=2:build=--define=override_eigen_strong_inline=true --default_override=1:build:libc++=--action_env=CC --default_override=1:build:libc++=--action_env=CXX --default_override=1:build:libc++=--action_env=CXXFLAGS=-stdlib=libc++ --default_override=1:build:libc++=--action_env=PATH --default_override=1:build:libc++=--define --default_override=1:build:libc++=force_libcpp=enabled --default_override=1:build:libc++=--linkopt --default_override=1:build:libc++=-fuse-ld=lld --default_override=1:build:rbe_cpu_linux=--config=rbe_linux --default_override=1:build:rbe_cpu_linux=--host_crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:toolchain --default_override=1:build:rbe_cpu_linux=--crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:toolchain --default_override=1:build:rbe_cpu_linux=--extra_toolchains=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010:cc-toolchain-k8 --default_override=1:build:rbe_cpu_linux=--extra_execution_platforms=@ubuntu16.04-manylinux2010-py3_config_platform//:platform --default_override=1:build:rbe_cpu_linux=--extra_execution_platforms=@ubuntu16.04-manylinux2010-py3_config_platform//:platform --default_override=1:build:rbe_cpu_linux=--host_platform=@ubuntu16.04-manylinux2010-py3_config_platform//:platform --default_override=1:build:rbe_cpu_linux=--platforms=@ubuntu16.04-manylinux2010-py3_config_platform//:platform --default_override=1:build:android=--crosstool_top=//external:android/crosstool --default_override=1:build:android=--host_crosstool_top=@bazel_tools//tools/cpp:toolchain --default_override=1:build:android=--noenable_platform_specific_config --default_override=1:build:android=--copt=-w --default_override=1:build:android=--cxxopt=-std=c++14 --default_override=1:build:android=--host_cxxopt=-std=c++14 --default_override=1:build:android=--define=with_xla_support=false --default_override=1:build:rbe_linux=--config=rbe --default_override=1:build:rbe_linux=--action_env=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin --default_override=1:build:rbe_linux=--host_javabase=@bazel_toolchains//configs/ubuntu16_04_clang/1.1:jdk8 --default_override=1:build:rbe_linux=--javabase=@bazel_toolchains//configs/ubuntu16_04_clang/1.1:jdk8 --default_override=1:build:rbe_linux=--host_java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --default_override=1:build:rbe_linux=--java_toolchain=@bazel_tools//tools/jdk:toolchain_hostjdk8 --default_override=1:build:rbe_linux=--config=avx_linux --default_override=1:build:rbe_linux=--linkopt=-lrt --default_override=1:build:rbe_linux=--host_linkopt=-lrt --default_override=1:build:rbe_linux=--linkopt=-lm --default_override=1:build:rbe_linux=--host_linkopt=-lm --default_override=1:build:v2=--define=tf_api_version=2 --default_override=1:build:v2=--action_env=TF2_BEHAVIOR=1 --default_override=1:build:macos_arm64=--cpu=darwin_arm64 --default_override=1:build:android_arm=--config=android --default_override=1:build:android_arm=--cpu=armeabi-v7a --default_override=1:build:android_arm=--fat_apk_cpu=armeabi-v7a --default_override=1:build:avx_linux=--copt=-mavx --default_override=1:build:avx_linux=--host_copt=-mavx --default_override=1:build:android_arm64=--config=android --default_override=1:build:android_arm64=--cpu=arm64-v8a --default_override=1:build:android_arm64=--fat_apk_cpu=arm64-v8a --default_override=1:build:dbg=-c --default_override=1:build:dbg=dbg --default_override=1:build:dbg=--per_file_copt=+.,-tensorflow.@-g0 --default_override=1:build:dbg=--per_file_copt=+tensorflow/core/kernels.*@-g0 --default_override=1:build:dbg=--cxxopt --default_override=1:build:dbg=-DTF_LITE_DISABLE_X86_NEON --default_override=1:build:dbg=--copt --default_override=1:build:dbg=-DDEBUG_BUILD --default_override=1:build:android_x86_64=--config=android --default_override=1:build:android_x86_64=--cpu=x86_64 --default_override=1:build:android_x86_64=--fat_apk_cpu=x86_64 --default_override=1:build:macos=--apple_platform_type=macos --default_override=1:build:macos=--copt=-DGRPC_BAZEL_BUILD --default_override=1:build:macos=--copt=-w --default_override=1:build:macos=--define=PREFIX=/usr --default_override=1:build:macos=--define=LIBDIR=$(PREFIX)/lib --default_override=1:build:macos=--define=INCLUDEDIR=$(PREFIX)/include --default_override=1:build:macos=--define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --default_override=1:build:macos=--cxxopt=-std=c++14 --default_override=1:build:macos=--host_cxxopt=-std=c++14 --default_override=1:build:rbe_linux_cuda_nvcc_py39=--config=rbe_linux_cuda11.2_nvcc_py3.9 --default_override=1:build:mkl_threadpool=--define=build_with_mkl=true --default_override=1:build:mkl_threadpool=--define=enable_mkl=true --default_override=1:build:mkl_threadpool=--define=tensorflow_mkldnn_contraction_kernel=0 --default_override=1:build:mkl_threadpool=--define=build_with_mkl_opensource=true --default_override=1:build:mkl_threadpool=-c --default_override=1:build:mkl_threadpool=opt --default_override=1:build:ios=--apple_platform_type=ios --default_override=1:build:ios=--apple_bitcode=embedded --default_override=1:build:ios=--copt=-fembed-bitcode --default_override=1:build:ios=--copt=-Wno-c++11-narrowing --default_override=1:build:ios=--noenable_platform_specific_config --default_override=1:build:ios=--copt=-w --default_override=1:build:ios=--cxxopt=-std=c++14 --default_override=1:build:ios=--host_cxxopt=-std=c++14 --default_override=1:build:ios=--define=with_xla_support=false --default_override=1:build:ios_armv7=--config=ios --default_override=1:build:ios_armv7=--cpu=ios_armv7 --default_override=1:build:rbe=--repo_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 --default_override=1:build:rbe=--google_default_credentials --default_override=1:build:rbe=--bes_backend=buildeventservice.googleapis.com --default_override=1:build:rbe=--bes_results_url=https://source.cloud.google.com/results/invocations --default_override=1:build:rbe=--bes_timeout=600s --default_override=1:build:rbe=--define=EXECUTOR=remote --default_override=1:build:rbe=--distinct_host_configuration=false --default_override=1:build:rbe=--flaky_test_attempts=3 --default_override=1:build:rbe=--jobs=200 --default_override=1:build:rbe=--remote_executor=grpcs://remotebuildexecution.googleapis.com --default_override=1:build:rbe=--remote_timeout=3600 --default_override=1:build:rbe=--spawn_strategy=remote,worker,standalone,local --default_override=1:build:rbe=--remote_download_toplevel --default_override=1:build:ios_arm64=--config=ios --default_override=1:build:ios_arm64=--cpu=ios_arm64 --default_override=1:build:ios_i386=--config=ios --default_override=1:build:ios_i386=--cpu=ios_i386 --default_override=1:build:windows=--copt=/W0 --default_override=1:build:windows=--copt=/D_USE_MATH_DEFINES --default_override=1:build:windows=--host_copt=/D_USE_MATH_DEFINES --default_override=1:build:windows=--cxxopt=/std:c++14 --default_override=1:build:windows=--host_cxxopt=/std:c++14 --default_override=1:build:windows=--config=monolithic --default_override=1:build:windows=--copt=-DWIN32_LEAN_AND_MEAN --default_override=1:build:windows=--host_copt=-DWIN32_LEAN_AND_MEAN --default_override=1:build:windows=--copt=-DNOGDI --default_override=1:build:windows=--host_copt=-DNOGDI --default_override=1:build:windows=--copt=/experimental:preprocessor --default_override=1:build:windows=--host_copt=/experimental:preprocessor --default_override=1:build:windows=--linkopt=/DEBUG --default_override=1:build:windows=--host_linkopt=/DEBUG --default_override=1:build:windows=--linkopt=/OPT:REF --default_override=1:build:windows=--host_linkopt=/OPT:REF --default_override=1:build:windows=--linkopt=/OPT:ICF --default_override=1:build:windows=--host_linkopt=/OPT:ICF --default_override=1:build:windows=--verbose_failures --default_override=1:build:windows=--distinct_host_configuration=false --default_override=1:build:ios_x86_64=--config=ios --default_override=1:build:ios_x86_64=--cpu=ios_x86_64 --default_override=1:common=--experimental_repo_remote_exec --default_override=1:build:ios_fat=--config=ios --default_override=1:build:ios_fat=--ios_multi_cpus=armv7,arm64,i386,x86_64 --default_override=1:build:mkl_aarch64=--define=build_with_mkl_aarch64=true --default_override=1:build:mkl_aarch64=--define=enable_mkl=true --default_override=1:build:mkl_aarch64=--define=tensorflow_mkldnn_contraction_kernel=0 --default_override=1:build:mkl_aarch64=--define=build_with_mkl_opensource=true --default_override=1:build:mkl_aarch64=--define=build_with_openmp=true --default_override=1:build:mkl_aarch64=-c --default_override=1:build:mkl_aarch64=opt --default_override=1:build:monolithic=--define --default_override=1:build:monolithic=framework_shared_object=false --default_override=1:build:rbe_linux_cuda_nvcc_py36=--config=rbe_linux_cuda11.2_nvcc_py3.6 --default_override=2:test=--flaky_test_attempts=3 --default_override=2:test=--test_size_filters=small,medium --default_override=1:build:rbe_win_py37=--config=rbe --default_override=1:build:rbe_win_py37=--repo_env=TF_PYTHON_CONFIG_REPO=@windows_py37_config_python --default_override=1:build:rbe_win_py37=--python_path=C:\Python37\python.exe --default_override=1:build:mkl=--define=build_with_mkl=true --default_override=1:build:mkl=--define=enable_mkl=true --default_override=1:build:mkl=--define=tensorflow_mkldnn_contraction_kernel=0 --default_override=1:build:mkl=--define=build_with_openmp=true --default_override=1:build:mkl=-c --default_override=1:build:mkl=opt --default_override=1:test:rbe=--test_env=USER=anon --default_override=1:build:cuda=--repo_env --default_override=1:build:cuda=TF_NEED_CUDA=1 --default_override=1:build:cuda=--crosstool_top=@local_config_cuda//crosstool:toolchain --default_override=1:build:cuda=--@local_config_cuda//:enable_cuda --default_override=1:build:native_arch_linux=--copt=-march=native --default_override=1:build:numa=--define=with_numa_support=true --default_override=1:build:cuda_clang=--config=cuda --default_override=1:build:cuda_clang=--repo_env --default_override=1:build:cuda_clang=TF_CUDA_CLANG=1 --default_override=1:build:cuda_clang=--@local_config_cuda//:cuda_compiler=clang --default_override=1:build:rbe_linux_cuda_nvcc=--config=rbe_linux_cuda_nvcc_py36 --default_override=1:build:nogcp=--define=no_gcp_support=true --default_override=1:build:tpu=--define=with_tpu_support=true --default_override=1:build:tensorrt=--repo_env --default_override=1:build:tensorrt=TF_NEED_TENSORRT=1 --default_override=1:build:rocm=--crosstool_top=@local_config_rocm//crosstool:toolchain --default_override=1:build:rocm=--define=using_rocm=true --default_override=1:build:rocm=--define=using_rocm_hipcc=true --default_override=1:build:rocm=--repo_env --default_override=1:build:rocm=TF_NEED_ROCM=1 --default_override=1:build:noaws=--define=no_aws_support=true --default_override=1:build:nohdfs=--define=no_hdfs_support=true --default_override=1:build:verbose_logs=--output_filter= --default_override=1:build:stackdriver_support=--define=stackdriver_support=true --default_override=1:build:nonccl=--define=no_nccl_support=true --default_override=1:build:dynamic_kernels=--define=dynamic_loaded_kernels=true --default_override=1:build:dynamic_kernels=--copt=-DAUTOLOAD_DYNAMIC_KERNELS --default_override=1:build:c++17=--cxxopt=-std=c++1z --default_override=1:build:c++17=--cxxopt=-stdlib=libc++ --default_override=1:build:c++1z=--config=c++17 --default_override=1:build:c++17_gcc=--cxxopt=-std=c++1z --default_override=1:build:c++1z_gcc=--config=c++17_gcc --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.6=--config=rbe_linux_cuda11.2_nvcc_base --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.6=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.6 --default_override=1:build:short_logs=--output_filter=DONT_MATCH_ANYTHING --default_override=1:build:avx_win=--copt=/arch=AVX --default_override=1:build:avx2_win=--copt=/arch=AVX2 --default_override=1:build:rbe_linux_cuda_nvcc_py37=--config=rbe_linux_cuda11.2_nvcc_py3.7 --default_override=1:build:v1=--define=tf_api_version=1 --default_override=1:build:v1=--action_env=TF2_BEHAVIOR=0 --default_override=1:build:xla=--define=with_xla_supprt=true --default_override=1:build:rbe_linux_cuda_base=--config=rbe_linux --default_override=1:build:rbe_linux_cuda_base=--config=cuda --default_override=1:build:rbe_linux_cuda_base=--config=tensorrt --default_override=1:build:rbe_linux_cuda_base=--action_env=TF_CUDA_VERSION=11 --default_override=1:build:rbe_linux_cuda_base=--action_env=TF_CUDNN_VERSION=8 --default_override=1:build:rbe_linux_cuda_base=--repo_env=REMOTE_GPU_TESTING=1 --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--config=rbe_linux_cuda_base --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--host_crosstool_top=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--crosstool_top=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--extra_toolchains=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain-linux-x86_64 --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--extra_execution_platforms=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--host_platform=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--platforms=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--repo_env=TF_CUDA_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--repo_env=TF_TENSORRT_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_tensorrt --default_override=1:build:rbe_linux_cuda11.2_nvcc_base=--repo_env=TF_NCCL_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_nccl --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.7=--config=rbe_linux_cuda11.2_nvcc_base --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.7=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.7 --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.8=--config=rbe_linux_cuda11.2_nvcc_base --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.8=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.8 --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.9=--config=rbe_linux_cuda11.2_nvcc_base --default_override=1:build:rbe_linux_cuda11.2_nvcc_py3.9=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.9 --default_override=1:build:rbe_linux_cuda_nvcc_py38=--config=rbe_linux_cuda11.2_nvcc_py3.8 --default_override=1:build:rbe_gpu_linux=--config=rbe_linux_cuda_nvcc --default_override=1:build:rbe_linux_cuda_clang_base=--config=rbe_linux_cuda_base --default_override=1:build:rbe_linux_cuda_clang_base=--repo_env --default_override=1:build:rbe_linux_cuda_clang_base=TF_CUDA_CLANG=1 --default_override=1:build:rbe_linux_cuda_clang_base=--@local_config_cuda//:cuda_compiler=clang --default_override=1:build:rbe_linux_cuda_clang_base=--crosstool_top=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain --default_override=1:build:rbe_linux_cuda_clang_base=--extra_toolchains=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain-linux-x86_64 --default_override=1:build:rbe_linux_cuda_clang_base=--extra_execution_platforms=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda_clang_base=--host_platform=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda_clang_base=--platforms=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_platform//:platform --default_override=1:build:rbe_linux_cuda_clang_base=--repo_env=TF_CUDA_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda --default_override=1:build:rbe_linux_cuda_clang_base=--repo_env=TF_TENSORRT_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_tensorrt --default_override=1:build:rbe_linux_cuda_clang_base=--repo_env=TF_NCCL_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_nccl --default_override=1:build:rbe_linux_cuda_clang_py27=--config=rbe_linux_cuda_clang_base --default_override=1:build:rbe_linux_cuda_clang_py27=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python2.7 --default_override=1:build:rbe_linux_cuda_clang_py35=--config=rbe_linux_cuda_clang_base --default_override=1:build:rbe_linux_cuda_clang_py35=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.5 --default_override=1:build:rbe_linux_cuda_clang_py36=--config=rbe_linux_cuda_clang_base --default_override=1:build:rbe_linux_cuda_clang_py36=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.6 --default_override=1:build:rbe_linux_cuda_clang_py37=--config=rbe_linux_cuda_clang_base --default_override=1:build:rbe_linux_cuda_clang_py37=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.7 --default_override=1:build:rbe_linux_cuda_clang_py38=--config=rbe_linux_cuda_clang_base --default_override=1:build:rbe_linux_cuda_clang_py38=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-clang_manylinux2010-cuda11.2-cudnn8.1-tensorrt7.2_config_python3.8 --default_override=1:build:rbe_linux_rocm_base=--config=rbe_linux --default_override=1:build:rbe_linux_rocm_base=--repo_env=TF_NEED_ROCM=1 --default_override=1:build:rbe_linux_rocm_base=--crosstool_top=@ubuntu18.04-gcc7_manylinux2010-rocm_config_rocm//crosstool:toolchain --default_override=1:build:rbe_linux_rocm_base=--extra_toolchains=@ubuntu18.04-gcc7_manylinux2010-rocm_config_rocm//crosstool:toolchain-linux-x86_64 --default_override=1:build:rbe_linux_rocm_base=--extra_execution_platforms=@ubuntu18.04-gcc7_manylinux2010-rocm_config_platform//:platform --default_override=1:build:rbe_linux_rocm_base=--host_platform=@ubuntu18.04-gcc7_manylinux2010-rocm_config_platform//:platform --default_override=1:build:rbe_linux_rocm_base=--platforms=@ubuntu18.04-gcc7_manylinux2010-rocm_config_platform//:platform --default_override=1:build:rbe_linux_rocm_base=--action_env=TF_ROCM_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_rocm --default_override=1:build:rbe_linux_rocm_base=--define=using_rocm_hipcc=true --default_override=1:build:rbe_linux_rocm_py2.7=--config=rbe_linux_rocm_base --default_override=1:build:rbe_linux_rocm_py2.7=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_python2.7 --default_override=1:build:rbe_linux_rocm_py3.5=--config=rbe_linux_rocm_base --default_override=1:build:rbe_linux_rocm_py3.5=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_python3.5 --default_override=1:build:rbe_linux_rocm_py3.6=--config=rbe_linux_rocm_base --default_override=1:build:rbe_linux_rocm_py3.6=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_python3.6 --default_override=1:build:rbe_linux_rocm_py3.7=--config=rbe_linux_rocm_base --default_override=1:build:rbe_linux_rocm_py3.7=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_python3.7 --default_override=1:build:rbe_linux_rocm_py3.8=--config=rbe_linux_rocm_base --default_override=1:build:rbe_linux_rocm_py3.8=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu18.04-gcc7_manylinux2010-rocm_config_python3.8 --default_override=1:build:rbe_linux_py2=--config=rbe_linux --default_override=1:build:rbe_linux_py2=--repo_env=PYTHON_BIN_PATH=/usr/bin/python2 --default_override=1:build:rbe_linux_py2=--python_path=/usr/bin/python2 --default_override=1:build:rbe_linux_py2=--repo_env=TF_PYTHON_CONFIG_REPO=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/py --default_override=1:build:rbe_linux_py3=--config=rbe_linux --default_override=1:build:rbe_linux_py3=--python_path=/usr/bin/python3 --default_override=1:build:rbe_linux_py3=--repo_env=TF_PYTHON_CONFIG_REPO=@ubuntu16.04-manylinux2010-py3_config_python --default_override=1:build:rbe_win=--config=rbe --default_override=1:build:rbe_win=--crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/win/tf_win_06242021:toolchain --default_override=1:build:rbe_win=--extra_toolchains=@org_tensorflow//third_party/toolchains/preconfig/win/tf_win_06242021:cc-toolchain-x64_windows --default_override=1:build:rbe_win=--host_javabase=@org_tensorflow//third_party/toolchains/preconfig/win:windows_jdk8 --default_override=1:build:rbe_win=--javabase=@org_tensorflow//third_party/toolchains/preconfig/win:windows_jdk8 --default_override=1:build:rbe_win=--extra_execution_platforms=@org_tensorflow//third_party/toolchains/preconfig/win:rbe_windows_ltsc2019 --default_override=1:build:rbe_win=--host_platform=@org_tensorflow//third_party/toolchains/preconfig/win:rbe_windows_ltsc2019 --default_override=1:build:rbe_win=--platforms=@org_tensorflow//third_party/toolchains/preconfig/win:rbe_windows_ltsc2019 --default_override=1:build:rbe_win=--shell_executable=C:\tools\msys64\usr\bin\bash.exe --default_override=1:build:rbe_win=--experimental_strict_action_env=true --default_override=1:build:rbe_win=--define=override_eigen_strong_inline=true --default_override=1:build:rbe_win=--jobs=100 --default_override=1:build:rbe_win=--remote_download_minimal --default_override=1:build:rbe_win=--enable_runfiles --default_override=1:build:rbe_win=--nobuild_python_zip --default_override=1:build:rbe_win_py38=--config=rbe --default_override=1:build:rbe_win_py38=--repo_env=PYTHON_BIN_PATH=C:\Python38\python.exe --default_override=1:build:rbe_win_py38=--repo_env=PYTHON_LIB_PATH=C:\Python38\lib\site-packages --default_override=1:build:rbe_win_py38=--repo_env=TF_PYTHON_CONFIG_REPO=@org_tensorflow//third_party/toolchains/preconfig/win_1803/py38 --default_override=1:build:rbe_win_py38=--python_path=C:\Python38\python.exe --default_override=1:build:tensorflow_testing_rbe=--project_id=tensorflow-testing --default_override=1:common:tensorflow_testing_rbe_linux=--remote_instance_name=projects/tensorflow-testing/instances/default_instance --default_override=1:build:tensorflow_testing_rbe_linux=--config=tensorflow_testing_rbe --default_override=1:common:tensorflow_testing_rbe_win=--remote_instance_name=projects/tensorflow-testing/instances/windows --default_override=1:build:tensorflow_testing_rbe_win=--config=tensorflow_testing_rbe --default_override=1:build:elinux=--crosstool_top=@local_config_embedded_arm//:toolchain --default_override=1:build:elinux=--host_crosstool_top=@bazel_tools//tools/cpp:toolchain --default_override=1:build:elinux_aarch64=--config=elinux --default_override=1:build:elinux_aarch64=--cpu=aarch64 --default_override=1:build:elinux_aarch64=--distinct_host_configuration=true --default_override=1:build:elinux_armhf=--config=elinux --default_override=1:build:elinux_armhf=--cpu=armhf --default_override=1:build:elinux_armhf=--distinct_host_configuration=true --default_override=2:build:opt=--copt=/arch:AVX --default_override=2:build:opt=--host_copt=/arch:AVX --default_override=2:test:v1=--test_tag_filters=-benchmark-test,-no_oss,-no_windows,-gpu,-oss_serial --default_override=2:test:v1=--build_tag_filters=-benchmark-test,-no_oss,-no_windows,-gpu --default_override=2:test:v2=--test_tag_filters=-benchmark-test,-no_oss,-no_windows,-gpu,-oss_serial,-v1only --default_override=2:test:v2=--build_tag_filters=-benchmark-test,-no_oss,-no_windows,-gpu,-v1only --default_override=1:build:release_base=--config=v2 --default_override=1:build:release_base=--distinct_host_configuration=false --default_override=1:test:release_base=--flaky_test_attempts=3 --default_override=1:test:release_base=--test_size_filters=small,medium --default_override=1:build:msan=--strip=never --default_override=1:build:msan=--copt --default_override=1:build:msan=-fsanitize=memory --default_override=1:build:msan=--copt --default_override=1:build:msan=-DMEMORY_SANITIZER --default_override=1:build:msan=--copt --default_override=1:build:msan=-g --default_override=1:build:msan=--copt --default_override=1:build:msan=-O3 --default_override=1:build:msan=--copt --default_override=1:build:msan=-fno-omit-frame-pointer --default_override=1:build:msan=--linkopt --default_override=1:build:msan=-fsanitize=memory --default_override=1:build:release_cpu_linux=--config=release_base --default_override=1:build:release_cpu_linux=--config=avx_linux --default_override=1:build:release_cpu_linux=--crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda11.2:toolchain --default_override=1:test:release_cpu_linux=--test_env=LD_LIBRARY_PATH --default_override=1:build:release_cpu_macos=--config=release_base --default_override=1:build:release_cpu_macos=--config=avx_linux --default_override=1:build:release_gpu_base=--config=cuda --default_override=1:build:release_gpu_base=--action_env=TF_CUDA_VERSION=11 --default_override=1:build:release_gpu_base=--action_env=TF_CUDNN_VERSION=8 --default_override=1:build:release_gpu_base=--repo_env=TF_CUDA_COMPUTE_CAPABILITIES=sm_35,sm_50,sm_60,sm_70,sm_75,compute_80 --default_override=1:build:release_gpu_linux=--config=release_cpu_linux --default_override=1:build:release_gpu_linux=--config=release_gpu_base --default_override=1:build:release_gpu_linux=--config=tensorrt --default_override=1:build:release_gpu_linux=--action_env=CUDA_TOOLKIT_PATH=/usr/local/cuda-11.2 --default_override=1:build:release_gpu_linux=--action_env=LD_LIBRARY_PATH=/usr/local/cuda:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/tensorrt/lib --default_override=1:build:release_gpu_linux=--action_env=GCC_HOST_COMPILER_PATH=/usr/bin/gcc-5 --default_override=1:build:release_gpu_linux=--crosstool_top=@org_tensorflow//third_party/toolchains/preconfig/ubuntu16.04/gcc7_manylinux2010-nvcc-cuda11.2:toolchain --default_override=1:build:release_cpu_windows=--config=release_base --default_override=1:build:release_cpu_windows=--config=avx_win --default_override=1:build:release_cpu_windows=--define=no_tensorflow_py_deps=true --default_override=1:build:release_cpu_windows=--copt=/d2ReducedOptimizeHugeFunctions --default_override=1:build:release_cpu_windows=--host_copt=/d2ReducedOptimizeHugeFunctions --default_override=1:build:release_gpu_windows=--config=release_cpu_windows --default_override=1:build:release_gpu_windows=--config=release_gpu_base --default_override=1:build:asan=--strip=never --default_override=1:build:asan=--copt --default_override=1:build:asan=-fsanitize=address --default_override=1:build:asan=--copt --default_override=1:build:asan=-DADDRESS_SANITIZER --default_override=1:build:asan=--copt --default_override=1:build:asan=-g --default_override=1:build:asan=--copt --default_override=1:build:asan=-O3 --default_override=1:build:asan=--copt --default_override=1:build:asan=-fno-omit-frame-pointer --default_override=1:build:asan=--linkopt --default_override=1:build:asan=-fsanitize=address --default_override=1:build:ubsan=--strip=never --default_override=1:build:ubsan=--copt --default_override=1:build:ubsan=-fsanitize=undefined --default_override=1:build:ubsan=--copt --default_override=1:build:ubsan=-DUNDEFINED_BEHAVIOR_SANITIZER --default_override=1:build:ubsan=--copt --default_override=1:build:ubsan=-g --default_override=1:build:ubsan=--copt --default_override=1:build:ubsan=-O3 --default_override=1:build:ubsan=--copt --default_override=1:build:ubsan=-fno-omit-frame-pointer --default_override=1:build:ubsan=--linkopt --default_override=1:build:ubsan=-fsanitize=undefined --default_override=1:build:ubsan=--linkopt --default_override=1:build:ubsan=-lubsan --client_env=ALLUSERSPROFILE=C:\ProgramData --client_env=APPDATA=C:\Users\admin\AppData\Roaming "--client_env=BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC" --client_env=BAZEL_VC_FULL_VERSION=14.29.30133 "--client_env=BAZEL_VS=C:\Program Files (x86)\Microsoft Visual Studio\2019\Community" "--client_env=CMAKE_ROOT=C:\Program Files\CMake\bin" "--client_env=CommonProgramFiles=C:\Program Files\Common Files" "--client_env=CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files" "--client_env=CommonProgramW6432=C:\Program Files\Common Files" --client_env=COMPUTERNAME=DESKTOP-P7PT7J9 --client_env=ComSpec=C:\WINDOWS\system32\cmd.exe "--client_env=CUDA_PATH_V11_4=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4" --client_env=DriverData=C:\Windows\System32\Drivers\DriverData --client_env=HOMEDRIVE=C: --client_env=HOMEPATH=\Users\admin --client_env=LOCALAPPDATA=C:\Users\admin\AppData\Local --client_env=LOGONSERVER=\DESKTOP-P7PT7J9 --client_env=NUMBER_OF_PROCESSORS=12 "--client_env=NVTOOLSEXT_PATH=C:\Program Files\NVIDIA Corporation\NvToolsExt\" "--client_env=OPENSSL_CONF=C:\Program Files\OpenSSL-Win64\bin\openssl.cfg" --client_env=OS=Windows_NT "--client_env=PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\libnvvp;C:\Python39\Scripts\;C:\Python39\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\ProgramData\chocolatey\bin;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files\PuTTY\;C:\Program Files\Git\cmd;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\NVIDIA Corporation\Nsight Compute 2021.2.2\;%userprofile%\AppData\Local\Microsoft\WindowsApps;C:\Program Files\dotnet\;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\bin\..\extras\CUPTI\lib64;C:\cudnn\bin;C:\Users\admin\AppData\Local\Programs\Python\Python39\;C:\Users\admin\AppData\Local\Programs\Python\Python39\Scripts;C:\Users\admin\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\admin\AppData\Roaming\Python\Python39\Scripts;C:\Users\admin.dotnet\tools;D:\vcpkg\downloads\tools\bazel\4.1.0-windows" --client_env=PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL --client_env=PROCESSOR_ARCHITECTURE=AMD64 "--client_env=PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 165 Stepping 2, GenuineIntel" --client_env=PROCESSOR_LEVEL=6 --client_env=PROCESSOR_REVISION=a502 --client_env=ProgramData=C:\ProgramData "--client_env=ProgramFiles=C:\Program Files" "--client_env=ProgramFiles(x86)=C:\Program Files (x86)" "--client_env=ProgramW6432=C:\Program Files" "--client_env=PSModulePath=C:\Users\admin\Documents\WindowsPowerShell\Modules;C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules" --client_env=PUBLIC=C:\Users\Public --client_env=PYTHON_ROOT=C:\Users\admin\AppData\Local\Programs\Python\Python39\ --client_env=SYSTEMDRIVE=C: --client_env=SYSTEMROOT=C:\WINDOWS --client_env=TEMP=C:\Users\admin\AppData\Local\Temp --client_env=TMP=C:\Users\admin\AppData\Local\Temp --client_env=USERDOMAIN=DESKTOP-P7PT7J9 --client_env=USERDOMAIN_ROAMINGPROFILE=DESKTOP-P7PT7J9 --client_env=USERNAME=admin --client_env=USERPROFILE=C:\Users\admin --client_env=VCPKG_DEFAULT_BINARY_CACHE=d:\vcpkg\archives --client_env=VCPKG_DOWNLOADS=d:\vcpkg\downloads --client_env=windir=C:\WINDOWS --client_env=WSLENV=WT_SESSION::WT_PROFILE_ID --client_env=WT_PROFILE_ID={61c54bbd-c2c6-5271-96e7-009a87ff44bf} --client_env=WT_SESSION=08a100df-c409-4daf-948c-20d9ed0a3f73 --client_env=ZES_ENABLE_SYSMAN=1 --client_env=BAZEL_SH=C:\Windows\system32\bash.exe --client_cwd=c:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg @local_config_cc//...

shelllet commented 2 years ago

is vcpkg's problem?

laszlocsomor commented 2 years ago

This error is interesting: Bazel client can't even start the server because the command is too long. So you don't get to the local_config_cc config step yet.

But in #14232 you wrote:

Auto-Configuration Error: Couldn't find undname.exe under C:\Program Files\Microsoft Visual Studio\2022\Community\VC, please check your VC installation and set BAZEL_VC environment variable correctly.
ERROR: Analysis of target '//tensorflow:tensorflow_cc.dll' failed; build aborted: Analysis failed
INFO: Elapsed time: 6.906s

which comes from the bazel server, so somehow you got past the "ERROR" point.

The command is long because Bazel is picking up a lot of settings from the bazelrc files. You can tell it to ignore rc files like so:

set CC_CONFIGURE_DEBUG=1

bazel --batch --ignore_all_rc_files query @local_config_cc//...
shelllet commented 2 years ago

so strange. bazel --batch --ignore_all_rc_files query @local_config_cc//...

PS C:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg> bazel --batch --ignore_all_rc_files query @local_config_cc//... ERROR: Traceback (most recent call last): File "C:/vcpkg/buildtrees/tensorflow-cc/x64-windows-dbg/third_party/py/python_configure.bzl", line 284, column 42, in remote_python_configure = repository_rule( Error in repository_rule: in call to repository_rule(), parameter 'remotable' is experimental and thus unavailable with the current flags. It may be enabled by setting --experimental_repo_remote_exec ERROR: error loading package '': in C:/vcpkg/buildtrees/tensorflow-cc/x64-windows-dbg/tensorflow/workspace2.bzl: Extension file 'third_party/py/python_configure.bzl' has errors Loading: 0 packages loaded Fetching @tf_toolchains; fetching Fetching @io_bazel_rules_closure; fetching Fetching @tf_runtime; fetching PS C:\vcpkg\buildtrees\tensorflow-cc\x64-windows-dbg>

shelllet commented 2 years ago

if i manually build:

bazel build //tensorflow/tools/pip_package:build_pip_package

image

but python3.exe exist in that folder.

laszlocsomor commented 2 years ago

I think you're in the wrong directory. x64-windows-dbg looks like an output directory. Change to the root directory of the Tensorflow source, i.e. the directory with the WORKSPACE file, and run Bazel from there.

For python, use a Windows-native Python3 binary, not an MSYS one.

shelllet commented 2 years ago

image

WORKSPACE exist in x64-windows-dbg.

image

image

seems bazel script can't get env variable value.

shelllet commented 2 years ago

Does bazel read the cached environment variables？

laszlocsomor commented 2 years ago

PYTHON_BIN_PATH must point to the binary itself, not just the directory containing it.

Does bazel read the cached environment variables？

What do you mean by cached?

laszlocsomor commented 2 years ago

Hold on, my memory serves wrong. PYTHON_BIN_PATH doesn't seem to be used anywhere:

https://cs.opensource.google/search?q=PYTHON_BIN_PATH&ss=bazel%2Fbazel

shelllet commented 2 years ago

https://github.com/tensorflow/tensorflow/blob/master/third_party/py/python_configure.bzl

"""Repository rule for Python autoconfiguration. python_configure depends on the following environment variables:

laszlocsomor commented 2 years ago

There, "location of python binary", I think that means the binary itself, so C:\users\admin\...\python39\python.exe or whatever it is.

shelllet commented 2 years ago

PYTHON_BIN_PATH must point to the binary itself, not just the directory containing it.

Does bazel read the cached environment variables？

What do you mean by cached?

bazel script can't get the real env value , so i think it read value from another place .

shelllet commented 2 years ago

There, "location of python binary", I think that means the binary itself, so C:\users\admin\...\python39\python.exe or whatever it is.

I have tried it, but I still get the same error image

shelllet commented 2 years ago

bazel script can't get the real env (PYTHON_BIN_PATH) variable's value , this same as BAZEL_VC and BAZEL_VS BAZEL_VC_FULL_VERSION.

laszlocsomor commented 2 years ago

From the --define PYTHON_BIN_PATH=... in the error message, I suspect something still picks up bazelrc files. Are you running Bazel with --ignore_all_rc_files?

Sorry, I probably can't help with Python problems.

To debug the VC version detection issue, try creating a clean Bazel workspace (= a new directory with an empty WORKSPACE and an empty BUILD file), and run the bazel query command I sent earlier from that directory.

shelllet commented 2 years ago

like this ? thks, i will try to create new directory and test.

image

laszlocsomor commented 2 years ago

Ah, sorry if my instructions were misleading.

I meant using the --ignore_all_rc_files flag only for debugging why Bazel picks up the wrong VC version.

Don't use this flag if you're actually trying to build something. The bazelrc files are necessary; but for debugging, i wanted to rule them out.

shelllet commented 2 years ago

i can't get more useful message from command bazel --batch --ignore_all_rc_files query @local_config_cc//...,

becuase it faild to execute. image

laszlocsomor commented 2 years ago

Yes, I think you need to try the query command in an empty workspace. (A directory with nothing but an empty WORKSPACE and BUILD file in it.)

JackBoosY commented 2 years ago

The question is why undname is not found in the Visual Studio 2022 directory. image I think it is correct for Bazel to use 2022 first, but there is the above problem.

shelllet commented 2 years ago

windows_cc_configure.bzl does not seem to support vs2022. is this cause the above problem?

code snippet in windows_cc_configure.bzl.

  for path in [
        "Microsoft Visual Studio\\2019\\Preview\\VC",
        "Microsoft Visual Studio\\2019\\BuildTools\\VC",
        "Microsoft Visual Studio\\2019\\Community\\VC",
        "Microsoft Visual Studio\\2019\\Professional\\VC",
        "Microsoft Visual Studio\\2019\\Enterprise\\VC",
        "Microsoft Visual Studio\\2017\\BuildTools\\VC",
        "Microsoft Visual Studio\\2017\\Community\\VC",
        "Microsoft Visual Studio\\2017\\Professional\\VC",
        "Microsoft Visual Studio\\2017\\Enterprise\\VC",
        "Microsoft Visual Studio 14.0\\VC",
    ]:
def _is_vs_2017_or_2019(vc_path):
...
meteorcloudy commented 2 years ago

@andrewkatson It looks like we don't have proper support for Vs2022 yet, can you help look into this?

JackBoosY commented 2 years ago

@andrewkatson It looks like we don't have proper support for Vs2022 yet, can you help look into this?

See log content:

Auto-Configuration Error: Couldn't find undname.exe under C:\Program Files\Microsoft Visual Studio\2022\Community\VC, please check your VC installation and set BAZEL_VC environment variable correctly.
ERROR: Analysis of target '//tensorflow:tensorflow_cc.dll' failed; build aborted: Analysis failed

Please note The Visual Studio 2022 path C:\Program Files\Microsoft Visual Studio\2022\Community\VC is correct.

meteorcloudy commented 2 years ago

@JackBoosY Thanks! Have you figured out why undname.exe no longer exists in VS 2022? It's needed to build TensorFlow on Windows: https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/tools/def_file_filter/def_file_filter_configure.bzl;l=35?

JackBoosY commented 2 years ago

@meteorcloudy See my comment https://github.com/bazelbuild/bazel/issues/10772#issuecomment-962924219

image

Please note the relative path of undname in Visual Studio has never been changed.

meteorcloudy commented 2 years ago

Oh, I see, then find_vc_path must have returned the wrong result. Can you show me what's under the VS 2022 path? Maybe the directory structure changed again? https://github.com/bazelbuild/bazel/blob/master/tools/cpp/windows_cc_configure.bzl#L260

JackBoosY commented 2 years ago

@meteorcloudy As I said, the path structure of all files has not changed.

SuperKogito commented 1 year ago

The path is actually different. Although, Bazel does not account for VS2022 yet but the if condition for VS2017 and VS2019 can still do the trick. The difference between the old version and the new version is the ../Hostx64/.. label. This is lowercase in your case for example and mine too (VS2022). However, the windows_cc_configure.bz script: line 325 looks for an uppercase ../HostX64/.. label.

The solution here is either to edit ../Hostx64/.. label (https://github.com/bazelbuild/bazel/blob/9c96529fca4a135c162e35ce2882834b879438fb/tools/cpp/windows_cc_configure.bzl#L325C1-L326C1) and change the lines to 

    if _is_vs_2017_or_2019(repository_ctx, vc_path):
        lib = msvc_vars_x64["%{msvc_env_lib_x64}"]
        full_version = _get_vc_full_version(repository_ctx, vc_path)
        tools_path = "%s\\Tools\\MSVC\\%s\\bin\\Hostx64\\%s" % (vc_path, full_version, target_arch)

        # For native windows(10) on arm64 builds host toolchain runs in an emulated x86 environment
        if not repository_ctx.path(tools_path).exists:
            tools_path = "%s\\Tools\\MSVC\\%s\\bin\\Hostx86\\%s" % (vc_path, full_version, target_arch)

or you rename \bin\\Hostx64\\ to \bin\\HostX64\\

also it is vital that these environment variables point to the right paths: 

BAZEL_SH=C:\msys64\usr\bin\bash.exe
BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC
BAZEL_VC_FULL_VERSION=14.36.32532

This was tested with: Windows 10 Bazel 6.2.1 Python 3.9.7 Tensorflow 2.12

fyi: I used the first solution.

JackBoosY commented 1 year ago

cc @Cheney-W to check this path.

meteorcloudy commented 1 year ago

Relevant issue and pr: https://github.com/bazelbuild/bazel/issues/18592 https://github.com/bazelbuild/bazel/pull/18608