FloopCZ / tensorflow_cc

Build and install TensorFlow C++ API library.
MIT License
759 stars 183 forks source link

error downloads file from llvm repository #294

Closed yaira24 closed 2 years ago

yaira24 commented 2 years ago

user@user-Default-string:~/tensorflow_cc/tensorflow_cc/build$ cmake -DALLOW_CUDA=ON -DLOCAL_RAM_RESOURCES=2048 -DLOCAL_CPU_RESOURCES=4 .. -- The C compiler identification is GNU 9.4.0 -- The CXX compiler identification is GNU 9.4.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done -- Generating done -- Build files have been written to: /home/user/tensorflow_cc/tensorflow_cc/build user@user-Default-string:~/tensorflow_cc/tensorflow_cc/build$ sudo make Scanning dependencies of target tensorflow_base [ 12%] Creating directories for 'tensorflow_base' [ 25%] Performing download step (download, verify and extract) for 'tensorflow_base' -- Downloading... dst='/home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/v2.9.0.tar.gz' timeout='none' -- Using src='https://github.com/tensorflow/tensorflow/archive/v2.9.0.tar.gz' -- Downloading... done -- extracting... src='/home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/v2.9.0.tar.gz' dst='/home/user/tensorflow_cc/tensorflow_cc/build/tensorflow' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 37%] No patch step for 'tensorflow_base' [ 50%] No update step for 'tensorflow_base' [ 62%] No configure step for 'tensorflow_base' [ 75%] Performing build step for 'tensorflow_base' CUDA support disabled You have bazel 5.0.0 installed. Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=mkl_aarch64 # Build with oneDNN and Compute Library for the Arm Architecture (ACL). --config=monolithic # Config for mostly static monolithic build. --config=numa # Build with NUMA support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. --config=v1 # Build with TensorFlow 1 API instead of TF 2 API. Preconfigured Bazel build configs to DISABLE default on features: --config=nogcp # Disable GCP support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished Extracting Bazel installation... Starting local Bazel server and connecting to it... INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=203 INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.tf_configure.bazelrc: 'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/local/lib/python3.8/dist-packages --python_path=/usr/bin/python3 INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: 'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils INFO: Found applicable config definition build:short_logs in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:opt in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.tf_configure.bazelrc: --copt=-march=haswell --host_copt=-march=haswell INFO: Found applicable config definition build:monolithic in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define framework_shared_object=false INFO: Found applicable config definition build:linux in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/093ed77f7d50f75b376f40a71ea86e08cedb8b80.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found INFO: Repository llvm-raw instantiated at: /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/WORKSPACE:11:14: in /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/tensorflow/workspace3.bzl:42:9: in workspace /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/third_party/llvm/workspace.bzl:10:20: in repo /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/third_party/repo.bzl:128:21: in tf_http_archive Repository rule _tf_http_archive defined at: /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/third_party/repo.bzl:81:35: in WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found WARNING: Download from https://github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz failed: class java.io.IOException Read timed out ERROR: An error occurred during the fetch of repository 'llvm-raw': Traceback (most recent call last): File "/home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/third_party/repo.bzl", line 64, column 33, in _tf_http_archive_impl ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz, https://github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz] to /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/llvm-raw/temp13528757691160253593/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz: Read timed out ERROR: /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/WORKSPACE:11:14: fetching _tf_http_archive rule //external:llvm-raw: Traceback (most recent call last): File "/home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/third_party/repo.bzl", line 64, column 33, in _tf_http_archive_impl ctx.download_and_extract( Error in download_and_extract: java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz, https://github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz] to /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/llvm-raw/temp13528757691160253593/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz: Read timed out ERROR: no such package '@llvm-raw//utils/bazel': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz, https://github.com/llvm/llvm-project/archive/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz] to /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/llvm-raw/temp13528757691160253593/1cb299165c859533e22f2ed05eb2abd5071544df.tar.gz: Read timed out INFO: Elapsed time: 172.578s INFO: 0 processes. FAILED: Build did NOT complete successfully (0 packages loaded) make[2]: [CMakeFiles/tensorflow_base.dir/build.make:120: tensorflow-stamp/tensorflow_base-build] Error 1 make[1]: [CMakeFiles/Makefile2:76: CMakeFiles/tensorflow_base.dir/all] Error 2 make: *** [Makefile:130: all] Error 2

FloopCZ commented 2 years ago

This sometimes happens in case of connection outages. Have you tried again?

yaira24 commented 2 years ago

I tried several times on the same day and get the same error. I tried today and got this error on the same lib find a fix to remove WORKSPACE files by using the command "bazel clean --expunge" ./tensorflow_cc/build/tensorflow/tensorflow/compiler/mlir/hlo/WORKSPACE ./tensorflow_cc/build/tensorflow/WORKSPACE and clean build dir that build is finished ok thank you for the help, you can close the case

user@user-Default-string:~/tensorflow_cc/tensorflow_cc/build$ sudo make [sudo] password for user: [ 12%] Performing build step for 'tensorflow_base' CUDA support disabled You have bazel 5.0.0 installed. Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details. --config=mkl # Build with MKL support. --config=mkl_aarch64 # Build with oneDNN and Compute Library for the Arm Architecture (ACL). --config=monolithic # Config for mostly static monolithic build. --config=numa # Build with NUMA support. --config=dynamic_kernels # (Experimental) Build kernels into separate shared objects. --config=v1 # Build with TensorFlow 1 API instead of TF 2 API. Preconfigured Bazel build configs to DISABLE default on features: --config=nogcp # Disable GCP support. --config=nonccl # Disable NVIDIA NCCL support. Configuration finished INFO: Options provided by the client: Inherited 'common' options: --isatty=1 --terminal_columns=203 INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: Inherited 'common' options: --experimental_repo_remote_exec INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: 'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.tf_configure.bazelrc: 'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/local/lib/python3.8/dist-packages --python_path=/usr/bin/python3 INFO: Reading rc options for 'build' from /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: 'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils INFO: Found applicable config definition build:short_logs in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING INFO: Found applicable config definition build:v2 in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1 INFO: Found applicable config definition build:opt in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.tf_configure.bazelrc: --copt=-march=haswell --host_copt=-march=haswell INFO: Found applicable config definition build:monolithic in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define framework_shared_object=false INFO: Found applicable config definition build:linux in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --copt=-w --host_copt=-w --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++14 --host_cxxopt=-std=c++14 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes INFO: Found applicable config definition build:dynamic_kernels in file /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/081771d4a0e9d7d3aa0eed2ef389fa4700dfb23e.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found INFO: --discard_analysis_cache was used in the previous build, discarding analysis cache. DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1596824487 -0400" DEBUG: Repository io_bazel_rules_docker instantiated at: /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/WORKSPACE:23:14: in /home/user/tensorflow_cc/tensorflow_cc/build/tensorflow/tensorflow/workspace0.bzl:107:34: in workspace /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/bazel_toolchains/repositories/repositories.bzl:35:23: in repositories Repository rule git_repository defined at: /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in INFO: Analyzed 2 targets (0 packages loaded, 19897 targets configured). INFO: Found 2 targets... ERROR: /root/.cache/bazel/_bazel_root/166dc008255d87ba9a41633d7eada8a0/external/llvm-project/mlir/BUILD.bazel:3329:11: Compiling mlir/lib/Dialect/GPU/IR/GPUDialect.cpp failed: undeclared inclusion(s) in rule '@llvm-project//mlir:GPUDialect': this rule is missing dependency declarations for the following files included by 'mlir/lib/Dialect/GPU/IR/GPUDialect.cpp': 'bazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/GPUBaseIncGen/mlir/Dialect/GPU/GPUOpsDialect.h.inc' 'bazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/GPUBaseIncGen/mlir/Dialect/GPU/GPUOpsDialect.cpp.inc' INFO: Elapsed time: 1231.495s, Critical Path: 95.17s INFO: 991 processes: 9 internal, 982 local. FAILED: Build did NOT complete successfully make[2]: [CMakeFiles/tensorflow_base.dir/build.make:120: tensorflow-stamp/tensorflow_base-build] Error 1 make[1]: [CMakeFiles/Makefile2:76: CMakeFiles/tensorflow_base.dir/all] Error 2 make: *** [Makefile:130: all] Error 2