microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.05k stars 2.83k forks source link

[Build] Error #14940

Closed zshn25 closed 1 year ago

zshn25 commented 1 year ago

Describe the issue

subprocess.CalledProcessError: Command '['/home/zsuri/cmake-3.25.2-linux-aarch64/bin/cmake', '--build', '/home/zsuri/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2.

Urgency

No response

Target platform

NVIDIA Jetson Xavier NX

Build script

./build.sh --config Release --update --build --parallel --build_wheel  --use_tensorrt --cuda_home /usr/local/cuda --cudnn_home /usr/lib/aarch64-linux-gnu  --tensorrt_home /usr/lib/aarch64-linux-gnu

Error / output


[ 77%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/quantization/quant_gemm.cc.o
/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cuda/bert/rotary_embedding_util.h(178): error: non-void function "onnxruntime::cuda::add(onnxruntime::cuda::Float8_, onnxruntime::cuda::Float8_)" should return a value

1 error detected in the compilation of "/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cuda/bert/add_bias_transpose.cu".
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/build.make:5152: CMakeFiles/onnxruntime_providers_cuda.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cuda/bert/add_bias_transpose.cu.o] Error 255
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/quantization/quantize_ops.cc.o
[ 77%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/sample.cc.o
make[1]: *** [CMakeFiles/Makefile2:1953: CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/skip_layer_norm.cc.o
[ 77%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/tokenizer.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/beam_search.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/beam_search_parameters.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/beam_search_scorer.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/dump_tensor.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/generation_device_helper.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/greedy_search.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/greedy_search_parameters.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/logits_processor.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/sampling.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/sampling_parameters.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/sequences.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/subgraph_base.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/subgraph_gpt.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/subgraph_t5_decoder.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/transformers/subgraph_t5_encoder.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/unique.cc.o
[ 78%] Building CXX object CMakeFiles/onnxruntime_providers.dir/home/zsuri/onnxruntime/onnxruntime/contrib_ops/cpu/word_conv_embedding.cc.o
[ 78%] Linking CXX static library libonnxruntime_providers.a
[ 78%] Built target onnxruntime_providers
make: *** [Makefile:166: all] Error 2
Traceback (most recent call last):
  File "/home/zsuri/onnxruntime/tools/ci_build/build.py", line 2765, in <module>
    sys.exit(main())
  File "/home/zsuri/onnxruntime/tools/ci_build/build.py", line 2662, in main
    build_targets(args, cmake_path, build_dir, configs, num_parallel_jobs, args.target)
  File "/home/zsuri/onnxruntime/tools/ci_build/build.py", line 1420, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/home/zsuri/onnxruntime/tools/ci_build/build.py", line 769, in run_subprocess
    return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
  File "/home/zsuri/onnxruntime/tools/python/util/run.py", line 49, in run
    completed_process = subprocess.run(
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/zsuri/cmake-3.25.2-linux-aarch64/bin/cmake', '--build', '/home/zsuri/onnxruntime/build/Linux/Release', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2

Visual Studio Version

No response

GCC / Compiler Version

CMAKE 3.25.2-linux-aarch64

jchen351 commented 1 year ago

Removing the label quantization from the issue because it is not related to quantization.

@wangyems Can you please take a look at it?

wangyems commented 1 year ago

please have a try https://github.com/microsoft/onnxruntime/compare/main...wangyems-patch-1?quick_pull=1

zshn25 commented 1 year ago

Dear @wangyems, thank you for the PR #14944. I tested it and it solves this issue.