llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
27.86k stars 11.48k forks source link

Test clang-tools-extra/test/clang-doc/basic-project.test failing when built using non-ninja generator #97507

Closed dyung closed 1 month ago

dyung commented 2 months ago

We have an internal job which attempts to build the LLVM tree using the Visual Studio 2019 builder configuration (non-ninja) and it recently started hitting a test failure of clang-tools-extra/test/clang-doc/basic-project.test which I bisected back to when f14ad74478494cbd9d23af841d6f6b2808afda71 (#96358) was committed. The problem is that the change hard-codes how to find the dependent assets as ../share/clang-doc. This is fine in ninja builds, but fails with the Visual Studio generator because the configuration name is part of the path.

For example, in a ninja build, the value of AssetsPath is (correctly) C:\Dev\git\merge\build\bin\..\share\clang-doc\index.js. But when using a Visual Studio generator, the value of AssetsPath is now C:\Dev\git\merge\build\Debug\bin\..\share\clang-doc\index.js which refers to a non-existent directory. In this particular case, it requires an extra .. in the path to find the expected directory.

Lit test output:

FAIL: Clang Tools :: clang-doc/basic-project.test (1 of 1)
******************** TEST 'Clang Tools :: clang-doc/basic-project.test' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 1
rm -rf C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp && mkdir -p C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build
# executed command: rm -rf 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp'
# executed command: mkdir -p 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs' 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build'
# RUN: at line 2
sed 's|$test_dir|C:/Dev/git/merge/clang-tools-extra/test/clang-doc|g' C:\Dev\git\merge\clang-tools-extra\test\clang-doc/Inputs/basic-project/database_template.json > C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json
# executed command: sed 's|$test_dir|C:/Dev/git/merge/clang-tools-extra/test/clang-doc|g' 'C:\Dev\git\merge\clang-tools-extra\test\clang-doc/Inputs/basic-project/database_template.json'
# RUN: at line 3
clang-doc --format=html --output=C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs --executor=all-TUs C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json
# executed command: clang-doc --format=html '--output=C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/docs' --executor=all-TUs 'C:\Dev\git\merge\build\tools\clang\tools\extra\test\clang-doc\Output\basic-project.test.tmp/build/compile_commands.json'
# .---command stdout------------
# | Emiting docs in html format.
# `-----------------------------
# .---command stderr------------
# | default index.js file missing at C:\Dev\git\merge\build\Debug\bin\..\share\clang-doc\index.js
# |
# `-----------------------------
# error: command failed with exit status: 1
cor3ntin commented 1 month ago

There still are some failing builds caused by basic-project.test

https://lab.llvm.org/buildbot/#/builders/35/builds/1367 https://github.com/llvm/llvm-project/pull/100576 https://buildkite.com/llvm-project/github-pull-requests/builds/85141 https://buildkite.com/llvm-project/github-pull-requests/builds/85249 https://github.com/search?q=repo%3Allvm%2Fllvm-project+basic-project.test&type=pullrequests

Can you look into it? Thanks! (Disabling the test might be an acceptable short term solution)

Endilll commented 1 month ago

I disabled the test in 88549cf47cafc4c4a6042393ee07fc2dc20566cc