search

chipsalliance / verible

Verible is a suite of SystemVerilog developer tools, including a parser, style-linter, formatter and language server
https://chipsalliance.github.io/verible/
Other
1.37k stars 212 forks source link

Build error on Windows #1669

Open LoveMyPillow opened 1 year ago

LoveMyPillow commented 1 year ago

The issue is an extension from the existed issue #958 . However, it seems that the build guidance on windows is still not released, so I open a new issue here.

I try to compile verible on windows10, developer mode as #796 mentioned, and use wsl2 to run bash on windows(because shell.nix said that it need bash). When I run the command: bazel build -c opt --enable_runfiles //... or bazel test --test_output=errors //...

the console print as below: ERROR: D:/s-verible/verible/common/util/BUILD:86:8: Executing genrule //common/util:version_header [for tool] failed: (Exit 127): bash.exe failed: error executing command (from target //common/util:version_header) C:\Windows\system32\bash.exe -c ... (remaining 1 argument skipped) /usr/bin/env: ‘bash\r’: No such file or directory It seems like something about CRLF and CR(I guess, as mention on #796 ), and I don't think I am on the right way to compile this project on windows... Thanks!

hzeller commented 1 year ago

This looks like you're missing the bash shell (which you also need to execute a lot of the end2end tests). So suggest installing that. Maybe it is also /usr/bin/env that it has issues with ? Can you replace the line in common/util/create_version_header.sh with a #! path that points to your bash binary ? I see if I can formulate that in a way so that it is more robust with bazel.

I think you also need the LLVM toolchain as the windows compiler has some incompatibility in the way we assume I think certain iterators work.

The CI runs builds on windows and also releases the artifact in the releases. Maybe this can help you in getting a set-up that works for you.

In general, @corco did a lot of excellent work making it possible to compile on Windows. Maybe he can chime in for questions.

hzeller commented 1 year ago

I've simplified a few things, can you try the latest version in git and see if the compile proceeds further ?

LoveMyPillow commented 1 year ago

I've simplified a few things, can you try the latest version in git and see if the compile proceeds further ?

yes, the compile proceeds further, but it meets new errors. And as to the bash shell and LLVM toolchain, I have installed them before this issue opened.

The error log is attached.

//============================================= D:\S-Verible\verible>bazel build -c opt --enable_runfiles //... INFO: Analyzed 1894 targets (126 packages loaded, 6330 targets configured). INFO: Found 1894 targets... INFO: From Executing genrule //verilog/parser:verilog_lex: verilog/parser/verilog.lex:305: warning, rule cannot be matched verilog/parser/verilog.lex:971: warning, rule cannot be matched ERROR: D:/s-verible/verible/verilog/parser/BUILD:110:24: Executing genrule //verilog/parser:verilog_y_moved failed: (Exit 1): bash.exe failed: error executing command (from target //verilog/parser:verilog_y_moved) C:\Windows\system32\bash.exe -c ... (remaining 1 argument skipped) write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe write failed 32: Broken pipe

hzeller commented 1 year ago

Can you try removing the first line (with #!/bin/sh) in the following two files and see what happens ?

LoveMyPillow commented 1 year ago

Do you mean drop the first line and the content of this line "#! /bin/sh"? Yes, the warning message disappeared but error message remained.

hzeller commented 1 year ago

yes, the first line that contains the /bin/sh

What is the output if you run

bazel build verilog/parser:verilog_y_moved --sandbox_debug
LoveMyPillow commented 1 year ago

The output is: D:\S-Verible\verible>bazel build verilog/parser:verilog_y_moved --sandbox_debug INFO: Build options --compilation_mode and --enable_runfiles have changed, discarding analysis cache. INFO: Analyzed target //verilog/parser:verilog_y_moved (0 packages loaded, 161 targets configured). INFO: Found 1 target... ERROR: D:/s-verible/verible/verilog/parser/BUILD:110:24: Executing genrule //verilog/parser:verilog_y_moved failed: (Exit 1): bash.exe failed: error executing command (from target //verilog/parser:verilog_y_moved) C:\Windows\system32\bash.exe -c ... (remaining 1 argument skipped) write failed 32: Broken pipe write failed 32: Broken pipe //too many duplicated meesage and I hide them :) write failed 32: Broken pipe Target //verilog/parser:verilog_y_moved failed to build Use --verbose_failures to see the command lines of failed build steps. INFO: Elapsed time: 8.383s, Critical Path: 7.68s INFO: 5 processes: 4 internal, 1 local. FAILED: Build did NOT complete successfully

And I noticed that the program suggest me to use --verbose_failures option to see the error message, and I attached the result here. Hope this can do some help.

D:\S-Verible\verible>bazel build verilog/parser:verilog_y_moved --sandbox_debug --verbose_failures INFO: Analyzed target //verilog/parser:verilog_y_moved (0 packages loaded, 0 targets configured). INFO: Found 1 target... ERROR: D:/s-verible/verible/verilog/parser/BUILD:110:24: Executing genrule //verilog/parser:verilog_y_moved failed: (Exit 1): bash.exe failed: error executing command (from target //verilog/parser:verilog_y_moved) cd /d C:/users/10268/_bazel_10268/mxjvn6ym/execroot/com_google_verible SET PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Program Files\Common Files\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;D:\S-Git\cmd;C:\Program Files\Wolfram Research\WolframScript\;D:\S-Bazel;D:\S-Python\Python310\Scripts\;D:\S-Python\Python310\;%USERPROFILE%\AppData\Local\Microsoft\WindowsApps;;D:\intelFPGA\20.1\ModelSim\modelsim_ase\win32aloem;D:\S-Microsoft VS Code\bin;D:\S-Wind.NET.Client\WindNET\bin;D:\S-Wind.NET.Client\WindNET\x64 SET RUNFILES_MANIFEST_ONLY=1 C:\Windows\system32\bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/x64_windows-opt-exec-2B5CBBC6/bin/common/parser/move_yacc_stack_symbols.exe < bazel-out/x64_windows-fastbuild/bin/verilog/parser/verilog.tab.cc > bazel-out/x64_windows-fastbuild/bin/verilog/parser/verilog-moved.tab.cc # Configuration: b26347f9d87ef03821af840fb9b4d879c23c93cd30e48cee845dbeefe1752b5a # Execution platform: @local_config_platform//:host write failed 32: Broken pipe write failed 32: Broken pipe //too many duplicated meesage and I hide them :) write failed 32: Broken pipe Target //verilog/parser:verilog_y_moved failed to build INFO: Elapsed time: 1.013s, Critical Path: 0.59s INFO: 2 processes: 2 internal. FAILED: Build did NOT complete successfully

hzeller commented 1 year ago

mmh, I am wondering why the sh_binary(), which is just a shellscript, becomes move_yacc_stack_symbols.exe; maybe that is what bazel generates on Windows.

Maybe that works not well on that platform. I'll see if I can formulate that inline as sed script that worked with the version_header before. Will be back at the computer in an hour.

corco commented 1 year ago

You may want to take a look at Github's Windows configuration. It has git bash, msys2 bash and WSL bash installed.

I don't remember how the machine I used was configured (I don't use Windows myself...), but I'm pretty sure the bash shell it was using was git bash or MSYS2 bash and not the WSL one... Bazel does a lot of stuff with shell script on Windows, such as generate .exe from them as @hzeller pointed out; it may simply not work in a WSL context.

Bazel itself seems to have (issue with WSL)[https://github.com/bazelbuild/bazel/issues/10941]

LoveMyPillow commented 1 year ago

You may want to take a look at Github's Windows configuration. It has git bash, msys2 bash and WSL bash installed.

I don't remember how the machine I used was configured (I don't use Windows myself...), but I'm pretty sure the bash shell it was using was git bash or MSYS2 bash and not the WSL one... Bazel does a lot of stuff with shell script on Windows, such as generate .exe from them as @hzeller pointed out; it may simply not work in a WSL context.

Bazel itself seems to have (issue with WSL)[https://github.com/[bazelbuild/bazel/issues/10941](https://github.com/bazelbuild/bazel/issues/10941)]

Oh, it seems to point out the key, I will try another bash today.

LoveMyPillow commented 1 year ago

When I use git bash by set the windows environment variable BAZEL_SH to git bash.exe, the project was built successfully and all 568 tests invoked by build test -c opt //... passed.

During compilation, some warning messages were printed: //============================Part of warning messages============================== INFO: From Compiling adler32.c [for tool]: clang-cl: warning: argument unused during compilation: '/std:c++17' [-Wunused-command-line-argument] INFO: From Compiling deflate.c [for tool]: clang-cl: warning: argument unused during compilation: '/std:c++17' [-Wunused-command-line-argument] INFO: From Compiling gzwrite.c [for tool]: clang-cl: warning: argument unused during compilation: '/std:c++17' [-Wunused-command-line-argument] INFO: From Compiling inffast.c [for tool]: clang-cl: warning: argument unused during compilation: '/std:c++17' [-Wunused-command-line-argument] INFO: From Linking external/com_google_protobuf/protoc.exe [for tool]: lld-link: warning: ignoring unknown argument '-lpthread' lld-link: warning: ignoring unknown argument '-lm' lld-link: warning: ignoring unknown argument '-lpthread' lld-link: warning: ignoring unknown argument '-lm' lld-link: warning: ignoring unknown argument '-lpthread' lld-link: warning: ignoring unknown argument '-lm' lld-link: warning: ignoring unknown argument '-lpthread' lld-link: warning: ignoring unknown argument '-lm' //===========================================================================

It seems that these warnings don't affect the test result. Thank you all.

hzeller commented 1 year ago

In #1675 , I skipped using shell scripts for the file modifications but directly do it in the starlark script, which hopefully simplifies it enough to compile hopefully without the BAZEL_SH change you had to do (though shell tests that heavily rely on a working shell might still not work).

Would you like to write a small paragraph w.r.t. to compilation on Windows (what to install, what environment variables to set etc.) that we can put in the developers section of the README @LoveMyPillow ? So that the next person compiling it on Windows can get it working quickly.

LoveMyPillow commented 1 year ago

The following paragraph is what I try to do for building Verible, I don't know if JAVA jdk, python3, gcc and others are needed because I saw them on shell.nix but the .bazelrc specified the compiler to clang-cl, so they were omitted by me. Addtiionally, I use the latest version of these tools. And I have to admit that English is my native language, maybe you will find some grammer errors, :) //===================================================================== To build Verible on Windows, you need:

durongze commented 1 year ago

WARNING: Ignoring JAVA_HOME, because it must point to a JDK, not a JRE. ERROR: C:/users/administrator/_bazel_administrator/rs3thaef/external/local_config_sh/BUILD:6:14: syntax error at 'C': expected , INFO: Repository rules_ruby instantiated at: D:/code/verilog/verible/WORKSPACE:152:14: in C:/users/administrator/_bazel_administrator/rs3thaef/external/com_google_protobuf/protobuf_deps.bzl:109:24: in protobuf_deps C:/users/administrator/_bazel_administrator/rs3thaef/external/com_google_protobuf/protobuf_deps.bzl:20:17: in _github_archive Repository rule http_archive defined at: C:/users/administrator/_bazel_administrator/rs3thaef/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in INFO: repository @rules_ruby' used the following cache hits instead of downloading the corresponding file.

hzeller commented 1 year ago

The Verible binaries are regularly compiled in the CI using the following set-up:

https://github.com/chipsalliance/verible/blob/master/.github/workflows/verible-ci.yml#L481-L512

Here is a typical CI run https://github.com/chipsalliance/verible/actions/runs/5673921578/job/15376447668

If it is not working for you, maybe you can compare and see where your set-up differs ?

corco commented 1 year ago

As far as I recall, WSL2 bash is not supported by bazel. You need a Windows-flavored bash shell like msys2.

In development, I used powershell to start bazel, and it detected/used the bash shell provided by Git Bash for bash scripts.