Verilator Binary Broken

[sifferman]

Starting with release 2023-10-21, the Verilator binary gives the error "the coroutine header requires -fcoroutines". There is no error if I build from master.

System Info

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

Test

module tb;
    initial begin
        #1;
        $finish();
    end
endmodule

verilator --binary tb.sv

Full Error

make: Entering directory './obj_dir'
g++ -Os  -I.  -MMD -I./share/verilator/include -I./share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fcf-protection=none -Wno-bool-operation -Wno-overloaded-virtual -Wno-shadow -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-parameter -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable     -DVL_TIME_CONTEXT    -c -o verilated.o ./share/verilator/include/verilated.cpp
g++ -Os  -I.  -MMD -I./share/verilator/include -I./share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=0 -DVM_TRACE_FST=0 -DVM_TRACE_VCD=0 -faligned-new -fcf-protection=none -Wno-bool-operation -Wno-overloaded-virtual -Wno-shadow -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-parameter -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable     -DVL_TIME_CONTEXT    -c -o verilated_timing.o ./share/verilator/include/verilated_timing.cpp
In file included from ./share/verilator/include/verilated_timing.h:45,
                 from ./share/verilator/include/verilated_timing.cpp:23:
/usr/include/c++/11/coroutine:334:2: error: #error "the coroutine header requires -fcoroutines"
  334 | #error "the coroutine header requires -fcoroutines"
      |  ^~~~~
In file included from ./share/verilator/include/verilated_timing.cpp:23:
./share/verilator/include/verilated_timing.h:98:10: error: ‘coroutine_handle’ in namespace ‘std’ does not name a template type
   98 |     std::coroutine_handle<> m_coro;  // The wrapped coroutine handle
      |          ^~~~~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:112:44: error: expected ‘)’ before ‘<’ token
  112 |     VlCoroutineHandle(std::coroutine_handle<> coro, VlProcessRef process, VlFileLineDebug fileline)
      |                      ~                     ^
      |                                            )
./share/verilator/include/verilated_timing.h:114:9: error: expected unqualified-id before ‘,’ token
  114 |         , m_process{process}
      |         ^
./share/verilator/include/verilated_timing.h:115:9: error: expected unqualified-id before ‘,’ token
  115 |         , m_fileline{fileline} {
      |         ^
./share/verilator/include/verilated_timing.h:115:32: error: expected unqualified-id before ‘{’ token
  115 |         , m_fileline{fileline} {
      |                                ^
./share/verilator/include/verilated_timing.h: In constructor ‘VlCoroutineHandle::VlCoroutineHandle(VlProcessRef)’:
./share/verilator/include/verilated_timing.h:108:11: error: class ‘VlCoroutineHandle’ does not have any field named ‘m_coro’
  108 |         : m_coro{nullptr}
      |           ^~~~~~
./share/verilator/include/verilated_timing.h: In constructor ‘VlCoroutineHandle::VlCoroutineHandle(VlCoroutineHandle&&)’:
./share/verilator/include/verilated_timing.h:122:11: error: class ‘VlCoroutineHandle’ does not have any field named ‘m_coro’
  122 |         : m_coro{std::exchange(moved.m_coro, nullptr)}
      |           ^~~~~~
./share/verilator/include/verilated_timing.h:122:38: error: ‘class VlCoroutineHandle’ has no member named ‘m_coro’
  122 |         : m_coro{std::exchange(moved.m_coro, nullptr)}
      |                                      ^~~~~~
In file included from ./share/verilator/include/verilated.h:42,
                 from ./share/verilator/include/verilated_timing.h:28,
                 from ./share/verilator/include/verilated_timing.cpp:23:
./share/verilator/include/verilated_timing.h: In destructor ‘VlCoroutineHandle::~VlCoroutineHandle()’:
./share/verilator/include/verilated_timing.h:129:25: error: ‘m_coro’ was not declared in this scope
  129 |         if (VL_UNLIKELY(m_coro)) {
      |                         ^~~~~~
./share/verilator/include/verilatedos.h:66:45: note: in definition of macro ‘VL_UNLIKELY’
   66 | # define VL_UNLIKELY(x) __builtin_expect(!!(x), 0)  // Prefer over C++20 [[unlikely]]
      |                                             ^
In file included from ./share/verilator/include/verilated_timing.cpp:23:
./share/verilator/include/verilated_timing.h: In member function ‘auto& VlCoroutineHandle::operator=(VlCoroutineHandle&&)’:
./share/verilator/include/verilated_timing.h:139:9: error: ‘m_coro’ was not declared in this scope
  139 |         m_coro = std::exchange(moved.m_coro, nullptr);
      |         ^~~~~~
./share/verilator/include/verilated_timing.h:139:38: error: ‘class VlCoroutineHandle’ has no member named ‘m_coro’
  139 |         m_coro = std::exchange(moved.m_coro, nullptr);
      |                                      ^~~~~~
./share/verilator/include/verilated_timing.h: In member function ‘auto VlDelayScheduler::delay(uint64_t, VlProcessRef, const char*, int)’:
./share/verilator/include/verilated_timing.h:204:32: error: ‘std::coroutine_handle’ has not been declared
  204 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                ^~~
./share/verilator/include/verilated_timing.h:204:53: error: expected ‘,’ or ‘...’ before ‘<’ token
  204 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                                     ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlDelayScheduler::delay(uint64_t, VlProcessRef, const char*, int)::Awaitable::await_suspend(int)’:
./share/verilator/include/verilated_timing.h:205:59: error: ‘coro’ was not declared in this scope
  205 |              queue.push_back({delay, VlCoroutineHandle{coro, process, fileline}});
      |                                                        ^~~~

./share/verilator/include/verilated_timing.h:205:82: error: no matching function for call to ‘VlCoroutineHandle::VlCoroutineHandle(<brace-enclosed initializer list>)’
  205 | ue.push_back({delay, VlCoroutineHandle{coro, process, fileline}});
      |                                                               ^

./share/verilator/include/verilated_timing.h:121:5: note: candidate: ‘VlCoroutineHandle::VlCoroutineHandle(VlCoroutineHandle&&)’
  121 |     VlCoroutineHandle(VlCoroutineHandle&& moved)
      |     ^~~~~~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:121:5: note:   candidate expects 1 argument, 3 provided
./share/verilator/include/verilated_timing.h:107:5: note: candidate: ‘VlCoroutineHandle::VlCoroutineHandle(VlProcessRef)’
  107 |     VlCoroutineHandle(VlProcessRef process)
      |     ^~~~~~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:107:5: note:   candidate expects 1 argument, 3 provided
./share/verilator/include/verilated_timing.h:205:32: error: no matching function for call to ‘std::vector<VlDelayScheduler::VlDelayedCoroutine>::push_back(<brace-enclosed initializer list>)’
  205 |                 queue.push_back({delay, VlCoroutineHandle{coro, process, fileline}});
      |                 ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/c++/11/vector:67,
                 from /usr/include/c++/11/functional:62,
                 from /usr/include/c++/11/pstl/glue_algorithm_defs.h:13,
                 from /usr/include/c++/11/algorithm:74,
                 from ./share/verilator/include/verilated.h:48,
                 from ./share/verilator/include/verilated_timing.h:28,
                 from ./share/verilator/include/verilated_timing.cpp:23:
/usr/include/c++/11/bits/stl_vector.h:1187:7: note: candidate: ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = VlDelayScheduler::VlDelayedCoroutine; _Alloc = std::allocator<VlDelayScheduler::VlDelayedCoroutine>; std::vector<_Tp, _Alloc>::value_type = VlDelayScheduler::VlDelayedCoroutine]’
 1187 |       push_back(const value_type& __x)
      |       ^~~~~~~~~
/usr/include/c++/11/bits/stl_vector.h:1187:35: note:   no known conversion for argument 1 from ‘<brace-enclosed initializer list>’ to ‘const value_type&’ {aka ‘const VlDelayScheduler::VlDelayedCoroutine&’}
 1187 |       push_back(const value_type& __x)
      |                 ~~~~~~~~~~~~~~~~~~^~~
/usr/include/c++/11/bits/stl_vector.h:1203:7: note: candidate: ‘void std::vector<_Tp, _Alloc>::push_back(std::vector<_Tp, _Alloc>::value_type&&) [with _Tp = VlDelayScheduler::VlDelayedCoroutine; _Alloc = std::allocator<VlDelayScheduler::VlDelayedCoroutine>; std::vector<_Tp, _Alloc>::value_type = VlDelayScheduler::VlDelayedCoroutine]’
 1203 |       push_back(value_type&& __x)
      |       ^~~~~~~~~
/usr/include/c++/11/bits/stl_vector.h:1203:30: note:   no known conversion for argument 1 from ‘<brace-enclosed initializer list>’ to ‘std::vector<VlDelayScheduler::VlDelayedCoroutine>::value_type&&’ {aka ‘VlDelayScheduler::VlDelayedCoroutine&&’}
 1203 |       push_back(value_type&& __x)
      |                 ~~~~~~~~~~~~~^~~
In file included from ./share/verilator/include/verilated_timing.cpp:23:
./share/verilator/include/verilated_timing.h: In member function ‘auto VlTriggerScheduler::trigger(bool, VlProcessRef, const char*, const char*, int)’:
./share/verilator/include/verilated_timing.h:259:32: error: ‘std::coroutine_handle’ has not been declared
  259 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                ^~~
./share/verilator/include/verilated_timing.h:259:53: error: expected ‘,’ or ‘...’ before ‘<’ token
  259 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                                     ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlTriggerScheduler::trigger(bool, VlProcessRef, const char*, const char*, int)::Awaitable::await_suspend(int)’:
./share/verilator/include/verilated_timing.h:260:40: error: ‘coro’ was not declared in this scope
  260 |                 suspended.emplace_back(coro, process, fileline);
      |                                        ^~~~
./share/verilator/include/verilated_timing.h: In member function ‘auto VlDynamicTriggerScheduler::awaitable(VlProcessRef, VlDynamicTriggerScheduler::VlCoroutineVec&, const char*, int)’:
./share/verilator/include/verilated_timing.h:307:32: error: ‘std::coroutine_handle’ has not been declared
  307 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                ^~~
./share/verilator/include/verilated_timing.h:307:53: error: expected ‘,’ or ‘...’ before ‘<’ token
  307 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                                     ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlDynamicTriggerScheduler::awaitable(VlProcessRef, VlDynamicTriggerScheduler::VlCoroutineVec&, const char*, int)::Awaitable::await_suspend(int)’:
./share/verilator/include/verilated_timing.h:308:40: error: ‘coro’ was not declared in this scope
  308 |                 suspended.emplace_back(coro, process, fileline);
      |                                        ^~~~
./share/verilator/include/verilated_timing.h: At global scope:
./share/verilator/include/verilated_timing.h:357:24: error: ‘std::coroutine_handle’ has not been declared
  357 |     void await_suspend(std::coroutine_handle<> coro) const { coro.destroy(); }
      |                        ^~~
./share/verilator/include/verilated_timing.h:357:45: error: expected ‘,’ or ‘...’ before ‘<’ token
  357 |     void await_suspend(std::coroutine_handle<> coro) const { coro.destroy(); }
      |                                             ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlForever::await_suspend(int) const’:
./share/verilator/include/verilated_timing.h:357:62: error: ‘coro’ was not declared in this scope
  357 | id await_suspend(std::coroutine_handle<> coro) const { coro.destroy(); }
      |                                                        ^~~~

./share/verilator/include/verilated_timing.h: In member function ‘auto VlForkSync::join(VlProcessRef, const char*, int)’:
./share/verilator/include/verilated_timing.h:392:32: error: ‘std::coroutine_handle’ has not been declared
  392 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                ^~~
./share/verilator/include/verilated_timing.h:392:53: error: expected ‘,’ or ‘...’ before ‘<’ token
  392 |             void await_suspend(std::coroutine_handle<> coro) {
      |                                                     ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlForkSync::join(VlProcessRef, const char*, int)::Awaitable::await_suspend(int)’:
./share/verilator/include/verilated_timing.h:393:33: error: ‘coro’ was not declared in this scope
  393 |                 join->m_susp = {coro, process, fileline};
      |                                 ^~~~
./share/verilator/include/verilated_timing.h:393:56: error: no match for ‘operator=’ (operand types are ‘VlCoroutineHandle’ and ‘<brace-enclosed initializer list>’)
  393 |                 join->m_susp = {coro, process, fileline};
      |                                                        ^
./share/verilator/include/verilated_timing.h:138:11: note: candidate: ‘auto& VlCoroutineHandle::operator=(VlCoroutineHandle&&)’
  138 |     auto& operator=(VlCoroutineHandle&& moved) {
      |           ^~~~~~~~
./share/verilator/include/verilated_timing.h:138:41: note:   no known conversion for argument 1 from ‘<brace-enclosed initializer list>’ to ‘VlCoroutineHandle&&’
  138 |     auto& operator=(VlCoroutineHandle&& moved) {
      |                     ~~~~~~~~~~~~~~~~~~~~^~~~~
./share/verilator/include/verilated_timing.h: At global scope:
./share/verilator/include/verilated_timing.h:409:14: error: ‘coroutine_handle’ in namespace ‘std’ does not name a template type
  409 |         std::coroutine_handle<> m_continuation;  // Coroutine to resume after this one finishes
      |              ^~~~~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:417:14: error: ‘suspend_never’ in namespace ‘std’ does not name a type
  417 |         std::suspend_never initial_suspend() const { return {}; }
      |              ^~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:421:14: error: ‘suspend_never’ in namespace ‘std’ does not name a type
  421 |         std::suspend_never final_suspend() noexcept;
      |              ^~~~~~~~~~~~~
./share/verilator/include/verilated_timing.h:456:24: error: ‘std::coroutine_handle’ has not been declared
  456 |     void await_suspend(std::coroutine_handle<> coro) { m_promisep->m_continuation = coro; }
      |                        ^~~
./share/verilator/include/verilated_timing.h:456:45: error: expected ‘,’ or ‘...’ before ‘<’ token
  456 |     void await_suspend(std::coroutine_handle<> coro) { m_promisep->m_continuation = coro; }
      |                                             ^
./share/verilator/include/verilated_timing.h: In member function ‘void VlCoroutine::await_suspend(int)’:
./share/verilator/include/verilated_timing.h:456:68: error: ‘struct VlCoroutine::VlPromise’ has no member named ‘m_continuation’
  456 | it_suspend(std::coroutine_handle<> coro) { m_promisep->m_continuation = coro; }
      |                                                        ^~~~~~~~~~~~~~

./share/verilator/include/verilated_timing.h:456:85: error: ‘coro’ was not declared in this scope
  456 | d::coroutine_handle<> coro) { m_promisep->m_continuation = coro; }
      |                                                            ^~~~

In file included from ./share/verilator/include/verilated.h:42,
                 from ./share/verilator/include/verilated_timing.h:28,
                 from ./share/verilator/include/verilated_timing.cpp:23:
./share/verilator/include/verilated_timing.cpp: In member function ‘void VlCoroutineHandle::resume()’:
./share/verilator/include/verilated_timing.cpp:31:19: error: ‘m_coro’ was not declared in this scope
   31 |     if (VL_LIKELY(m_coro)) {
      |                   ^~~~~~
./share/verilator/include/verilatedos.h:65:43: note: in definition of macro ‘VL_LIKELY’
   65 | # define VL_LIKELY(x) __builtin_expect(!!(x), 1)  // Prefer over C++20 [[likely]]
      |                                           ^
./share/verilator/include/verilated_timing.cpp: In destructor ‘VlCoroutine::VlPromise::~VlPromise()’:
./share/verilator/include/verilated_timing.cpp:217:9: error: ‘m_continuation’ was not declared in this scope
  217 |     if (m_continuation) m_continuation.destroy();
      |         ^~~~~~~~~~~~~~
./share/verilator/include/verilated_timing.cpp: At global scope:
./share/verilator/include/verilated_timing.cpp:220:6: error: ‘suspend_never’ in namespace ‘std’ does not name a type
  220 | std::suspend_never VlCoroutine::VlPromise::final_suspend() noexcept {
      |      ^~~~~~~~~~~~~
make: *** [./share/verilator/include/verilated.mk:278: verilated_timing.o] Error 1
make: Leaving directory './obj_dir'
%Error: make -C obj_dir -f Vtb.mk -j 1 exited with 2
%Error: Command Failed ulimit -s unlimited 2>/dev/null; exec verilator_bin --binary tb.sv

[sifferman]

Could the issue be with default/patches/verilated.mk.in? Although it looks like it's only being applied for Darwin releases, so I don't know why the Linux release would be affected

Maybe this commit was what broke things: verilator/verilator 8b44a54b

[sifferman]

Yes. I tried building Verilator but with verilator/verilator/include/verilated.mk.in replaced with YosysHQ/oss-cad-suite-build/default/patches/verilated.mk.in, and the error ("the coroutine header requires -fcoroutines") was replicated.

I believe removing the patch file will correct the error.

[sifferman]

Alright, this is the commit that broke things:

https://github.com/verilator/verilator/commit/c7a0613c57827f0f9267fbfb0a064b5133a76678#diff-49473dca262eeab3b4a43002adb08b4db31020d190caaad1594b47f1d5daa810L374-R376

< _MY_CXX_CHECK_IFELSE(
<   -fcoroutines-ts,
<   [CFG_CXXFLAGS_COROUTINES="-fcoroutines-ts"],
<   [CFG_CXXFLAGS_COROUTINES="-fcoroutines"])
---
> _MY_CXX_CHECK_SET(CFG_CXXFLAGS_COROUTINES,-fcoroutines-ts)
> _MY_CXX_CHECK_SET(CFG_CXXFLAGS_COROUTINES,-fcoroutines)
> _MY_CXX_CHECK_SET(CFG_CXXFLAGS_COROUTINES,-fcoroutines-ts -Wno-deprecated-experimental-coroutine)

Verilator used to always include -fcoroutines by default, but now it only includes -fcoroutines if it is supported. I believe that the error is because OSS CAD Suite builds Verilator with a C++ compiler that does not support coroutines. This can be corrected building Verilator with >=g++10

[mmicko]

Thanks for looking into this issue. This is generaly hard to support on multiple linux distributions way we are building it, since it will always be affected by compiler on host machine as well.

Anyway I am in process of updating base OS docker image, but it will take some time.

[sifferman]

Would it be possible to just do a hotfix until the OS docker image is updated? I could throw together a patch if that would be helpful

[mmicko]

@sifferman sure. please do. Will be happy to try it out