search

4ms / metamodule

MetaModule virtual patch module firmware
Other
9 stars 0 forks source link

VCV CI fails on latest Github ubuntu runner #280

Closed danngreen closed 4 months ago

danngreen commented 4 months ago

The issue started with Github runner upgrade to 20240609.1.0 (worked in 20240603.1.0).

Re-running an action that previous succeeded using image version 20240603.1.0, fails now with image version 20240609.1.0

For example, this succeeded: https://github.com/4ms/metamodule/actions/runs/9459666614/attempts/1 and re-running it with the latest runner failed: https://github.com/4ms/metamodule/actions/runs/9459666614

They freeze on DLDCore.

Windows and Linux both fail, and these two run on the ubuntu runner. Macos succeeds, but it runs on a macos runner.

danngreen commented 4 months ago

If I comment out compilation of CoreModules, then the CI build reports a compiler bug:

In file included from src/hub/hub_module.hh:3,
                 from src/hub/knob_set_menu.hh:2,
                 from src/hub/hub_module_widget.hh:5,
                 from src/hub/hub_elements.hh:4,
                 from src/hub/hub_medium.cc:5:
src/hub/hub_knob_mappings.hh: In instantiation of 'HubKnobMappings<NumKnobs, MaxMapsPerPot, MaxKnobSets>::HubKnobMappings() [with long unsigned int NumKnobs = 12; long unsigned int MaxMapsPerPot = 8; long unsigned int MaxKnobSets = 8]':
src/hub/hub_module.hh:15:8:   required from here
src/hub/hub_knob_mappings.hh:395:22: internal compiler error: Segmentation fault
  395 |         KnobMultiMap nullmap{};
      |                      ^~~~~~~
0x7fddc5cc2e3f __libc_start_main
    ???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.

The Linux build is using g++ version: x86_64-ubuntu16.04-linux-gnu-g++ (crosstool-NG 1.25.0.206_e63c408) 13.1.0

Update: Building the above with -freport-bug tells us:

The bug is not reproducible, so it is likely a hardware or OS problem.
danngreen commented 4 months ago

I found out that commenting out these lines in the info file would let DLD compile:

AltParamChoiceLabeled{{{to_mm<72>(29.0), to_mm<72>(13.0), Center, "Soft Clip A", ""}, 2, 1}, {"On", "Off"}},
AltParamChoiceLabeled{{{to_mm<72>(229.0), to_mm<72>(13.0), Center, "Soft Clip B", ""}, 2, 1}, {"On", "Off"}},
AltParamChoiceLabeled{{{to_mm<72>(12.0), to_mm<72>(13.0), Center, "AutoMute A", ""}, 2, 1}, {"On", "Off"}},
AltParamChoiceLabeled{{{to_mm<72>(212.0), to_mm<72>(13.0), Center, "AutoMute B", ""}, 2, 1}, {"On", "Off"}},
//..etc
AltParamChoiceLabeled{{{to_mm<72>(254.17), to_mm<72>(53.39), Center, "Crossfade B", ""}, 7, 4}, {"1ms", "2ms", "4ms", "8ms", "25ms", "100ms", "250ms"}},

In fact, any info file that contained more than 4 Elements constructed with an AltParamChoice or AltParamChoiceLabeled would crash the CI machine. Running the CI with docker on my own machine, I can see that when g++-13 tries to compile any file that includes an info with >= 4 AltParams, then CPU hits 100% and RAM usage slowly climbs until g++ terminates.

Even these four lines in any file will crash the CI runner:

static constexpr MetaModule::Element e1 = MetaModule::AltParamChoiceLabeled{};
static constexpr MetaModule::Element e2 = MetaModule::AltParamChoiceLabeled{};
static constexpr MetaModule::Element e3 = MetaModule::AltParamChoiceLabeled{};
static constexpr MetaModule::Element e4 = MetaModule::AltParamChoiceLabeled{};

If we construct just AltParamChoiceLabeled (and not a std::variant with that type), then the CI will not crash.

I haven't found a bug report for this yet, and it might not be a bug, it might just be a huge memory burden that the CI runner can't handle (but most dev's machines handle it OK, at least)

I did find this, from VS C++: https://developercommunity.visualstudio.com/t/compile-time-explosion-with-static-stdvariant/1660892

Seems similar... I tried creating constructors for the AltParam types, but it didn't change anything.

What I did find almost works is to re-order the elements of std::variant. Putting the AltParam members towards the top lets CI compile some files, but still fails in others (here it fails with the SHEV: https://github.com/4ms/metamodule/actions/runs/9704837444/job/26785815699 )

danngreen commented 4 months ago

Using gcc 13.2 solves the issue for the linux build. I've forked and updated the containers to use this.

But windows is using gcc 14.1.0 and it now fails with another compile bug at a new location:

x86_64-w64-mingw32-g++ -DTEST -DVCVRACK -std=c++11 -Wsuggest-override  -std=c++2a  -I../shared/cpputil -I../shared/patch_convert/ryml/rapidyaml/src -I../shared/patch_convert/ryml/rapidyaml/ext/c4core/src -I../shared/patch_convert/ryml -Isrc -I../shared -I../firmware/metamodule-plugin-sdk/metamodule-core-interface -I../shared/CoreModules/4ms -I../shared/CoreModules/4ms/core/tapo/stmlib -I../shared/CoreModules/4ms/core/tapo/ -I../shared/CoreModules/4ms/core/alpaca/include -I../shared/CoreModules/4ms/core/looping-delay/src -I../shared/CoreModules/4ms/core/ -I../shared/CoreModules/4ms/core/peg-common/ -I../shared/CoreModules/4ms/core/peg-common/mocks -g2 -fPIC -I/home/build/rack-plugin-toolchain/Rack-SDK-win-x64/include -I/home/build/rack-plugin-toolchain/Rack-SDK-win-x64/dep/include -MMD -MP -g -O3 -funsafe-math-optimizations -fno-omit-frame-pointer -Wall -Wextra -Wno-unused-parameter -march=nehalem -D_USE_MATH_DEFINES -municode  -c -o build/../shared/CoreModules/4ms/core/tapo/control.cc.o ../shared/CoreModules/
during RTL pass: dwarf2
../shared/CoreModules/4ms/core/tapo/control.cc: In member function 'void TapoDelay::Control::Calibrate()':
../shared/CoreModules/4ms/core/tapo/control.cc:80:1: internal compiler error: in dwarf2out_frame_debug_expr, at dwarf2cfi.cc:2130
   80 | }
      | ^
0x20352ac internal_error(char const*, ...)
    ???:0
0x780f51 fancy_abort(char const*, int, char const*)
    ???:0
Please submit a full bug report, with preprocessed source (by using -freport-bug).
Please include the complete backtrace with any bug report.
danngreen commented 4 months ago

Finally! using gcc 13.2.0 worked on windows.

All targets are building.

It's a little disturbing that gcc 13.1.0 and 14.1.0 mingw32 and 13.1.0 linux report compile bugs. Not sure what to make of that...

For posterity, here are the compilers that either crash due to RAM usage when compiled on the github runner, or inform us they encountered a "compiler bug" when compiling the vcv project:

There might be differences in libc++ version involved as well.