search

dotnet / aspire

An opinionated, cloud ready stack for building observable, production ready, distributed applications in .NET
https://learn.microsoft.com/dotnet/aspire
MIT License
3.39k stars 350 forks source link

[tests] Playwright dependent tests fail randomly on CI/Linux #4623

Open radical opened 2 weeks ago

radical commented 2 weeks ago

Some helix agents failing with Bad return status for module build on kernel: 6.5.0-1022-azure (x86_64) when installing some packages with sudo apt-get -o DPkg::Lock::Timeout=120 install -y libgbm1 libxkbcommon0:

Loading new lttng-modules-2.13.8 DKMS files...
Building for 6.5.0-1022-azure
Building initial module for 6.5.0-1022-azure
Error! Bad return status for module build on kernel: 6.5.0-1022-azure (x86_64)
Consult /var/lib/dkms/lttng-modules/2.13.8/build/make.log for more information.
dpkg: error processing package lttng-modules-dkms (--configure):
 installed lttng-modules-dkms package post-installation script subprocess returned error exit status 10

Only some helix agents seem to fail. For example - failure case from a failing build. And a working case from a working build.

This has been happening pretty randomly, but frequently enough to be blocking dotnet/aspire PRs. This helix run has all passing jobs.

I haven't been able to connect to a helix-repro-vm to reproduce the issue. Does it sound like an issue with some particular running VMs, or the image itself? The suggested alternative was https://github.com/dotnet/dnceng/issues/3122 - which would mean that we can drop this apt-get invocation.

cc @joperezr @eerhardt

(moved from https://github.com/dotnet/dnceng/issues/3140)

radical commented 2 weeks ago

make.log on build:

make: Entering directory '/usr/src/linux-headers-6.5.0-1022-azure'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
  You are using:           gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:27: File ./mm/slab.h not found. Probe "kmem" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:80: File ./arch/x86/kvm/lapic.h not found. Probe "kvm" x86-specific is disabled. Use full kernel source tree to enable it.
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_backend.o
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:187: Files ./fs/btrfs/*.h not found. Probe "btrfs" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:206: Files ./mm/internal.h not found. Probe "compaction" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:215: Files ./fs/ext4/*.h not found. Probe "ext4" is disabled. Use full kernel source tree to enable it.
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/Kbuild:244: File ./drivers/base/regmap/trace.h not found. Probe "regmap" is disabled. Need Linux 4.1+ kernel source tree to enable it.
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-sched.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_frontend.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-irq.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-timer.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_iterator.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-module.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_vfs.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-power.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_splice.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-statedump.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_mmap.o
/var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_mmap.c: In function ‘lib_ring_buffer_mmap_buf’:
/var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_mmap.c:116:23: error: assignment of read-only member ‘vm_flags’
  116 |         vma->vm_flags |= VM_DONTEXPAND;
      |                       ^~
make[4]: *** [scripts/Makefile.build:251: /var/lib/dkms/lttng-modules/2.13.8/build/src/lib/ringbuffer/ring_buffer_mmap.o] Error 1
make[3]: *** [scripts/Makefile.build:488: /var/lib/dkms/lttng-modules/2.13.8/build/src/lib] Error 2
make[3]: *** Waiting for unfinished jobs....
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-9p.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-i2c.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-kvm.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-x86-irq-vectors.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-x86-exceptions.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-signal.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-block.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-napi.o
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-skb.o
In file included from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/define_trace.h:87,
                 from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/instrumentation/events/skb.h:92,
                 from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-skb.c:29:
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:133:6: error: conflicting types for ‘trace_consume_skb’; have ‘void(struct sk_buff *)’
  133 | void trace_##_name(_proto);
      |      ^~~~~~
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:45:9: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP’
   45 |         LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP(map, name, map, PARAMS(proto), PARAMS(args))
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/instrumentation/events/skb.h:64:1: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_MAP’
   64 | LTTNG_TRACEPOINT_EVENT_MAP(consume_skb,
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/trace/events/skb.h:10,
                 from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-skb.c:18:
./include/linux/tracepoint.h:255:28: note: previous definition of ‘trace_consume_skb’ with type ‘void(struct sk_buff *, void *)’
  255 |         static inline void trace_##name(proto)                          \
      |                            ^~~~~~
./include/linux/tracepoint.h:432:9: note: in expansion of macro ‘__DECLARE_TRACE’
  432 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
      |         ^~~~~~~~~~~~~~~
./include/linux/tracepoint.h:566:9: note: in expansion of macro ‘DECLARE_TRACE’
  566 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
      |         ^~~~~~~~~~~~~
./include/trace/events/skb.h:54:1: note: in expansion of macro ‘TRACE_EVENT’
   54 | TRACE_EVENT(consume_skb,
      | ^~~~~~~~~~~
make[4]: *** [scripts/Makefile.build:251: /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-skb.o] Error 1
make[4]: *** Waiting for unfinished jobs....
  CC [M]  /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-net.o
make[3]: *** [scripts/Makefile.build:488: /var/lib/dkms/lttng-modules/2.13.8/build/src/probes] Error 2
make[2]: *** [scripts/Makefile.build:488: /var/lib/dkms/lttng-modules/2.13.8/build/src] Error 2
make[1]: *** [/usr/src/linux-headers-6.5.0-1022-azure/Makefile:2039: /var/lib/dkms/lttng-modules/2.13.8/build] Error 2
make: *** [Makefile:234: __sub-make] Error 2
make: Leaving directory '/usr/src/linux-headers-6.5.0-1022-azure'
radical commented 2 weeks ago

Relevant tests were disabled in https://github.com/dotnet/aspire/pull/4694 .