search

furkanmustafa / forcefully-remove-bootfb

releases memory areas used for BOOTFB for linux/kvm/vfio/pci-passthrough
GNU General Public License v3.0
22 stars 6 forks source link

not working on proxmox 7.2 + linux-headers-5.15.30-2-pve #1

Closed kukat closed 2 years ago

kukat commented 2 years ago

make and ./remove.sh was successful

root@pve:~/forcefully-remove-bootfb# ./remove.sh
++ grep BOOTFB /proc/iomem
++ awk '{ print $1 }'
+ range=d0000000-d1fa3fff
+ IFS=-
+ read -ra ADDR
+ start=d0000000
+ end=d1fa3fff
+ sleep 2
+ insmod build/force-remove-bootfb.ko bootfb_start=0xd0000000 bootfb_end=0xd1fa3fff
+ rmmod force-remove-bootfb

but after reboot, BOOTFB is still there

cc800000-efffffff : PCI Bus 0000:00
  cc800000-cc800fff : 0000:00:1f.5
  d0000000-dfffffff : 0000:00:02.0
    d0000000-d1fa3fff : BOOTFB
  e0000000-e0ffffff : 0000:00:02.0
  e1000000-e10fffff : PCI Bus 0000:02

start vm 

root@pve:~# qm start 102
kvm: -device vfio-pci,host=00:02.0,addr=0x18,x-igd-opregion=on: Failed to mmap 0000:00:02.0 BAR 2. Performance may be slow

PVE version

root@pve:~/forcefully-remove-bootfb# pveversion -V
proxmox-ve: 7.2-1 (running kernel: 5.15.30-2-pve)
pve-manager: 7.2-3 (running version: 7.2-3/c743d6c1)
pve-kernel-helper: 7.2-2
pve-kernel-5.15: 7.2-1
pve-kernel-5.13: 7.1-9
pve-kernel-5.15.30-2-pve: 5.15.30-3
pve-kernel-5.13.19-6-pve: 5.13.19-15
pve-kernel-5.13.19-2-pve: 5.13.19-4
ceph-fuse: 15.2.15-pve1
corosync: 3.1.5-pve2
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.22-pve2
libproxmox-acme-perl: 1.4.2
libproxmox-backup-qemu0: 1.2.0-1
libpve-access-control: 7.1-8
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.1-6
libpve-guest-common-perl: 4.1-2
libpve-http-server-perl: 4.1-1
libpve-storage-perl: 7.2-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 4.0.12-1
lxcfs: 4.0.12-pve1
novnc-pve: 1.3.0-3
proxmox-backup-client: 2.1.8-1
proxmox-backup-file-restore: 2.1.8-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.4-10
pve-cluster: 7.2-1
pve-container: 4.2-1
pve-docs: 7.2-2
pve-edk2-firmware: 3.20210831-2
pve-firewall: 4.2-5
pve-firmware: 3.4-1
pve-ha-manager: 3.3-4
pve-i18n: 2.7-1
pve-qemu-kvm: 6.2.0-5
pve-xtermjs: 4.16.0-1
qemu-server: 7.2-2
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.7.1~bpo11+1
vncterm: 1.7-1
zfsutils-linux: 2.1.4-pve1
furkanmustafa commented 2 years ago

but after reboot, BOOTFB is still there

You need to run remove.sh every time you boot.

d0000000-d1fa3fff : BOOTFB

I don't know, but if this address range is never changing, something like below might work;

# make install # <= this probably won't work. we need to implement this in Makefile
echo "options force-remove-bootfb bootfb_start=0xd0000000 bootfb_end=0xd1fa3fff" | sudo tee /etc/modprobe.d/force-remove-bootfb.conf
echo "force-remove-bootfb" | sudo tee -a /etc/modules # <= not sure if this file is same in your distro

TODO: dkms.conf TODO: make install support TODO: Figure out if it is possible to find the address range inside the module code automatically.

furkanmustafa commented 2 years ago

Created issues, #2, #3, #4 and #5.

furkanmustafa commented 2 years ago

If you can manually proceed and have results, please share updates and close this issue.

kukat commented 2 years ago

Thank you @furkanmustafa for your reply

I got SSL errors when make install

root@pve:~/forcefully-remove-bootfb# make
mkdir -p "/root/forcefully-remove-bootfb/build"
touch "/root/forcefully-remove-bootfb/build/Makefile"
make -C /lib/modules/5.15.30-2-pve/build M=/root/forcefully-remove-bootfb/build src=/root/forcefully-remove-bootfb modules
make[1]: Entering directory '/usr/src/linux-headers-5.15.30-2-pve'
  CC [M]  /root/forcefully-remove-bootfb/build/force-remove-bootfb.o
  MODPOST /root/forcefully-remove-bootfb/build/Module.symvers
  CC [M]  /root/forcefully-remove-bootfb/build/force-remove-bootfb.mod.o
  LD [M]  /root/forcefully-remove-bootfb/build/force-remove-bootfb.ko
make[1]: Leaving directory '/usr/src/linux-headers-5.15.30-2-pve'
root@pve:~/forcefully-remove-bootfb# make install
make -C /lib/modules/5.15.30-2-pve/build M=/root/forcefully-remove-bootfb/build modules_install
make[1]: Entering directory '/usr/src/linux-headers-5.15.30-2-pve'
arch/x86/Makefile:142: CONFIG_X86_X32 enabled but no binutils support
  INSTALL /lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko
  SIGN    /lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko
At main.c:160:
- SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
sign-file: certs/signing_key.pem: No such file or directory
  DEPMOD  /lib/modules/5.15.30-2-pve
Warning: modules_install: missing 'System.map' file. Skipping depmod.
make[1]: Leaving directory '/usr/src/linux-headers-5.15.30-2-pve'

the address range seems to be consistent, always shows d0000000-d1fa3fff

root@pve:~/forcefully-remove-bootfb# grep BOOTFB /proc/iomem | awk '{ print $1 }'
d0000000-d1fa3fff

I have tried to activate force-remove-bootfb

root@pve:~/forcefully-remove-bootfb# cat /etc/modprobe.d/force-remove-bootfb.conf
options force-remove-bootfb bootfb_start=0xd0000000 bootfb_end=0xd1fa3fff
root@pve:~/forcefully-remove-bootfb# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
force-remove-bootfb

but can't see it in lsmod after reboot, and BOOTFB is still there in /proc/iomem

kukat commented 2 years ago

dmesg for your reference:

[    0.000000] Linux version 5.15.30-2-pve (build@proxmox) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP PVE 5.15.30-3 (Fri, 22 Apr 2022 18:08:27 +0200) ()
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.30-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on video=efifb:off video=vesafb:off video=simplefb:off
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Hygon HygonGenuine
[    0.000000]   Centaur CentaurHauls
[    0.000000]   zhaoxin   Shanghai  
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: Enabled xstate features 0x1f, context size is 960 bytes, using 'compacted' format.
[    0.000000] signal: max sigframe size: 2032
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009efff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009f000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000aea27fff] usable
[    0.000000] BIOS-e820: [mem 0x00000000aea28000-0x00000000aea28fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000aea29000-0x00000000aea5afff] usable
[    0.000000] BIOS-e820: [mem 0x00000000aea5b000-0x00000000aea5bfff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000aea5c000-0x00000000c279afff] usable
[    0.000000] BIOS-e820: [mem 0x00000000c279b000-0x00000000c2c9afff] type 20
[    0.000000] BIOS-e820: [mem 0x00000000c2c9b000-0x00000000c3c7efff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000c3c7f000-0x00000000c3e7efff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x00000000c3e7f000-0x00000000c3efefff] ACPI data
[    0.000000] BIOS-e820: [mem 0x00000000c3eff000-0x00000000c3efffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000c3f00000-0x00000000cc7fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000042f7fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.70 by HP
[    0.000000] efi: ACPI=0xc3efe000 ACPI 2.0=0xc3efe014 TPMFinalLog=0xc3e0e000 SMBIOS=0xc2f66000 ESRT=0xc2f64218 MEMATTR=0xbeb3f298 
[    0.000000] secureboot: Secure boot could not be determined (mode 0)
[    0.000000] SMBIOS 3.2 present.
[    0.000000] DMI: HP HP ProDesk 600 G6 Desktop Mini PC/8715, BIOS S22 Ver. 02.10.10 03/16/2022
[    0.000000] tsc: Detected 2300.000 MHz processor
[    0.000000] tsc: Detected 2299.968 MHz TSC
[    0.000009] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000013] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000021] last_pfn = 0x42f800 max_arch_pfn = 0x400000000
[    0.000245] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.001400] last_pfn = 0xc3f00 max_arch_pfn = 0x400000000
[    0.011137] esrt: Reserving ESRT space from 0x00000000c2f64218 to 0x00000000c2f64250.
[    0.011158] Using GB pages for direct mapping
[    0.011512] secureboot: Secure boot could not be determined (mode 0)
[    0.011514] RAMDISK: [mem 0x30773000-0x343b0fff]
[    0.011517] ACPI: Early table checksum verification disabled
[    0.011520] ACPI: RSDP 0x00000000C3EFE014 000024 (v02 HPQOEM)
[    0.011525] ACPI: XSDT 0x00000000C3EC2188 0000FC (v01 HPQOEM SLIC-BPC 00000000 INTL 20160422)
[    0.011532] ACPI: FACP 0x00000000C3EF4000 000114 (v06 HPQOEM SLIC-BPC 00000000 HP   00000001)
[    0.011539] ACPI: DSDT 0x00000000C3ED1000 01FF58 (v02 HPQOEM 8715     00000000 INTL 20160527)
[    0.011544] ACPI: FACS 0x00000000C3E09000 000040
[    0.011547] ACPI: SSDT 0x00000000C3EFC000 00012A (v02 HP     ShmTable 00000001 INTL 20160527)
[    0.011551] ACPI: SSDT 0x00000000C3EF9000 0020AD (v02 CpuRef CpuSsdt  00003000 INTL 20160527)
[    0.011555] ACPI: SSDT 0x00000000C3EF8000 000DF9 (v02 CtdpB  CtdpB    00001000 INTL 20160527)
[    0.011559] ACPI: MSDM 0x00000000C3EF7000 000055 (v03 HPQOEM SLIC-BPC 00000000 HP   00000001)
[    0.011563] ACPI: SLIC 0x00000000C3EF6000 000176 (v01 HPQOEM SLIC-BPC 00000001 HP   00000001)
[    0.011567] ACPI: WSMT 0x00000000C3EF5000 000028 (v01 HPQOEM 8715     00000001 HP   00000001)
[    0.011571] ACPI: HPET 0x00000000C3EF3000 000038 (v01 HPQOEM 8715     00000001 HP   00000001)
[    0.011575] ACPI: APIC 0x00000000C3EF2000 000164 (v04 HPQOEM 8715     00000001 HP   00000001)
[    0.011578] ACPI: MCFG 0x00000000C3EF1000 00003C (v01 HPQOEM 8715     00000001 HP   00000001)
[    0.011582] ACPI: SSDT 0x00000000C3ED0000 000734 (v01 HPQOEM PtidDevc 00001000 INTL 20160527)
[    0.011586] ACPI: SSDT 0x00000000C3ECF000 000C40 (v02 HPQOEM HP_XHC_P 00000000 INTL 20160527)
[    0.011590] ACPI: SSDT 0x00000000C3ECB000 003139 (v02 SaSsdt SaSsdt   00003000 INTL 20160527)
[    0.011594] ACPI: LPIT 0x00000000C3ECA000 000094 (v01 INTEL  CML      20170001 INTL 20160422)
[    0.011598] ACPI: WSMT 0x00000000C3EC9000 000028 (v01 INTEL  CML      20170001 INTL 20160422)
[    0.011602] ACPI: SSDT 0x00000000C3EC8000 0005FB (v02 INTEL  Tpm2Tabl 00001000 INTL 20160527)
[    0.011607] ACPI: TPM2 0x00000000C3EC7000 000034 (v04 INTEL  CML      20170001 INTL 20160422)
[    0.011611] ACPI: SSDT 0x00000000C3EC6000 0003DF (v02 Intel  UcsiAcpi 00001000 INTL 20160527)
[    0.011615] ACPI: SSDT 0x00000000C3EC5000 00006E (v02 Intel  UcsiCntr 00001000 INTL 20160527)
[    0.011618] ACPI: OEML 0x00000000C3EC4000 000028 (v03 INTEL  CML      20170001 INTL 20160422)
[    0.011622] ACPI: SSDT 0x00000000C3EAF000 012698 (v01 HP     HPINDTWL 00001000 INTL 20160527)
[    0.011626] ACPI: SSDT 0x00000000C3EC3000 000032 (v01 HP     HPCONDEV 00001000 INTL 20160527)
[    0.011630] ACPI: SSDT 0x00000000C3EAE000 000069 (v01 HP     HPCAHWID 00001000 INTL 20160527)
[    0.011634] ACPI: DMAR 0x00000000C3EFD000 0000C8 (v01 INTEL  CML      00000002      01000013)
[    0.011638] ACPI: ASF! 0x00000000C3EAD000 0000A0 (v32 HPQOEM  UYAMIHC 20170001 INTL 20160422)
[    0.011642] ACPI: BGRT 0x00000000C3EAC000 000038 (v01 INTEL  CML      20170001 INTL 20160422)
[    0.011646] ACPI: FPDT 0x00000000C3EAB000 000044 (v01 INTEL  CML      20170001 INTL 20160422)
[    0.011649] ACPI: Reserving FACP table memory at [mem 0xc3ef4000-0xc3ef4113]
[    0.011651] ACPI: Reserving DSDT table memory at [mem 0xc3ed1000-0xc3ef0f57]
[    0.011653] ACPI: Reserving FACS table memory at [mem 0xc3e09000-0xc3e0903f]
[    0.011654] ACPI: Reserving SSDT table memory at [mem 0xc3efc000-0xc3efc129]
[    0.011656] ACPI: Reserving SSDT table memory at [mem 0xc3ef9000-0xc3efb0ac]
[    0.011657] ACPI: Reserving SSDT table memory at [mem 0xc3ef8000-0xc3ef8df8]
[    0.011659] ACPI: Reserving MSDM table memory at [mem 0xc3ef7000-0xc3ef7054]
[    0.011660] ACPI: Reserving SLIC table memory at [mem 0xc3ef6000-0xc3ef6175]
[    0.011662] ACPI: Reserving WSMT table memory at [mem 0xc3ef5000-0xc3ef5027]
[    0.011663] ACPI: Reserving HPET table memory at [mem 0xc3ef3000-0xc3ef3037]
[    0.011664] ACPI: Reserving APIC table memory at [mem 0xc3ef2000-0xc3ef2163]
[    0.011666] ACPI: Reserving MCFG table memory at [mem 0xc3ef1000-0xc3ef103b]
[    0.011667] ACPI: Reserving SSDT table memory at [mem 0xc3ed0000-0xc3ed0733]
[    0.011669] ACPI: Reserving SSDT table memory at [mem 0xc3ecf000-0xc3ecfc3f]
[    0.011670] ACPI: Reserving SSDT table memory at [mem 0xc3ecb000-0xc3ece138]
[    0.011672] ACPI: Reserving LPIT table memory at [mem 0xc3eca000-0xc3eca093]
[    0.011673] ACPI: Reserving WSMT table memory at [mem 0xc3ec9000-0xc3ec9027]
[    0.011675] ACPI: Reserving SSDT table memory at [mem 0xc3ec8000-0xc3ec85fa]
[    0.011676] ACPI: Reserving TPM2 table memory at [mem 0xc3ec7000-0xc3ec7033]
[    0.011678] ACPI: Reserving SSDT table memory at [mem 0xc3ec6000-0xc3ec63de]
[    0.011679] ACPI: Reserving SSDT table memory at [mem 0xc3ec5000-0xc3ec506d]
[    0.011681] ACPI: Reserving OEML table memory at [mem 0xc3ec4000-0xc3ec4027]
[    0.011682] ACPI: Reserving SSDT table memory at [mem 0xc3eaf000-0xc3ec1697]
[    0.011684] ACPI: Reserving SSDT table memory at [mem 0xc3ec3000-0xc3ec3031]
[    0.011685] ACPI: Reserving SSDT table memory at [mem 0xc3eae000-0xc3eae068]
[    0.011687] ACPI: Reserving DMAR table memory at [mem 0xc3efd000-0xc3efd0c7]
[    0.011688] ACPI: Reserving ASF! table memory at [mem 0xc3ead000-0xc3ead09f]
[    0.011690] ACPI: Reserving BGRT table memory at [mem 0xc3eac000-0xc3eac037]
[    0.011691] ACPI: Reserving FPDT table memory at [mem 0xc3eab000-0xc3eab043]
[    0.012275] No NUMA configuration found
[    0.012276] Faking a node at [mem 0x0000000000000000-0x000000042f7fffff]
[    0.012290] NODE_DATA(0) allocated [mem 0x42f7d4000-0x42f7fdfff]
[    0.012729] Zone ranges:
[    0.012730]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.012732]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.012734]   Normal   [mem 0x0000000100000000-0x000000042f7fffff]
[    0.012736]   Device   empty
[    0.012738] Movable zone start for each node
[    0.012741] Early memory node ranges
[    0.012742]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.012744]   node   0: [mem 0x0000000000100000-0x00000000aea27fff]
[    0.012745]   node   0: [mem 0x00000000aea29000-0x00000000aea5afff]
[    0.012747]   node   0: [mem 0x00000000aea5c000-0x00000000c279afff]
[    0.012748]   node   0: [mem 0x00000000c3eff000-0x00000000c3efffff]
[    0.012749]   node   0: [mem 0x0000000100000000-0x000000042f7fffff]
[    0.012752] Initmem setup node 0 [mem 0x0000000000001000-0x000000042f7fffff]
[    0.012757] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.012794] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.017940] On node 0, zone DMA32: 1 pages in unavailable ranges
[    0.018530] On node 0, zone DMA32: 1 pages in unavailable ranges
[    0.018615] On node 0, zone DMA32: 5988 pages in unavailable ranges
[    0.042938] On node 0, zone Normal: 16640 pages in unavailable ranges
[    0.042966] On node 0, zone Normal: 2048 pages in unavailable ranges
[    0.042987] Reserving Intel graphics memory at [mem 0xc8800000-0xcc7fffff]
[    0.043360] ACPI: PM-Timer IO Port: 0x1808
[    0.043369] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.043371] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
[    0.043372] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
[    0.043373] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
[    0.043374] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
[    0.043375] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
[    0.043377] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
[    0.043378] ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1])
[    0.043379] ACPI: LAPIC_NMI (acpi_id[0x09] high edge lint[0x1])
[    0.043380] ACPI: LAPIC_NMI (acpi_id[0x0a] high edge lint[0x1])
[    0.043381] ACPI: LAPIC_NMI (acpi_id[0x0b] high edge lint[0x1])
[    0.043382] ACPI: LAPIC_NMI (acpi_id[0x0c] high edge lint[0x1])
[    0.043383] ACPI: LAPIC_NMI (acpi_id[0x0d] high edge lint[0x1])
[    0.043384] ACPI: LAPIC_NMI (acpi_id[0x0e] high edge lint[0x1])
[    0.043385] ACPI: LAPIC_NMI (acpi_id[0x0f] high edge lint[0x1])
[    0.043386] ACPI: LAPIC_NMI (acpi_id[0x10] high edge lint[0x1])
[    0.043388] ACPI: LAPIC_NMI (acpi_id[0x11] high edge lint[0x1])
[    0.043389] ACPI: LAPIC_NMI (acpi_id[0x12] high edge lint[0x1])
[    0.043390] ACPI: LAPIC_NMI (acpi_id[0x13] high edge lint[0x1])
[    0.043391] ACPI: LAPIC_NMI (acpi_id[0x14] high edge lint[0x1])
[    0.043454] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
[    0.043458] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.043460] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.043465] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.043466] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.043475] e820: update [mem 0xbdbe3000-0xbdc58fff] usable ==> reserved
[    0.043486] TSC deadline timer available
[    0.043487] smpboot: Allowing 12 CPUs, 0 hotplug CPUs
[    0.043512] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.043515] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]
[    0.043518] PM: hibernation: Registered nosave memory: [mem 0xaea28000-0xaea28fff]
[    0.043521] PM: hibernation: Registered nosave memory: [mem 0xaea5b000-0xaea5bfff]
[    0.043523] PM: hibernation: Registered nosave memory: [mem 0xbdbe3000-0xbdc58fff]
[    0.043526] PM: hibernation: Registered nosave memory: [mem 0xc279b000-0xc2c9afff]
[    0.043527] PM: hibernation: Registered nosave memory: [mem 0xc2c9b000-0xc3c7efff]
[    0.043528] PM: hibernation: Registered nosave memory: [mem 0xc3c7f000-0xc3e7efff]
[    0.043529] PM: hibernation: Registered nosave memory: [mem 0xc3e7f000-0xc3efefff]
[    0.043531] PM: hibernation: Registered nosave memory: [mem 0xc3f00000-0xcc7fffff]
[    0.043533] PM: hibernation: Registered nosave memory: [mem 0xcc800000-0xfeffffff]
[    0.043534] PM: hibernation: Registered nosave memory: [mem 0xff000000-0xffffffff]
[    0.043535] [mem 0xcc800000-0xfeffffff] available for PCI devices
[    0.043537] Booting paravirtualized kernel on bare hardware
[    0.043540] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.043549] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:12 nr_cpu_ids:12 nr_node_ids:1
[    0.043946] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u262144
[    0.043956] pcpu-alloc: s212992 r8192 d28672 u262144 alloc=1*2097152
[    0.043959] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 [0] 08 09 10 11 -- -- -- -- 
[    0.044008] Built 1 zonelists, mobility grouping on.  Total pages: 4071963
[    0.044010] Policy zone: Normal
[    0.044011] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.30-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on video=efifb:off video=vesafb:off video=simplefb:off
[    0.044083] DMAR: IOMMU enabled
[    0.044124] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.30-2-pve", will be passed to user space.
[    0.045172] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.045716] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[    0.045852] mem auto-init: stack:off, heap alloc:on, heap free:off
[    0.095401] Memory: 15976568K/16547040K available (16393K kernel code, 4345K rwdata, 10180K rodata, 2884K init, 4904K bss, 570212K reserved, 0K cma-reserved)
[    0.095408] random: get_random_u64 called from __kmem_cache_create+0x2f/0x530 with crng_init=0
[    0.095578] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=12, Nodes=1
[    0.095599] ftrace: allocating 48106 entries in 188 pages
[    0.117051] ftrace: allocated 188 pages with 5 groups
[    0.117187] rcu: Hierarchical RCU implementation.
[    0.117188] rcu:     RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=12.
[    0.117190]  Rude variant of Tasks RCU enabled.
[    0.117191]  Tracing variant of Tasks RCU enabled.
[    0.117192] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.117193] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=12
[    0.122356] NR_IRQS: 524544, nr_irqs: 2152, preallocated irqs: 16
[    0.122925] random: crng done (trusting CPU's manufacturer)
[    0.122959] Console: colour dummy device 80x25
[    0.122979] printk: console [tty0] enabled
[    0.123002] ACPI: Core revision 20210730
[    0.123343] hpet: HPET dysfunctional in PC10. Force disabled.
[    0.123345] APIC: Switch to symmetric I/O mode setup
[    0.123347] DMAR: Host address width 39
[    0.123348] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.123355] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.123358] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.123362] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.123365] DMAR: RMRR base: 0x000000c3b0c000 end: 0x000000c3b2bfff
[    0.123368] DMAR: RMRR base: 0x000000c8000000 end: 0x000000cc7fffff
[    0.123369] DMAR: RMRR base: 0x000000c3b72000 end: 0x000000c3bf1fff
[    0.123372] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.123373] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.123375] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.126687] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.126689] x2apic enabled
[    0.126720] Switched APIC routing to cluster x2apic.
[    0.135818] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x212717146a7, max_idle_ns: 440795291431 ns
[    0.135826] Calibrating delay loop (skipped), value calculated using timer frequency.. 4599.93 BogoMIPS (lpj=9199872)
[    0.135830] pid_max: default: 32768 minimum: 301
[    0.139786] LSM: Security Framework initializing
[    0.139796] Yama: becoming mindful.
[    0.139820] AppArmor: AppArmor initialized
[    0.139822] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.139822] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.139822] x86/cpu: SGX disabled by BIOS.
[    0.139822] CPU0: Thermal monitoring enabled (TM1)
[    0.139822] process: using mwait in idle threads
[    0.139822] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.139822] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.139822] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.139822] Spectre V2 : Mitigation: Enhanced IBRS
[    0.139822] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.139822] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.139822] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.139822] Freeing SMP alternatives memory: 40K
[    0.139822] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1647
[    0.139822] smpboot: CPU0: Intel(R) Core(TM) i5-10500T CPU @ 2.30GHz (family: 0x6, model: 0xa5, stepping: 0x0)
[    0.139822] Performance Events: PEBS fmt3+, Skylake events, 32-deep LBR, full-width counters, Intel PMU driver.
[    0.139822] ... version:                4
[    0.139822] ... bit width:              48
[    0.139822] ... generic registers:      4
[    0.139822] ... value mask:             0000ffffffffffff
[    0.139822] ... max period:             00007fffffffffff
[    0.139822] ... fixed-purpose events:   3
[    0.139822] ... event mask:             000000070000000f
[    0.139822] rcu: Hierarchical SRCU implementation.
[    0.139822] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[    0.139822] smp: Bringing up secondary CPUs ...
[    0.139822] x86: Booting SMP configuration:
[    0.139822] .... node  #0, CPUs:        #1  #2  #3  #4  #5  #6  #7  #8  #9 #10 #11
[    0.144562] smp: Brought up 1 node, 12 CPUs
[    0.144562] smpboot: Max logical packages: 1
[    0.144562] smpboot: Total of 12 processors activated (55199.23 BogoMIPS)
[    0.148480] devtmpfs: initialized
[    0.148480] x86/mm: Memory block size: 128MB
[    0.149065] ACPI: PM: Registering ACPI NVS region [mem 0xaea28000-0xaea28fff] (4096 bytes)
[    0.149065] ACPI: PM: Registering ACPI NVS region [mem 0xc3c7f000-0xc3e7efff] (2097152 bytes)
[    0.149065] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.149065] futex hash table entries: 4096 (order: 6, 262144 bytes, linear)
[    0.149065] pinctrl core: initialized pinctrl subsystem
[    0.149065] PM: RTC time: 04:18:04, date: 2022-05-06
[    0.149065] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.149065] DMA: preallocated 2048 KiB GFP_KERNEL pool for atomic allocations
[    0.149065] DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.149147] DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.149154] audit: initializing netlink subsys (disabled)
[    0.149163] audit: type=2000 audit(1651810684.012:1): state=initialized audit_enabled=0 res=1
[    0.149163] thermal_sys: Registered thermal governor 'fair_share'
[    0.149163] thermal_sys: Registered thermal governor 'bang_bang'
[    0.149163] thermal_sys: Registered thermal governor 'step_wise'
[    0.149163] thermal_sys: Registered thermal governor 'user_space'
[    0.149163] thermal_sys: Registered thermal governor 'power_allocator'
[    0.149163] EISA bus registered
[    0.149163] cpuidle: using governor ladder
[    0.149163] cpuidle: using governor menu
[    0.149163] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    0.149163] ACPI: bus type PCI registered
[    0.149163] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.149163] PCI: MMCONFIG for domain 0000 [bus 00-7f] at [mem 0xf0000000-0xf7ffffff] (base 0xf0000000)
[    0.149163] PCI: not using MMCONFIG
[    0.149163] PCI: Using configuration type 1 for base access
[    0.152103] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.153199] Kprobes globally optimized
[    0.153204] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.153204] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.153204] fbcon: Taking over console
[    0.153204] ACPI: Added _OSI(Module Device)
[    0.153204] ACPI: Added _OSI(Processor Device)
[    0.153204] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.153204] ACPI: Added _OSI(Processor Aggregator Device)
[    0.153204] ACPI: Added _OSI(Linux-Dell-Video)
[    0.153204] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.153204] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.217732] ACPI: 13 ACPI AML tables successfully acquired and loaded
[    0.220239] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[    0.225971] ACPI: Dynamic OEM Table Load:
[    0.225983] ACPI: SSDT 0xFFFF9EF1801F3000 0000FC (v02 PmRef  Cpu0Psd  00003000 INTL 20160527)
[    0.227244] ACPI: \_SB_.PR00: _OSC native thermal LVT Acked
[    0.228525] ACPI: Dynamic OEM Table Load:
[    0.228535] ACPI: SSDT 0xFFFF9EF181A02400 000400 (v02 PmRef  Cpu0Cst  00003001 INTL 20160527)
[    0.229935] ACPI: Dynamic OEM Table Load:
[    0.229944] ACPI: SSDT 0xFFFF9EF181A11000 00051E (v02 PmRef  Cpu0Ist  00003000 INTL 20160527)
[    0.231459] ACPI: Dynamic OEM Table Load:
[    0.231467] ACPI: SSDT 0xFFFF9EF1802A8C00 00016C (v02 PmRef  Cpu0Hwp  00003000 INTL 20160527)
[    0.232796] ACPI: Dynamic OEM Table Load:
[    0.232807] ACPI: SSDT 0xFFFF9EF180135000 000BEA (v02 PmRef  HwpLvt   00003000 INTL 20160527)
[    0.234487] ACPI: Dynamic OEM Table Load:
[    0.234496] ACPI: SSDT 0xFFFF9EF181A14800 000778 (v02 PmRef  ApIst    00003000 INTL 20160527)
[    0.236110] ACPI: Dynamic OEM Table Load:
[    0.236119] ACPI: SSDT 0xFFFF9EF181A00800 0003D7 (v02 PmRef  ApHwp    00003000 INTL 20160527)
[    0.237699] ACPI: Dynamic OEM Table Load:
[    0.237710] ACPI: SSDT 0xFFFF9EF181A27000 000D22 (v02 PmRef  ApPsd    00003000 INTL 20160527)
[    0.240474] ACPI: Dynamic OEM Table Load:
[    0.240483] ACPI: SSDT 0xFFFF9EF181A03400 0003CA (v02 PmRef  ApCst    00003000 INTL 20160527)
[    0.248105] ACPI: EC: EC started
[    0.248106] ACPI: EC: interrupt blocked
[    0.248136] ACPI: EC: EC_CMD/EC_SC=0x66, EC_DATA=0x62
[    0.248139] ACPI: \_SB_.PCI0.LPCB.EC0_: Boot DSDT EC used to handle transactions
[    0.248141] ACPI: Interpreter enabled
[    0.248207] ACPI: PM: (supports S0 S3 S4 S5)
[    0.248210] ACPI: Using IOAPIC for interrupt routing
[    0.248267] PCI: MMCONFIG for domain 0000 [bus 00-7f] at [mem 0xf0000000-0xf7ffffff] (base 0xf0000000)
[    0.250143] PCI: MMCONFIG at [mem 0xf0000000-0xf7ffffff] reserved in ACPI motherboard resources
[    0.250157] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.250673] ACPI: Enabled 6 GPEs in block 00 to 7F
[    0.259984] ACPI: PM: Power Resource [USBC]
[    0.272479] ACPI: PM: Power Resource [V0PR]
[    0.272781] ACPI: PM: Power Resource [V1PR]
[    0.273080] ACPI: PM: Power Resource [V2PR]
[    0.275040] ACPI: PM: Power Resource [WRST]
[    0.277357] ACPI: PM: Power Resource [PIN]
[    0.278697] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7e])
[    0.278706] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    0.282559] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug SHPCHotplug PME AER PCIeCapability LTR]
[    0.282561] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration
[    0.286964] PCI host bridge to bus 0000:00
[    0.286966] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.286969] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.286971] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.286973] pci_bus 0000:00: root bus resource [mem 0xcc800000-0xefffffff window]
[    0.286976] pci_bus 0000:00: root bus resource [mem 0x4000000000-0x7fffffffff window]
[    0.286978] pci_bus 0000:00: root bus resource [mem 0xfc800000-0xfdffffff window]
[    0.286980] pci_bus 0000:00: root bus resource [bus 00-7e]
[    0.288485] pci 0000:00:00.0: [8086:9b53] type 00 class 0x060000
[    0.292986] pci 0000:00:02.0: [8086:9bc8] type 00 class 0x030000
[    0.292996] pci 0000:00:02.0: reg 0x10: [mem 0xe0000000-0xe0ffffff 64bit]
[    0.293003] pci 0000:00:02.0: reg 0x18: [mem 0xd0000000-0xdfffffff 64bit pref]
[    0.293008] pci 0000:00:02.0: reg 0x20: [io  0x3000-0x303f]
[    0.293026] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.293029] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.294869] pci 0000:00:12.0: [8086:06f9] type 00 class 0x118000
[    0.294895] pci 0000:00:12.0: reg 0x10: [mem 0x4000107000-0x4000107fff 64bit]
[    0.299534] pci 0000:00:14.0: [8086:06ed] type 00 class 0x0c0330
[    0.299562] pci 0000:00:14.0: reg 0x10: [mem 0xe1220000-0xe122ffff 64bit]
[    0.299673] pci 0000:00:14.0: PME# supported from D3hot D3cold
[    0.301718] pci 0000:00:14.2: [8086:06ef] type 00 class 0x050000
[    0.301742] pci 0000:00:14.2: reg 0x10: [mem 0xe1236000-0xe1237fff 64bit]
[    0.301757] pci 0000:00:14.2: reg 0x18: [mem 0x4000106000-0x4000106fff 64bit]
[    0.306356] pci 0000:00:14.3: [8086:06f0] type 00 class 0x028000
[    0.306394] pci 0000:00:14.3: reg 0x10: [mem 0xe1230000-0xe1233fff 64bit]
[    0.306543] pci 0000:00:14.3: PME# supported from D0 D3hot D3cold
[    0.309835] pci 0000:00:16.0: [8086:06e0] type 00 class 0x078000
[    0.309860] pci 0000:00:16.0: reg 0x10: [mem 0x4000105000-0x4000105fff 64bit]
[    0.309962] pci 0000:00:16.0: PME# supported from D3hot
[    0.311947] pci 0000:00:16.3: [8086:06e3] type 00 class 0x070002
[    0.311966] pci 0000:00:16.3: reg 0x10: [io  0x3088-0x308f]
[    0.311975] pci 0000:00:16.3: reg 0x14: [mem 0xe123b000-0xe123bfff]
[    0.316601] pci 0000:00:17.0: [8086:06d2] type 00 class 0x010601
[    0.316624] pci 0000:00:17.0: reg 0x10: [mem 0xe1234000-0xe1235fff]
[    0.316636] pci 0000:00:17.0: reg 0x14: [mem 0xe123a000-0xe123a0ff]
[    0.316648] pci 0000:00:17.0: reg 0x18: [io  0x3080-0x3087]
[    0.316660] pci 0000:00:17.0: reg 0x1c: [io  0x3090-0x3093]
[    0.316672] pci 0000:00:17.0: reg 0x20: [io  0x3060-0x307f]
[    0.316684] pci 0000:00:17.0: reg 0x24: [mem 0xe1239000-0xe12397ff]
[    0.316822] pci 0000:00:17.0: PME# supported from D3hot
[    0.320532] pci 0000:00:1b.0: [8086:06c0] type 01 class 0x060400
[    0.320710] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.320779] pci 0000:00:1b.0: PTM enabled (root), 4ns granularity
[    0.324666] pci 0000:00:1b.4: [8086:06ac] type 01 class 0x060400
[    0.324846] pci 0000:00:1b.4: PME# supported from D0 D3hot D3cold
[    0.324913] pci 0000:00:1b.4: PTM enabled (root), 4ns granularity
[    0.328807] pci 0000:00:1f.0: [8086:0687] type 00 class 0x060100
[    0.332220] pci 0000:00:1f.3: [8086:06c8] type 00 class 0x040300
[    0.332278] pci 0000:00:1f.3: reg 0x10: [mem 0x4000100000-0x4000103fff 64bit]
[    0.332350] pci 0000:00:1f.3: reg 0x20: [mem 0x4000000000-0x40000fffff 64bit]
[    0.332493] pci 0000:00:1f.3: PME# supported from D3hot D3cold
[    0.336451] pci 0000:00:1f.4: [8086:06a3] type 00 class 0x0c0500
[    0.336533] pci 0000:00:1f.4: reg 0x10: [mem 0x4000104000-0x40001040ff 64bit]
[    0.336616] pci 0000:00:1f.4: reg 0x20: [io  0xefa0-0xefbf]
[    0.338495] pci 0000:00:1f.5: [8086:06a4] type 00 class 0x0c8000
[    0.338516] pci 0000:00:1f.5: reg 0x10: [mem 0xfe010000-0xfe010fff]
[    0.343091] pci 0000:00:1f.6: [8086:0d4c] type 00 class 0x020000
[    0.343137] pci 0000:00:1f.6: reg 0x10: [mem 0xe1200000-0xe121ffff]
[    0.343373] pci 0000:00:1f.6: PME# supported from D0 D3hot D3cold
[    0.343591] pci 0000:01:00.0: [15b7:5009] type 00 class 0x010802
[    0.343619] pci 0000:01:00.0: reg 0x10: [mem 0xe1100000-0xe1103fff 64bit]
[    0.343658] pci 0000:01:00.0: reg 0x20: [mem 0xe1104000-0xe11040ff 64bit]
[    0.344109] pci 0000:00:1b.0: PCI bridge to [bus 01]
[    0.344118] pci 0000:00:1b.0:   bridge window [mem 0xe1100000-0xe11fffff]
[    0.344231] pci 0000:02:00.0: [15b7:5006] type 00 class 0x010802
[    0.344259] pci 0000:02:00.0: reg 0x10: [mem 0xe1000000-0xe1003fff 64bit]
[    0.344297] pci 0000:02:00.0: reg 0x20: [mem 0xe1004000-0xe10040ff 64bit]
[    0.344613] pci 0000:00:1b.4: PCI bridge to [bus 02]
[    0.344621] pci 0000:00:1b.4:   bridge window [mem 0xe1000000-0xe10fffff]
[    0.346510] ACPI: EC: interrupt unblocked
[    0.346511] ACPI: EC: event unblocked
[    0.346524] ACPI: EC: EC_CMD/EC_SC=0x66, EC_DATA=0x62
[    0.346525] ACPI: EC: GPE=0x6e
[    0.346527] ACPI: \_SB_.PCI0.LPCB.EC0_: Boot DSDT EC initialization complete
[    0.346529] ACPI: \_SB_.PCI0.LPCB.EC0_: EC: Used to handle transactions and events
[    0.346594] iommu: Default domain type: Translated 
[    0.346594] iommu: DMA domain TLB invalidation policy: lazy mode 
[    0.346594] SCSI subsystem initialized
[    0.346594] libata version 3.00 loaded.
[    0.346594] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.346594] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.346594] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.346594] vgaarb: loaded
[    0.346594] ACPI: bus type USB registered
[    0.346594] usbcore: registered new interface driver usbfs
[    0.346594] usbcore: registered new interface driver hub
[    0.346594] usbcore: registered new device driver usb
[    0.346594] pps_core: LinuxPPS API ver. 1 registered
[    0.346594] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.346594] PTP clock support registered
[    0.346594] EDAC MC: Ver: 3.0.0
[    0.346594] Registered efivars operations
[    0.346594] NetLabel: Initializing
[    0.346594] NetLabel:  domain hash size = 128
[    0.346594] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.346594] NetLabel:  unlabeled traffic allowed by default
[    0.346594] PCI: Using ACPI for IRQ routing
[    0.389355] PCI: pci_cache_line_size set to 64 bytes
[    0.389515] pci 0000:00:1f.5: can't claim BAR 0 [mem 0xfe010000-0xfe010fff]: no compatible bridge window
[    0.389572] e820: reserve RAM buffer [mem 0x0009f000-0x0009ffff]
[    0.389574] e820: reserve RAM buffer [mem 0xaea28000-0xafffffff]
[    0.389576] e820: reserve RAM buffer [mem 0xaea5b000-0xafffffff]
[    0.389577] e820: reserve RAM buffer [mem 0xbdbe3000-0xbfffffff]
[    0.389578] e820: reserve RAM buffer [mem 0xc279b000-0xc3ffffff]
[    0.389580] e820: reserve RAM buffer [mem 0xc3f00000-0xc3ffffff]
[    0.389581] e820: reserve RAM buffer [mem 0x42f800000-0x42fffffff]
[    0.389586] clocksource: Switched to clocksource tsc-early
[    0.397516] VFS: Disk quotas dquot_6.6.0
[    0.397532] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.397643] AppArmor: AppArmor Filesystem Enabled
[    0.397685] pnp: PnP ACPI init
[    0.398104] system 00:00: [io  0x1800-0x18fe] has been reserved
[    0.398109] system 00:00: [mem 0xfd000000-0xfd69ffff] has been reserved
[    0.398111] system 00:00: [mem 0xfd6c0000-0xfd6cffff] has been reserved
[    0.398113] system 00:00: [mem 0xfd6f0000-0xfdffffff] has been reserved
[    0.398114] system 00:00: [mem 0xfe000000-0xfe01ffff] has been reserved
[    0.398116] system 00:00: [mem 0xfe200000-0xfe7fffff] has been reserved
[    0.398118] system 00:00: [mem 0xff000000-0xffffffff] has been reserved
[    0.398590] system 00:01: [io  0x2000-0x20fe] has been reserved
[    0.398790] system 00:02: [io  0x0680-0x069f] has been reserved
[    0.398792] system 00:02: [io  0x164e-0x164f] has been reserved
[    0.398995] system 00:04: [io  0x1854-0x1857] has been reserved
[    0.399260] system 00:05: [io  0x0200-0x023f] has been reserved
[    0.399262] system 00:05: [mem 0xfedb0000-0xfedbffff] has been reserved
[    0.399728] system 00:06: [mem 0xfd6e0000-0xfd6effff] has been reserved
[    0.399731] system 00:06: [mem 0xfd6d0000-0xfd6dffff] has been reserved
[    0.399732] system 00:06: [mem 0xfd6b0000-0xfd6bffff] has been reserved
[    0.399734] system 00:06: [mem 0xfd6a0000-0xfd6affff] has been reserved
[    0.400030] system 00:07: [mem 0xfed10000-0xfed17fff] has been reserved
[    0.400033] system 00:07: [mem 0xfed18000-0xfed18fff] has been reserved
[    0.400035] system 00:07: [mem 0xfed19000-0xfed19fff] has been reserved
[    0.400036] system 00:07: [mem 0xf0000000-0xf7ffffff] has been reserved
[    0.400038] system 00:07: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.400040] system 00:07: [mem 0xfed90000-0xfed93fff] could not be reserved
[    0.400041] system 00:07: [mem 0xfed45000-0xfed8ffff] has been reserved
[    0.400043] system 00:07: [mem 0xfee00000-0xfeefffff] has been reserved
[    0.400391] system 00:08: [mem 0xfe038000-0xfe038fff] has been reserved
[    0.400605] pnp: PnP ACPI: found 9 devices
[    0.406463] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.406530] NET: Registered PF_INET protocol family
[    0.406701] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.408936] tcp_listen_portaddr_hash hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    0.409045] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.409248] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes, linear)
[    0.409333] TCP: Hash tables configured (established 131072 bind 65536)
[    0.409423] MPTCP token hash table entries: 16384 (order: 6, 393216 bytes, linear)
[    0.409487] UDP hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.409535] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes, linear)
[    0.409603] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.409609] NET: Registered PF_XDP protocol family
[    0.409619] pci 0000:00:1f.5: BAR 0: assigned [mem 0xcc800000-0xcc800fff]
[    0.409639] pci 0000:00:1b.0: PCI bridge to [bus 01]
[    0.409645] pci 0000:00:1b.0:   bridge window [mem 0xe1100000-0xe11fffff]
[    0.409656] pci 0000:00:1b.4: PCI bridge to [bus 02]
[    0.409662] pci 0000:00:1b.4:   bridge window [mem 0xe1000000-0xe10fffff]
[    0.409673] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.409675] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.409677] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.409678] pci_bus 0000:00: resource 7 [mem 0xcc800000-0xefffffff window]
[    0.409680] pci_bus 0000:00: resource 8 [mem 0x4000000000-0x7fffffffff window]
[    0.409681] pci_bus 0000:00: resource 9 [mem 0xfc800000-0xfdffffff window]
[    0.409683] pci_bus 0000:01: resource 1 [mem 0xe1100000-0xe11fffff]
[    0.409685] pci_bus 0000:02: resource 1 [mem 0xe1000000-0xe10fffff]
[    0.410362] PCI: CLS 0 bytes, default 64
[    0.410394] DMAR: No ATSR found
[    0.410395] DMAR: No SATC found
[    0.410396] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.410398] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.410399] DMAR: IOMMU feature nwfs inconsistent
[    0.410400] DMAR: IOMMU feature pasid inconsistent
[    0.410400] DMAR: IOMMU feature eafs inconsistent
[    0.410401] DMAR: IOMMU feature prs inconsistent
[    0.410402] DMAR: IOMMU feature nest inconsistent
[    0.410402] DMAR: IOMMU feature mts inconsistent
[    0.410403] DMAR: IOMMU feature sc_support inconsistent
[    0.410404] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.410405] DMAR: dmar0: Using Queued invalidation
[    0.410408] DMAR: dmar1: Using Queued invalidation
[    0.410423] Trying to unpack rootfs image as initramfs...
[    0.410641] pci 0000:00:00.0: Adding to iommu group 0
[    0.410651] pci 0000:00:02.0: Adding to iommu group 1
[    0.410662] pci 0000:00:12.0: Adding to iommu group 2
[    0.410676] pci 0000:00:14.0: Adding to iommu group 3
[    0.410684] pci 0000:00:14.2: Adding to iommu group 3
[    0.410692] pci 0000:00:14.3: Adding to iommu group 4
[    0.410705] pci 0000:00:16.0: Adding to iommu group 5
[    0.410713] pci 0000:00:16.3: Adding to iommu group 5
[    0.410720] pci 0000:00:17.0: Adding to iommu group 6
[    0.410742] pci 0000:00:1b.0: Adding to iommu group 7
[    0.410761] pci 0000:00:1b.4: Adding to iommu group 8
[    0.410782] pci 0000:00:1f.0: Adding to iommu group 9
[    0.410790] pci 0000:00:1f.3: Adding to iommu group 9
[    0.410798] pci 0000:00:1f.4: Adding to iommu group 9
[    0.410807] pci 0000:00:1f.5: Adding to iommu group 9
[    0.410815] pci 0000:00:1f.6: Adding to iommu group 9
[    0.410833] pci 0000:01:00.0: Adding to iommu group 10
[    0.410859] pci 0000:02:00.0: Adding to iommu group 11
[    0.412461] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    0.412463] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.412464] software IO TLB: mapped [mem 0x00000000b92e6000-0x00000000bd2e6000] (64MB)
[    0.424594] Initialise system trusted keyrings
[    0.424603] Key type blacklist registered
[    0.424636] workingset: timestamp_bits=36 max_order=22 bucket_order=0
[    0.425929] zbud: loaded
[    0.426180] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.426309] fuse: init (API version 7.34)
[    0.426465] integrity: Platform Keyring initialized
[    0.436764] Key type asymmetric registered
[    0.436765] Asymmetric key parser 'x509' registered
[    0.436780] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[    0.436817] io scheduler mq-deadline registered
[    0.437486] pcieport 0000:00:1b.0: PME: Signaling with IRQ 122
[    0.437571] pcieport 0000:00:1b.0: AER: enabled with IRQ 122
[    0.437624] pcieport 0000:00:1b.0: DPC: enabled with IRQ 122
[    0.437626] pcieport 0000:00:1b.0: DPC: error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
[    0.438028] pcieport 0000:00:1b.4: PME: Signaling with IRQ 123
[    0.438111] pcieport 0000:00:1b.4: AER: enabled with IRQ 123
[    0.438160] pcieport 0000:00:1b.4: DPC: enabled with IRQ 123
[    0.438162] pcieport 0000:00:1b.4: DPC: error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
[    0.438319] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    0.438734] Monitor-Mwait will be used to enter C-1 state
[    0.438739] Monitor-Mwait will be used to enter C-2 state
[    0.438742] Monitor-Mwait will be used to enter C-3 state
[    0.438746] ACPI: \_SB_.PR00: Found 3 idle states
[    0.439474] input: Sleep Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0E:00/input/input0
[    0.439502] ACPI: button: Sleep Button [SLPB]
[    0.439534] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input1
[    0.439552] ACPI: button: Power Button [PWRB]
[    0.439586] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input2
[    0.439621] ACPI: button: Power Button [PWRF]
[    0.439875] smpboot: Estimated ratio of average max frequency by base frequency (times 1024): 1647
[    0.444044] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    0.445563] serial 0000:00:16.3: enabling device (0000 -> 0003)
[    0.466834] 0000:00:16.3: ttyS4 at I/O 0x3088 (irq = 19, base_baud = 115200) is a 16550A
[    0.467122] hpet_acpi_add: no address or irqs in _CRS
[    0.467169] Linux agpgart interface v0.103
[    0.471628] tpm_tis IFX0785:00: 2.0 TPM (device-id 0x1B, rev-id 22)
[    1.056090] loop: module loaded
[    1.056362] tun: Universal TUN/TAP device driver, 1.6
[    1.056390] PPP generic driver version 2.4.2
[    1.056443] i8042: PNP: No PS/2 controller found.
[    1.056493] mousedev: PS/2 mouse device common for all mice
[    1.056625] rtc_cmos 00:03: RTC can wake from S4
[    1.058017] rtc_cmos 00:03: registered as rtc0
[    1.058359] rtc_cmos 00:03: setting system clock to 2022-05-06T04:18:04 UTC (1651810684)
[    1.058374] rtc_cmos 00:03: alarms up to one month, y3k, 242 bytes nvram
[    1.058382] i2c_dev: i2c /dev entries driver
[    1.058885] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[    1.058911] device-mapper: uevent: version 1.0.3
[    1.058961] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    1.058978] platform eisa.0: Probing EISA bus 0
[    1.058982] platform eisa.0: EISA: Cannot allocate resource for mainboard
[    1.058984] platform eisa.0: Cannot allocate resource for EISA slot 1
[    1.058985] platform eisa.0: Cannot allocate resource for EISA slot 2
[    1.058986] platform eisa.0: Cannot allocate resource for EISA slot 3
[    1.058988] platform eisa.0: Cannot allocate resource for EISA slot 4
[    1.058989] platform eisa.0: Cannot allocate resource for EISA slot 5
[    1.058990] platform eisa.0: Cannot allocate resource for EISA slot 6
[    1.058991] platform eisa.0: Cannot allocate resource for EISA slot 7
[    1.058993] platform eisa.0: Cannot allocate resource for EISA slot 8
[    1.058994] platform eisa.0: EISA: Detected 0 cards
[    1.058997] intel_pstate: Intel P-state driver initializing
[    1.059615] intel_pstate: HWP enabled
[    1.059667] ledtrig-cpu: registered to indicate activity on CPUs
[    1.059684] EFI Variables Facility v0.08 2004-May-17
[    1.097541] intel_pmc_core intel_pmc_core.0:  initialized
[    1.097579] drop_monitor: Initializing network drop monitor service
[    1.097691] NET: Registered PF_INET6 protocol family
[    1.322676] Freeing initrd memory: 61688K
[    1.327111] Segment Routing with IPv6
[    1.327120] In-situ OAM (IOAM) with IPv6
[    1.327158] NET: Registered PF_PACKET protocol family
[    1.327173] Bridge firewalling registered
[    1.327219] Key type dns_resolver registered
[    1.327842] microcode: sig=0xa0650, pf=0x2, revision=0xbe
[    1.328239] microcode: Microcode Update Driver: v2.2.
[    1.328243] IPI shorthand broadcast: enabled
[    1.328249] sched_clock: Marking stable (1315299966, 12937599)->(1353276051, -25038486)
[    1.328532] registered taskstats version 1
[    1.328848] Loading compiled-in X.509 certificates
[    1.329060] zswap: loaded using pool lzo/zbud
[    1.329563] Key type ._fscrypt registered
[    1.329564] Key type .fscrypt registered
[    1.329565] Key type fscrypt-provisioning registered
[    1.329711] Key type trusted registered
[    1.331490] Key type encrypted registered
[    1.331493] AppArmor: AppArmor sha1 policy hashing enabled
[    1.332048] integrity: Loading X.509 certificate: UEFI:db
[    1.332084] integrity: Loaded X.509 cert 'HP Inc.: HP UEFI Secure Boot DB 2017: d9c01b50cfcae89d3b05345c163aa76e5dd589e7'
[    1.332085] integrity: Loading X.509 certificate: UEFI:db
[    1.332110] integrity: Loaded X.509 cert 'Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53'
[    1.332111] integrity: Loading X.509 certificate: UEFI:db
[    1.332122] integrity: Loaded X.509 cert 'Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed522988a1bd4'
[    1.333054] Loading compiled-in module X.509 certificates
[    1.333544] Loaded X.509 cert 'Build time autogenerated kernel key: d9d2bb2da0b5e07d517f883e5eae262d385f8270'
[    1.333545] ima: Allocated hash algorithm: sha1
[    1.372830] ima: No architecture policies found
[    1.372843] evm: Initialising EVM extended attributes:
[    1.372843] evm: security.selinux
[    1.372844] evm: security.SMACK64
[    1.372845] evm: security.SMACK64EXEC
[    1.372845] evm: security.SMACK64TRANSMUTE
[    1.372846] evm: security.SMACK64MMAP
[    1.372846] evm: security.apparmor
[    1.372846] evm: security.ima
[    1.372847] evm: security.capability
[    1.372847] evm: HMAC attrs: 0x1
[    1.373064] PM:   Magic number: 6:137:312
[    1.373271] RAS: Correctable Errors collector initialized.
[    1.374099] Freeing unused decrypted memory: 2036K
[    1.374506] Freeing unused kernel image (initmem) memory: 2884K
[    1.403953] Write protecting the kernel read-only data: 28672k
[    1.404525] Freeing unused kernel image (text/rodata gap) memory: 2036K
[    1.404596] Freeing unused kernel image (rodata/data gap) memory: 60K
[    1.414734] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.414738] Run /init as init process
[    1.414739]   with arguments:
[    1.414739]     /init
[    1.414740]   with environment:
[    1.414740]     HOME=/
[    1.414741]     TERM=linux
[    1.414741]     BOOT_IMAGE=/boot/vmlinuz-5.15.30-2-pve
[    1.443897] tsc: Refined TSC clocksource calibration: 2303.998 MHz
[    1.443903] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2135f5e6f15, max_idle_ns: 440795231538 ns
[    1.444001] clocksource: Switched to clocksource tsc
[    1.496200] acpi PNP0C14:01: duplicate WMI GUID 05901221-D566-11D1-B2F0-00A0C9062910 (first instance was on PNP0C14:00)
[    1.496360] acpi PNP0C14:02: duplicate WMI GUID 2B814318-4BE8-4707-9D84-A190A859B5D0 (first instance was on PNP0C14:00)
[    1.496362] acpi PNP0C14:02: duplicate WMI GUID 41227C2D-80E1-423F-8B8E-87E32755A0EB (first instance was on PNP0C14:00)
[    1.496363] acpi PNP0C14:02: duplicate WMI GUID 05901221-D566-11D1-B2F0-00A0C9062910 (first instance was on PNP0C14:00)
[    1.497529] ahci 0000:00:17.0: version 3.0
[    1.497858] e1000e: Intel(R) PRO/1000 Network Driver
[    1.497860] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.497909] ahci 0000:00:17.0: AHCI 0001.0301 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
[    1.497914] ahci 0000:00:17.0: flags: 64bit ncq sntf pm clo only pio slum part ems deso sadm sds apst 
[    1.497955] i801_smbus 0000:00:1f.4: enabling device (0000 -> 0003)
[    1.498312] i801_smbus 0000:00:1f.4: SPD Write Disable is set
[    1.498405] i801_smbus 0000:00:1f.4: SMBus using PCI interrupt
[    1.498412] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    1.498418] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 1
[    1.498497] e1000e 0000:00:1f.6: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[    1.498582] scsi host0: ahci
[    1.498673] ata1: SATA max UDMA/133 abar m2048@0xe1239000 port 0xe1239100 irq 124
[    1.499062] i2c i2c-0: 2/2 memory slots populated (from DMI)
[    1.499537] i2c i2c-0: Successfully instantiated SPD at 0x50
[    1.499702] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x110 quirks 0x0000000000009810
[    1.500044] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.15
[    1.500046] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.500048] usb usb1: Product: xHCI Host Controller
[    1.500049] usb usb1: Manufacturer: Linux 5.15.30-2-pve xhci-hcd
[    1.500050] usb usb1: SerialNumber: 0000:00:14.0
[    1.500139] hub 1-0:1.0: USB hub found
[    1.500159] hub 1-0:1.0: 16 ports detected
[    1.503533] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    1.503538] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[    1.503541] xhci_hcd 0000:00:14.0: Host supports USB 3.1 Enhanced SuperSpeed
[    1.503589] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.15
[    1.503592] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.503593] usb usb2: Product: xHCI Host Controller
[    1.503595] usb usb2: Manufacturer: Linux 5.15.30-2-pve xhci-hcd
[    1.503596] usb usb2: SerialNumber: 0000:00:14.0
[    1.503740] hub 2-0:1.0: USB hub found
[    1.503760] hub 2-0:1.0: 10 ports detected
[    1.504756] usb: port power management may be unreliable
[    1.577332] e1000e 0000:00:1f.6 0000:00:1f.6 (uninitialized): registered PHC clock
[    1.585151] nvme nvme0: pci function 0000:01:00.0
[    1.585202] nvme nvme1: pci function 0000:02:00.0
[    1.594315] nvme nvme1: 12/0/0 default/read/poll queues
[    1.597192]  nvme1n1: p1 p2 p3
[    1.601683] nvme nvme0: allocated 32 MiB host memory buffer.
[    1.602928] nvme nvme0: 12/0/0 default/read/poll queues
[    1.646256] e1000e 0000:00:1f.6 eth0: (PCI Express:2.5GT/s:Width x1) e8:d8:d1:59:ab:82
[    1.646260] e1000e 0000:00:1f.6 eth0: Intel(R) PRO/1000 Network Connection
[    1.646355] e1000e 0000:00:1f.6 eth0: MAC: 13, PHY: 12, PBA No: FFFFFF-0FF
[    1.815933] ata1: SATA link down (SStatus 4 SControl 300)
[    1.825252] e1000e 0000:00:1f.6 eno1: renamed from eth0
[    1.835879] usb 1-10: new full-speed USB device number 2 using xhci_hcd
[    1.990211] usb 1-10: New USB device found, idVendor=046d, idProduct=c534, bcdDevice=29.01
[    1.990214] usb 1-10: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    1.990215] usb 1-10: Product: USB Receiver
[    1.990216] usb 1-10: Manufacturer: Logitech
[    2.115953] usb 1-14: new full-speed USB device number 3 using xhci_hcd
[    2.266671] usb 1-14: New USB device found, idVendor=8087, idProduct=0026, bcdDevice= 0.02
[    2.266675] usb 1-14: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    2.270761] hid: raw HID events driver (C) Jiri Kosina
[    2.273743] usbcore: registered new interface driver usbhid
[    2.273744] usbhid: USB HID core driver
[    2.273976] usbcore: registered new interface driver usbmouse
[    2.273981] usbcore: registered new interface driver usbkbd
[    2.274635] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.0/0003:046D:C534.0001/input/input3
[    2.332193] hid-generic 0003:046D:C534.0001: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:14.0-10/input0
[    2.332387] input: Logitech USB Receiver Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/input/input4
[    2.332606] input: Logitech USB Receiver Consumer Control as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/input/input5
[    2.392099] input: Logitech USB Receiver System Control as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/input/input6
[    2.392363] hid-generic 0003:046D:C534.0002: input,hiddev0,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:14.0-10/input1
[    2.460249] logitech-djreceiver 0003:046D:C534.0001: hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:14.0-10/input0
[    2.620572] logitech-djreceiver 0003:046D:C534.0002: hiddev0,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:14.0-10/input1
[    2.681581] logitech-djreceiver 0003:046D:C534.0002: device of type eQUAD nano Lite (0x0a) connected on slot 1
[    2.681862] input: Logitech Wireless Keyboard PID:4023 Keyboard as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/0003:046D:4023.0003/input/input9
[    2.682025] hid-generic 0003:046D:4023.0003: input,hidraw2: USB HID v1.11 Keyboard [Logitech Wireless Keyboard PID:4023] on usb-0000:00:14.0-10/input1:1
[    2.683581] logitech-djreceiver 0003:046D:C534.0002: device of type eQUAD nano Lite (0x0a) connected on slot 2
[    2.691713] input: Logitech Wireless Mouse PID:4022 as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/0003:046D:4022.0004/input/input14
[    2.736195] logitech-hidpp-device 0003:046D:4022.0004: input,hidraw2: USB HID v1.11 Mouse [Logitech Wireless Mouse PID:4022] on usb-0000:00:14.0-10/input1:2
[    2.741834] input: Logitech Wireless Keyboard PID:4023 as /devices/pci0000:00/0000:00:14.0/usb1/1-10/1-10:1.1/0003:046D:C534.0002/0003:046D:4023.0003/input/input15
[    2.742040] logitech-hidpp-device 0003:046D:4023.0003: input,hidraw3: USB HID v1.11 Keyboard [Logitech Wireless Keyboard PID:4023] on usb-0000:00:14.0-10/input1:1
[    3.019890] raid6: avx2x4   gen() 40916 MB/s
[    3.087859] raid6: avx2x4   xor() 18704 MB/s
[    3.155871] raid6: avx2x2   gen() 42396 MB/s
[    3.223866] raid6: avx2x2   xor() 26177 MB/s
[    3.291869] raid6: avx2x1   gen() 35909 MB/s
[    3.359869] raid6: avx2x1   xor() 22685 MB/s
[    3.427891] raid6: sse2x4   gen() 17345 MB/s
[    3.495884] raid6: sse2x4   xor()  9752 MB/s
[    3.563889] raid6: sse2x2   gen() 17571 MB/s
[    3.631869] raid6: sse2x2   xor() 10786 MB/s
[    3.699869] raid6: sse2x1   gen() 14877 MB/s
[    3.767866] raid6: sse2x1   xor()  7292 MB/s
[    3.767866] raid6: using algorithm avx2x2 gen() 42396 MB/s
[    3.767867] raid6: .... xor() 26177 MB/s, rmw enabled
[    3.767868] raid6: using avx2x2 recovery algorithm
[    3.768730] xor: automatically using best checksumming function   avx       
[    3.778061] Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
[    3.841781] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[    3.916253] systemd[1]: Inserted module 'autofs4'
[    3.992415] systemd[1]: systemd 247.3-7 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=unified)
[    4.012038] systemd[1]: Detected architecture x86-64.
[    4.012646] systemd[1]: Set hostname to <pve>.
[    4.106032] systemd[1]: Queued start job for default target Graphical Interface.
[    4.128994] systemd[1]: Created slice system-getty.slice.
[    4.129261] systemd[1]: Created slice system-modprobe.slice.
[    4.129469] systemd[1]: Created slice system-postfix.slice.
[    4.129647] systemd[1]: Created slice system-systemd\x2dfsck.slice.
[    4.129806] systemd[1]: Created slice User and Session Slice.
[    4.129847] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[    4.129874] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    4.129987] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[    4.130017] systemd[1]: Reached target ceph target allowing to start/stop all ceph-fuse@.service instances at once.
[    4.130024] systemd[1]: Reached target ceph target allowing to start/stop all ceph*@.service instances at once.
[    4.130031] systemd[1]: Reached target Local Encrypted Volumes.
[    4.130059] systemd[1]: Reached target Paths.
[    4.130076] systemd[1]: Reached target Slices.
[    4.130087] systemd[1]: Reached target Swap.
[    4.130095] systemd[1]: Reached target System Time Set.
[    4.130150] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[    4.130210] systemd[1]: Listening on LVM2 poll daemon socket.
[    4.132359] systemd[1]: Listening on RPCbind Server Activation Socket.
[    4.132453] systemd[1]: Listening on Syslog Socket.
[    4.132518] systemd[1]: Listening on fsck to fsckd communication Socket.
[    4.132557] systemd[1]: Listening on initctl Compatibility Named Pipe.
[    4.132642] systemd[1]: Listening on Journal Audit Socket.
[    4.132693] systemd[1]: Listening on Journal Socket (/dev/log).
[    4.132757] systemd[1]: Listening on Journal Socket.
[    4.132951] systemd[1]: Listening on udev Control Socket.
[    4.132999] systemd[1]: Listening on udev Kernel Socket.
[    4.133435] systemd[1]: Mounting Huge Pages File System...
[    4.133878] systemd[1]: Mounting POSIX Message Queue File System...
[    4.134350] systemd[1]: Mounting RPC Pipe File System...
[    4.134878] systemd[1]: Mounting Kernel Debug File System...
[    4.135361] systemd[1]: Mounting Kernel Trace File System...
[    4.135401] systemd[1]: Condition check resulted in Kernel Module supporting RPCSEC_GSS being skipped.
[    4.136007] systemd[1]: Starting Set the console keyboard layout...
[    4.136494] systemd[1]: Starting Create list of static device nodes for the current kernel...
[    4.137143] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
[    4.137683] systemd[1]: Starting Load Kernel Module configfs...
[    4.138162] systemd[1]: Starting Load Kernel Module drm...
[    4.138697] systemd[1]: Starting Load Kernel Module fuse...
[    4.139111] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
[    4.139133] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[    4.140177] systemd[1]: Starting Journal Service...
[    4.141145] systemd[1]: Starting Load Kernel Modules...
[    4.142003] systemd[1]: Starting Remount Root and Kernel File Systems...
[    4.142932] systemd[1]: Starting Coldplug All udev Devices...
[    4.144549] systemd[1]: Mounted Huge Pages File System.
[    4.144642] systemd[1]: Mounted POSIX Message Queue File System.
[    4.144713] systemd[1]: Mounted Kernel Debug File System.
[    4.144778] systemd[1]: Mounted Kernel Trace File System.
[    4.145074] systemd[1]: Finished Set the console keyboard layout.
[    4.145428] systemd[1]: Finished Create list of static device nodes for the current kernel.
[    4.145632] systemd[1]: modprobe@configfs.service: Succeeded.
[    4.145810] systemd[1]: Finished Load Kernel Module configfs.
[    4.146019] systemd[1]: modprobe@fuse.service: Succeeded.
[    4.146443] systemd[1]: Finished Load Kernel Module fuse.
[    4.146481] VFIO - User Level meta-driver version: 0.3
[    4.146614] RPC: Registered named UNIX socket transport module.
[    4.146616] RPC: Registered udp transport module.
[    4.146617] RPC: Registered tcp transport module.
[    4.146617] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    4.147237] systemd[1]: Mounting FUSE Control File System...
[    4.147796] systemd[1]: Mounting Kernel Configuration File System...
[    4.148239] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro. Quota mode: none.
[    4.149552] systemd[1]: Mounted RPC Pipe File System.
[    4.149745] systemd[1]: modprobe@drm.service: Succeeded.
[    4.149972] systemd[1]: Finished Load Kernel Module drm.
[    4.150312] systemd[1]: Finished Remount Root and Kernel File Systems.
[    4.150434] systemd[1]: Mounted FUSE Control File System.
[    4.150519] systemd[1]: Mounted Kernel Configuration File System.
[    4.150973] vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    4.151526] systemd[1]: Started Device-mapper event daemon.
[    4.152011] systemd[1]: Condition check resulted in Rebuild Hardware Database being skipped.
[    4.152054] systemd[1]: Condition check resulted in Platform Persistent Storage Archival being skipped.
[    4.152748] systemd[1]: Starting Load/Save Random Seed...
[    4.153520] systemd[1]: Starting Create System Users...
[    4.161153] systemd[1]: Finished Load/Save Random Seed.
[    4.161234] systemd[1]: Condition check resulted in First Boot Complete being skipped.
[    4.162244] systemd[1]: Finished Create System Users.
[    4.162816] systemd[1]: Starting Create Static Device Nodes in /dev...
[    4.167975] vfio_pci: add [8086:9bc8[ffffffff:ffffffff]] class 0x000000/00000000
[    4.169728] systemd[1]: Finished Create Static Device Nodes in /dev.
[    4.170217] Loading iSCSI transport class v2.0-870.
[    4.170924] systemd[1]: Starting Rule-based Manager for Device Events and Files...
[    4.172569] iscsi: registered transport (tcp)
[    4.185246] iscsi: registered transport (iser)
[    4.185964] systemd[1]: Started Rule-based Manager for Device Events and Files.
[    4.188129] systemd[1]: Started Journal Service.
[    4.190481] spl: loading out-of-tree module taints kernel.
[    4.192097] znvpair: module license 'CDDL' taints kernel.
[    4.192100] Disabling lock debugging due to kernel taint
[    4.195086] systemd-journald[439]: Received client request to flush runtime journal.
[    4.217253] intel_pch_thermal 0000:00:12.0: enabling device (0000 -> 0002)
[    4.220602] mei_me 0000:00:16.0: enabling device (0000 -> 0002)
[    4.228873] input: PC Speaker as /devices/platform/pcspkr/input/input16
[    4.229832] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    4.230159] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    4.231580] ee1004 0-0050: 512 byte EE1004-compliant SPD EEPROM, read-only
[    4.231720] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    4.231725] cfg80211: failed to load regulatory.db
[    4.253800] hp_wmi: query 0x4 returned error 0x5
[    4.263351] hp_wmi: query 0xd returned error 0x5
[    4.268656] pstore: Using crash dump compression: deflate
[    4.268673] pstore: Registered efi as persistent store backend
[    4.268728] input: HP WMI hotkeys as /devices/virtual/input/input17
[    4.275345] hp_wmi: query 0x1b returned error 0x5
[    4.288068] Intel(R) Wireless WiFi driver for Linux
[    4.288127] iwlwifi 0000:00:14.3: enabling device (0000 -> 0002)
[    4.294341] Bluetooth: Core ver 2.22
[    4.294527] NET: Registered PF_BLUETOOTH protocol family
[    4.294529] Bluetooth: HCI device and connection manager initialized
[    4.294532] Bluetooth: HCI socket layer initialized
[    4.294534] Bluetooth: L2CAP socket layer initialized
[    4.294538] Bluetooth: SCO socket layer initialized
[    4.295356] RAPL PMU: API unit is 2^-32 Joules, 5 fixed counters, 655360 ms ovfl timer
[    4.295361] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
[    4.295362] RAPL PMU: hw unit of domain package 2^-14 Joules
[    4.295363] RAPL PMU: hw unit of domain dram 2^-14 Joules
[    4.295364] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
[    4.295365] RAPL PMU: hw unit of domain psys 2^-14 Joules
[    4.295395] iwlwifi 0000:00:14.3: api flags index 2 larger than supported by driver
[    4.295426] iwlwifi 0000:00:14.3: TLV_FW_FSEQ_VERSION: FSEQ Version: 89.3.35.37
[    4.296002] iwlwifi 0000:00:14.3: loaded firmware version 66.f1c864e0.0 QuZ-a0-hr-b0-66.ucode op_mode iwlmvm
[    4.296492] cryptd: max_cpu_qlen set to 1000
[    4.304242] AVX2 version of gcm_enc/dec engaged.
[    4.304294] AES CTR mode by8 optimization enabled
[    4.306603] usbcore: registered new interface driver btusb
[    4.308600] Bluetooth: hci0: Bootloader revision 0.4 build 0 week 30 2018
[    4.309606] Bluetooth: hci0: Device revision is 2
[    4.309610] Bluetooth: hci0: Secure boot is enabled
[    4.309612] Bluetooth: hci0: OTP lock is enabled
[    4.309613] Bluetooth: hci0: API lock is enabled
[    4.309614] Bluetooth: hci0: Debug lock is disabled
[    4.309615] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[    4.310478] Bluetooth: hci0: Failed to load Intel firmware file intel/ibt-19-0-4.sfi (-2)
[    4.311597] Bluetooth: hci0: Failed to read MSFT supported features (-56)
[    4.317517] iwlwifi 0000:00:14.3: Detected Intel(R) Wi-Fi 6 AX201 160MHz, REV=0x354
[    4.330161] thermal thermal_zone1: failed to read out thermal zone (-61)
[    4.375550] ZFS: Loaded module v2.1.4-pve1, ZFS pool version 5000, ZFS filesystem version 5
[    4.377941] intel_tcc_cooling: Programmable TCC Offset detected
[    4.379668] intel_rapl_common: Found RAPL domain package
[    4.379670] intel_rapl_common: Found RAPL domain core
[    4.379672] intel_rapl_common: Found RAPL domain uncore
[    4.379673] intel_rapl_common: Found RAPL domain dram
[    4.379674] intel_rapl_common: Found RAPL domain psys
[    4.441288] iwlwifi 0000:00:14.3: Detected RF HR B3, rfid=0x10a100
[    4.506528] iwlwifi 0000:00:14.3: base HW address: 58:96:1d:29:97:f3
[    4.523907] iwlwifi 0000:00:14.3 wlp0s20f3: renamed from wlan0
[    4.869976] audit: type=1400 audit(1651810688.305:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/lxc-start" pid=702 comm="apparmor_parser"
[    4.871014] audit: type=1400 audit(1651810688.305:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=706 comm="apparmor_parser"
[    4.871702] audit: type=1400 audit(1651810688.305:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=700 comm="apparmor_parser"
[    4.871706] audit: type=1400 audit(1651810688.305:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=700 comm="apparmor_parser"
[    4.871865] audit: type=1400 audit(1651810688.309:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=703 comm="apparmor_parser"
[    4.871870] audit: type=1400 audit(1651810688.309:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=703 comm="apparmor_parser"
[    4.871873] audit: type=1400 audit(1651810688.309:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=703 comm="apparmor_parser"
[    4.871910] audit: type=1400 audit(1651810688.309:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/chronyd" pid=704 comm="apparmor_parser"
[    4.874546] audit: type=1400 audit(1651810688.309:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="tcpdump" pid=699 comm="apparmor_parser"
[    4.891588] softdog: initialized. soft_noboot=0 soft_margin=60 sec soft_panic=0 (nowayout=0)
[    4.891591] softdog:              soft_reboot_cmd=<not set> soft_active_on_boot=0
[    5.162050] vmbr0: port 1(eno1) entered blocking state
[    5.162055] vmbr0: port 1(eno1) entered disabled state
[    5.162087] device eno1 entered promiscuous mode
[    5.680198] kauditd_printk_skb: 4 callbacks suppressed
[    5.680200] audit: type=1400 audit(1651810689.119:15): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="/usr/bin/lxc-start" pid=842 comm="apparmor_parser"
[    5.693082] audit: type=1400 audit(1651810689.131:16): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default" pid=851 comm="apparmor_parser"
[    5.693086] audit: type=1400 audit(1651810689.131:17): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-cgns" pid=851 comm="apparmor_parser"
[    5.693105] audit: type=1400 audit(1651810689.131:18): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-mounting" pid=851 comm="apparmor_parser"
[    5.693107] audit: type=1400 audit(1651810689.131:19): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxc-container-default-with-nesting" pid=851 comm="apparmor_parser"
[    7.273090] bpfilter: Loaded bpfilter_umh pid 1036
[    7.273237] Started bpfilter
[    7.988460] e1000e 0000:00:1f.6 eno1: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx
[    7.988558] vmbr0: port 1(eno1) entered blocking state
[    7.988560] vmbr0: port 1(eno1) entered forwarding state
[    7.988669] IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0: link becomes ready
furkanmustafa commented 2 years ago

:thinking: hmm..

Does modprobe force-remove-bootfb work?

kukat commented 2 years ago

🤔 hmm..

Does modprobe force-remove-bootfb work?

root@pve:~# modprobe force-remove-bootfb modprobe: FATAL: Module force-remove-bootfb not found in directory /lib/modules/5.15.30-2-pve

furkanmustafa commented 2 years ago

/lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko this file does not exist then?

furkanmustafa commented 2 years ago

Probably not caused by the SSL(Signing) errors

(Looking at here as a reference https://github.com/umlaeute/v4l2loopback/blob/main/Makefile#L56-L58)

kukat commented 2 years ago

/lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko this file does not exist then?

exists

root@pve:~# ll /lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko
-rw-r--r-- 1 root root 97920 May  6 12:08 /lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko
root@pve:~# file /lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko
/lib/modules/5.15.30-2-pve/extra/force-remove-bootfb.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), BuildID[sha1]=5a625046e70b7e3d29620a3e2833bbe17c3cfa89, with debug_info, not stripped
furkanmustafa commented 2 years ago

Looking at; https://eng.libretexts.org/Bookshelves/Computer_Science/Operating_Systems/Linux_-_The_Penguin_Marches_On_(McClanahan)/06%3A_Kernel_Module_Management/2.05%3A_Kernel_Module_Management_-_lsmod_Command/2.05.02%3A_Kernel_Module_Management_-_modprobe_Command

Your initial make install had this warning as well;

  DEPMOD  /lib/modules/5.15.30-2-pve
Warning: modules_install: missing 'System.map' file. Skipping depmod.

I don't know if it is a problem for modprobe. But maybe you can try depmod -a first, and then try modprobe force-remove-bootfb again ?

kukat commented 2 years ago

modprobe force-remove-bootfb after depmod -a works

after reboot, now i can see the force-remove-bootfb is loaded

root@pve:~# lsmod | grep -i bootfb
force_remove_bootfb    16384  0

dmesg says forcefully-remove-bootfb was executed

root@pve:~# dmesg | grep -i bootfb
[    4.207466] force_remove_bootfb: loading out-of-tree module taints kernel.
[    4.207584] forcefully-remove-bootfb loaded
[    4.207584] forcefully-remove-bootfb 0xd0000000-0xd1fa3fff

but BOOTFB still exists

root@pve:~# grep BOOTFB /proc/iomem | awk '{ print $1 }'
d0000000-d02fffff
furkanmustafa commented 2 years ago

Oh okay.. depmod is necessary for modprobe to work then.

Now, the address range has changed!

Probably because of order of events..

To prevent re-initializing(?) a framebuffer after freeing the bootfb mem region, you might want to enable these kernel parameters as well; video=efifb:off video=vesa:off video=simpledrmfb:off video=simplefb:off video=nvidiafb:off modprobe.blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm

kukat commented 2 years ago

add the parameters into /etc/default/grub

...
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on video=efifb:off video=vesa:off video=simpledrmfb:off video=simplefb:off video=nvidiafb:off modprobe.blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm"
GRUB_CMDLINE_LINUX=""
...

then update-grub

/boot/grub/grub.cfg looks good

...
menuentry 'Proxmox VE GNU/Linux' --class proxmox --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-b9987f0f-8846-4c70-8677-871fb06ae7aa' {
        load_video
        insmod gzio
        if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
        insmod part_gpt
        insmod lvm
        insmod ext2
        set root='lvmid/5y6WwS-HWCe-e5J8-Gzgp-p3T3-kZM9-DrUAyD/Q3KIww-mwlm-Fg9d-UbdE-l8Nd-WVzs-Gmwgk5'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint='lvmid/5y6WwS-HWCe-e5J8-Gzgp-p3T3-kZM9-DrUAyD/Q3KIww-mwlm-Fg9d-UbdE-l8Nd-WVzs-Gmwgk5'  b9987f0f-8846-4c70-8677-871fb06ae7aa
        else
          search --no-floppy --fs-uuid --set=root b9987f0f-8846-4c70-8677-871fb06ae7aa
        fi
        echo    'Loading Linux 5.15.30-2-pve ...'
        linux   /boot/vmlinuz-5.15.30-2-pve root=/dev/mapper/pve-root ro  quiet intel_iommu=on video=efifb:off video=vesa:off video=simpledrmfb:off video=simplefb:off video=nvidiafb:off modprobe.blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm blacklist=radeon,amdgpu,nouveau,nvidiafb,simpledrm
        echo    'Loading initial ramdisk ...'
        initrd  /boot/initrd.img-5.15.30-2-pve
}
...

reboot

root@pve:~# grep BOOTFB /proc/iomem | awk '{ print $1 }'
d0000000-d02fffff
furkanmustafa commented 2 years ago

aaaand it's still there :D I'm suspecting that a driver is still initializing it (after our module removes it).

okay.. instead of loading this module at boot time, when you loaded it manually post-boot, it worked fine, right?

If so, then it's just a timing issue. Something is still interfering with the modules and/or framebuffers after this. We want to load our module as late as possible.

You can probably do this by removing it from /etc/modules and make a custom init script.

(In my setup, I am running ./remove.sh everytime manually after boot)

kukat commented 2 years ago

okay.. instead of loading this module at boot time, when you loaded it manually post-boot, it worked fine, right?

no, it didn't work.

now I have removed/comment out from /etc/modules and /etc/modprobe.d/force-remove-bootfb.conf update-grub+ update-initramfs -u -k all

reboot

root@pve:~# grep BOOTFB /proc/iomem | awk '{ print $1 }'
d0000000-d02fffff
root@pve:~# cd forcefully-remove-bootfb/
root@pve:~/forcefully-remove-bootfb# ./remove.sh
++ grep BOOTFB /proc/iomem
++ awk '{ print $1 }'
+ range=d0000000-d02fffff
+ IFS=-
+ read -ra ADDR
+ start=d0000000
+ end=d02fffff
+ sleep 2
+ insmod build/force-remove-bootfb.ko bootfb_start=0xd0000000 bootfb_end=0xd02fffff
+ rmmod force-remove-bootfb
root@pve:~/forcefully-remove-bootfb# grep BOOTFB /proc/iomem | awk '{ print $1 }'
d0000000-d02fffff

still same

btw, I guess the address range was changed because of the kernel parameters, I didn't revert back the parameters so it didn't change back to d0000000-d1fa3fff

kukat commented 2 years ago

I just found out this is kernel issue, if I downgrade the kernel to 5.11.22-7-pve, it works out of the box.

it doesn't show BOOTFB in /proc/iomem at all.

Problem solved. thank you @furkanmustafa for your kindly help.

furkanmustafa commented 2 years ago

It's better to bisect and pinpoint this issue to a specific commit at some point (not necessarily now).

@kukat Can you share your Mainboard maker & model and BIOS version? ( dmidecode command would give these information )

Aterfax commented 2 years ago

Can also confirm this behaviour on my Proxmox 7.2-3 install, 5.13.19-6-pve = no issues, 5.15.30-2-pve = sticky BOOTFB which cannot be removed.

dmidecode -s bios-version

F33i

dmidecode -t baseboard


# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 3.3.0 present.

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
        Manufacturer: Gigabyte Technology Co., Ltd.
        Product Name: X570 AORUS ULTRA
        Version: x.x
        Serial Number: Default string
        Asset Tag: Default string
        Features:
                Board is a hosting board
                Board is replaceable
        Location In Chassis: Default string
        Chassis Handle: 0x0003
        Type: Motherboard
        Contained Object Handles: 0

Handle 0x0004, DMI type 10, 6 bytes
On Board Device Information
        Type: Video
        Status: Enabled
        Description:    To Be Filled By O.E.M.

Handle 0x0030, DMI type 41, 11 bytes
Onboard Device
        Reference Designation: RTL8111EPV
        Type: Ethernet
        Status: Disabled
        Type Instance: 1
        Bus Address: 0000:03:00.0
kukat commented 2 years ago

It's better to bisect and pinpoint this issue to a specific commit at some point (not necessarily now).

@kukat Can you share your Mainboard maker & model and BIOS version? ( dmidecode command would give these information )

It's HP ProDesk 600 G6 Desktop Mini PC https://www.hp.com/us-en/shop/pdp/hp-prodesk-600-g6-desktop-mini-pc-customizable-9bd72av-mb 

root@pve:~# dmidecode -s bios-version
S22 Ver. 02.10.10
root@pve:~# dmidecode -t baseboard
# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 3.2 present.

Handle 0x0003, DMI type 2, 17 bytes
Base Board Information
        Manufacturer: HP
        Product Name: 8715
        Version: KBC Version 09.97.00
        Serial Number: *****************
        Asset Tag:
        Features:
                Board is a hosting board
        Location In Chassis:
        Chassis Handle: 0x0000
        Type: Motherboard
        Contained Object Handles: 0

Handle 0x0033, DMI type 41, 11 bytes
Onboard Device
        Reference Designation: Onboard IGD
        Type: Video
        Status: Enabled
        Type Instance: 1
        Bus Address: 0000:00:02.0

Handle 0x0034, DMI type 41, 11 bytes
Onboard Device
        Reference Designation: Onboard Lan
        Type: Ethernet
        Status: Enabled
        Type Instance: 1
        Bus Address: 0000:00:1f.6

Handle 0x0039, DMI type 10, 6 bytes
On Board Device Information
        Type: Video
        Status: Enabled
        Description: 64 MB
furkanmustafa commented 2 years ago

Thanks for the information!

Btw, looks like I have some additional setup before running ./remove.sh.

# Release all vtconsoles
for vtdir in $(find /sys/class/vtconsole -name "vtcon*"); do
  vtname=$(basename ${vtdir}) # vtcon0 vtcon1 ..
  [ "${vtname}" != "vtconsole" ] || continue
  echo "Unbinding: ${vtname}"
  echo 0 > ${vtdir}/bind
done

# Release all framebuffers
for fbdir in $(find /sys/bus/platform/drivers -type d -name "*-framebuffer"); do
  fbnameprefix="$(basename $fbdir)." # efi-framebuffer.0 efi-framebuffer.1
  for fb in $(find $fbdir -name "${fbnameprefix}*"); do
    fbname=$(basename ${fb})
    echo "Unbinding: $fbname"
    echo ${fbname} > ${fbdir}/unbind
  done
done

# And then run `./remove.sh`

Can you try with above on 5.15 ? ( @kukat @Aterfax )