xcat2 / xcat-core

Code repo for xCAT core packages
Eclipse Public License 1.0
367 stars 172 forks source link

"genimage ubuntu18.04.1-x86_64-netboot-compute" generates broken image #5887

Closed jamesfreeman959 closed 5 years ago

jamesfreeman959 commented 5 years ago

Hi all,

I have been working with xcat 2.14.5 on Ubuntu Server 18.04.1 LTS for a few days and have completed most of the steps required to network boot a SuperMicro motherboard using a stateless image.

As part of the build process, I have copied the image from the loopback mounted ISO:

# /opt/xcat/bin/copycds-cdrom /iso/ubuntu-18.04.1-server-amd64.iso

And then generated the netboot image:

# genimage ubuntu18.04.1-x86_64-netboot-compute

The process appears to start well and the output from the command looks fine, but right at the end of the process I see the following line repeated over and over:

ldd: ./linux-vdso.so.1: No such file or directory

Just before this happens, the output from the command is:

Processing triggers for libc-bin (2.27-3ubuntu1) ...
Umount /proc, /dev, /sys, pkgdir and otherpkgdir to the rootimg.
Setting up the locales
Generating locales (this might take a while)...
  en_US.ISO-8859-1... done
  en_US.UTF-8... done
Generation complete.
Generating locales (this might take a while)...
  en_US.ISO-8859-1... done
  en_US.UTF-8... done
Generation complete.
Added ptp.ko as an autodetected dependency
Added pps_core.ko as an autodetected dependency
Added dca.ko as an autodetected dependency
Added i2c-algo-bit.ko as an autodetected dependency
Added mdio.ko as an autodetected dependency
Added libcrc32c.ko as an autodetected dependency
ldd: ./linux-vdso.so.1: No such file or directory
ldd: ./linux-vdso.so.1: No such file or directory

The image then seems to complete:

ldd: ./linux-vdso.so.1: No such file or directory
cd /tmp/xcatinitrd.23297;find .|cpio -H newc -o|gzip -9 -c - > /install/netboot/ubuntu18.04.1/x86_64/compute/initrd-stateless.gz
685063 blocks

And I then pack the image:

# packimage ubuntu18.04.1-x86_64-netboot-compute
Packing contents of /install/netboot/ubuntu18.04.1/x86_64/compute/rootimg
archive method:cpio
compress method:gzip

Apart from the linux-vdso.so.1 errors, everything looks like it have worked successfully. I then configure my node for network boot and power it on:

# rsetboot com1 net -u 
# nodeset com1 osimage=ubuntu18.04.1-x86_64-netboot-compute 
# rpower com1 boot

The node powers on, and loads the UEFI network stack, which in turn obtains an IP address, loads the boot image and boots the kernel. 13 seconds into the boot process, everything simply hangs and the boot does not proceed any further. The last drivers loaded by the kernel were the network drivers - however I am not clear whether the hang is driver related, or because it was trying to move on from the kernel to the next phase of the boot. I am a little concerned about the linux-vdso.so.1 errors - I am not sure if these mean the stateless boot image has an error in it, or whether it these can safely be ignored. The presence of the errors, combined with a boot image that doesn't quite get there does make me think something went wrong during the genimage phase. Can anyone confirm if this is indeed the case, and whether this is an issue, or simply something I need to deal with in a different manner?

Thanks in advance for your help,

James

cxhong commented 5 years ago

for ppc64el, we need to download mini.iso. I think it apply to x86_64 images also.

try to download ubuntu mini.iso. http://archive.ubuntu.com/ubuntu/dists/bionic-updates/main/installer-amd64/current/images/netboot/

and follow this instruction for ubuntu image: https://xcat-docs.readthedocs.io/en/stable/guides/admin-guides/manage_clusters/ppc64le/diskless/copy_image.html?highlight=mini.iso

bybai commented 5 years ago

@jamesfreeman959,

  1. Confirm you answer: For x86_64, we do not need to download mini.iso. I can reproduce genimage error as " ldd: ./linux-vdso.so.1: No such file or directory" on ubuntu18.04.1 x86_64 management node. But I think it is not a block issue for genimage or diskless provision, you can ignore the message. My diskless compute node can be provisioned successfully using commands.
    /# packimage ubuntu18.04.1-x86_64-netboot-compute
    Packing contents of /install/netboot/ubuntu18.04.1/x86_64/compute/rootimg
    archive method:cpio
    compress method:pigz
    /# rinstall c910f04x35v04 osimage=ubuntu18.04.1-x86_64-netboot-compute

    Since I cannot find linux-vdso.so.1 from the following ubuntu18.04.1 repo, genimage print cannot find linux-vdso.so.1,

    /# cat /etc/apt/sources.list
    deb http://archive.ubuntu.com/ubuntu bionic main restricted universe
    deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe
    deb http://archive.ubuntu.com/ubuntu bionic-security main restricted universe
    /# apt-get update
    Get:1 file:/install/xcat/xcat-core bionic InRelease [2568 B]
    Get:2 file:/install/xcat/xcat-dep bionic InRelease [2561 B]
    Get:1 file:/install/xcat/xcat-core bionic InRelease [2568 B]
    Get:2 file:/install/xcat/xcat-dep bionic InRelease [2561 B]
    Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease
    Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
    Get:5 http://archive.ubuntu.com/ubuntu bionic-security InRelease [83.2 kB]
    Fetched 172 kB in 2s (93.4 kB/s)
    Reading package lists... Done
    /# apt-file search linux-vdso.so.1
  2. Could you check apache2 log under /var/log/apache2 to see if it download rootimg.cpio.gz or something wrong?
    The correct log process like:
    xxxx - - [13/Dec/2018:00:30:55 -0500] "GET /tftpboot/xcat/xnba/nodes/c910f04x35v04 HTTP/1.1" 200 709 "-" "iPXE/1.0.3-130626"
    xxxx - - [13/Dec/2018:00:30:55 -0500] "GET /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel HTTP/1.1" 200 8257483 "-" "iPXE/1.0.3-130626"
    xxxx - - [13/Dec/2018:00:30:56 -0500] "GET /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/initrd-stateless.gz HTTP/1.1" 200 163733285 "-" "iPXE/1.0.3-130626"
    xxxx - - [13/Dec/2018:00:31:22 -0500] "GET //install/netboot/ubuntu18.04.1/x86_64/compute/rootimg.cpio.gz HTTP/1.1" 200 312224001 "-" "Wget/1.19.4 (linux-gnu)"
    xxx - - [13/Dec/2018:00:31:35 -0500] "GET /install/postscripts/ HTTP/1.1" 200 28363 "-" "Wget/1.19.4 (
jamesfreeman959 commented 5 years ago

Thank you both for your quick responses and confirming that the linux-vdso.so.1 errors can be ignored. I ran the boot process a second time, and it looks like it hangs after loading the initrd-stateless.gz image - there is no attempt to download rootimg.cpio.gz. I also note that there are no errors in /var/log/apache2/error.log

Here is the content from /var/log/apache2/access.log:

192.168.221.51 - - [13/Dec/2018:10:44:46 +0000] "GET /tftpboot/xcat/xnba/nodes/com1.uefi HTTP/1.1" 200 309 "-" "iPXE/1.0.3-130626"
192.168.221.51 - - [13/Dec/2018:10:44:46 +0000] "GET /tftpboot/xcat/elilo-x64.efi HTTP/1.1" 200 236326 "-" "iPXE/1.0.3-130626"
192.168.221.51 - - [13/Dec/2018:10:44:46 +0000] "GET /tftpboot/xcat/xnba/nodes/com1.elilo HTTP/1.1" 200 573 "-" "iPXE/1.0.3-130626"
192.168.221.51 - - [13/Dec/2018:10:44:46 +0000] "GET /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel HTTP/1.1" 200 8257483 "-" "iPXE/1.0.3-130626"
192.168.221.51 - - [13/Dec/2018:10:44:46 +0000] "GET /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/initrd-stateless.gz HTTP/1.1" 200 163832808 "-" "iPXE/1.0.3-130626"

How can I help to debug this?

immarvin commented 5 years ago

hi @jamesfreeman959 , can you provide the console output during boot up?

jamesfreeman959 commented 5 years ago

Sorry for the delay - it took me a little while to get SOL working - I can now share the complete kernel boot process with you, and I think it shows where it has broken. I hope this helps:

>>Checking Media Presence......
>>Media Present......
>>Start PXE over IPv4. Press ESC key to abort PXE boot.
  Station IP address is 192.168.221.51

  Server IP address is 192.168.221.252
  NBP filename is xcat/xnba.efi
  NBP filesize is 91744 Bytes

>>Checking Media Presence......
>>Media Present......
 Downloading NBP file...

  NBP file downloaded successfully.
xNBA initialising devices...ok

xCAT Network Boot Agent
iPXE 1.0.3-130626 -- Open Source Network Boot Firmware -- http://ipxe.org
Features: HTTP HTTPS iSCSI DNS TFTP EFI
net0: ac:1f:6b:45:f8:84 using <NULL> on EFI SNP (open)
  [Link:up, TX:0 TXE:0 RX:0 RXE:0]
DHCP (net0 ac:1f:6b:45:f8:84)... ok
net0: 192.168.221.51/255.255.255.0 gw 192.168.221.252
Next server: 192.168.221.252
Filename: http://${next-server}:80/tftpboot/xcat/xnba/nodes/com1.uefi
http://192.168.221.252:80/tftpboot/xcat/xnba/nodes/com1.uefi... ok
http://192.168.221.252:80/tftpboot/xcat/elilo-x64.efi... ok
ELILO v3.14 for EFI/x86_64
Loading kernel /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel...  done
Loading file /tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/initrd-stateless.gz...done
[    0.000000] Linux version 4.15.0-29-generic (buildd@lgw01-amd64-057) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018 (Ubuntu 4.15.0-29.31-generic 4.15.18)
[    0.000000] Command line: BOOT_IMAGE=net0:/tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel  imgurl=http://192.168.221.252://install/netboot/ubuntu18.04.1/x86_64/compute/rootimg.cpio.gz XCAT=192.168.221.252:3001  XCATHTTPPORT= console=tty0 console=ttyS1,115200n8r BOOTIF=01-ac-1f-6b-45-f8-84
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: xstate_offset[3]:  576, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  640, xstate_sizes[4]:   64
[    0.000000] x86/fpu: Enabled xstate features 0x1b, context size is 704 bytes, using 'compacted' format.
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000003efff] usable
[    0.000000] BIOS-e820: [mem 0x000000000003f000-0x000000000003ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000040000-0x000000000009ffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000000a0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000773abfff] usable
[    0.000000] BIOS-e820: [mem 0x00000000773ac000-0x00000000773bbfff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000773bc000-0x000000007d2cdfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007d2ce000-0x000000007ddaafff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007ddab000-0x000000007e185fff] usable
[    0.000000] BIOS-e820: [mem 0x000000007e186000-0x000000007e1cffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007e1d0000-0x000000007f171fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007f172000-0x000000007f23ffff] unusable
[    0.000000] BIOS-e820: [mem 0x000000007f240000-0x000000007f7fffff] usable
[    0.000000] BIOS-e820: [mem 0x000000007f800000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fd000000-0x00000000fe7fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000027fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.60 by American Megatrends
[    0.000000] efi:  ACPI 2.0=0x7e186000  ACPI=0x7e186000  SMBIOS=0x7f0c8000  SMBIOS 3.0=0x7f0c7000  MEMATTR=0x79e4e018  ESRT=0x79d78e98
[    0.000000] secureboot: Secure boot could not be determined (mode 0)
[    0.000000] SMBIOS 3.0.0 present.
[    0.000000] DMI: Supermicro Super Server/A2SDi-4C-HLN4F, BIOS 1.1a 09/18/2018
[    0.000000] e820: last_pfn = 0x280000 max_arch_pfn = 0x400000000
[    0.000000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT
[    0.000000] total RAM covered: 8184M
[    0.000000] Found optimal setting for mtrr clean up
[    0.000000]  gran_size: 64K  chunk_size: 16M     num_reg: 4      lose cover RAM: 0G
[    0.000000] e820: last_pfn = 0x7f800 max_arch_pfn = 0x400000000
[    0.000000] esrt: Reserving ESRT space from 0x0000000079d78e98 to 0x0000000079d78ed0.
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] Using GB pages for direct mapping
[    0.000000] RAMDISK: [mem 0x62aad000-0x6c6ebfff]
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000007E186000 000024 (v02 SUPERM)
[    0.000000] ACPI: XSDT 0x000000007E186098 0000AC (v01 SUPERM SMCI--MB 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 0x000000007E18AB38 000114 (v06 SUPERM SMCI--MB 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 0x000000007E1861D8 00495C (v02 SUPERM SMCI--MB 01072009 INTL 20061109)
[    0.000000] ACPI: FACS 0x000000007E1CE080 000040
[    0.000000] ACPI: FPDT 0x000000007E18AC50 000044 (v01 SUPERM SMCI--MB 01072009 AMI  00010013)
[    0.000000] ACPI: FIDT 0x000000007E18AC98 00009C (v01 SUPERM SMCI--MB 01072009 AMI  00010013)
[    0.000000] ACPI: SPMI 0x000000007E18AD38 000041 (v05 SUPERM SMCI--MB 00000000 AMI. 00000000)
[    0.000000] ACPI: MCFG 0x000000007E18AD80 00003C (v01 SUPERM SMCI--MB 01072009 MSFT 00000097)
[    0.000000] ACPI: WDAT 0x000000007E18ADC0 0001AC (v01 SUPERM SMCI--MB 01072009 MSFT 00010013)
[    0.000000] ACPI: APIC 0x000000007E18AF70 000078 (v04 INTEL  TIANO    00000001 MSFT 00000000)
[    0.000000] ACPI: BDAT 0x000000007E18AFE8 000030 (v01                 00000000      00000000)
[    0.000000] ACPI: HPET 0x000000007E18B018 000038 (v01 SUPERM SMCI--MB 00000001 MSFT 01000013)
[    0.000000] ACPI: UEFI 0x000000007E18B050 000042 (v01 SUPERM SMCI--MB 00000002      01000013)
[    0.000000] ACPI: SSDT 0x000000007E18B098 001901 (v02 PmRef  CpuPm    00003000 INTL 20061109)
[    0.000000] ACPI: DMAR 0x000000007E18C9A0 000070 (v01 INTEL  BDW      00000001 INTL 00000001)
[    0.000000] ACPI: HEST 0x000000007E18CA10 0000A8 (v01 INTEL  VND      00000001 INTL 00000001)
[    0.000000] ACPI: BERT 0x000000007E18CAB8 000030 (v01 INTEL  VND      00000001 INTL 00000001)
[    0.000000] ACPI: ERST 0x000000007E18CAE8 000230 (v01 INTEL  VND      00000001 INTL 00000001)
[    0.000000] ACPI: EINJ 0x000000007E18CD18 000150 (v01 INTEL  VND      00000001 INTL 00000001)
[    0.000000] ACPI: WSMT 0x000000007E18CE68 000028 (v01 u?              01072009 AMI  00010013)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000027fffffff]
[    0.000000] NODE_DATA(0) allocated [mem 0x27ffd5000-0x27fffffff]
[    0.000000] tsc: Using PIT calibration value
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.000000]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000027fffffff]
[    0.000000]   Device   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000003efff]
[    0.000000]   node   0: [mem 0x0000000000040000-0x000000000009ffff]
[    0.000000]   node   0: [mem 0x0000000000100000-0x00000000773abfff]
[    0.000000]   node   0: [mem 0x00000000773bc000-0x000000007d2cdfff]
[    0.000000]   node   0: [mem 0x000000007ddab000-0x000000007e185fff]
[    0.000000]   node   0: [mem 0x000000007f240000-0x000000007f7fffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000027fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000027fffffff]
[    0.000000] Reserved but unavailable: 97 pages
[    0.000000] tboot: non-0 tboot_addr but it is not of type E820_TYPE_RESERVED
[    0.000000] ACPI: PM-Timer IO Port: 0x1808
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
[    0.000000] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.000000] PM: Registered nosave memory: [mem 0x0003f000-0x0003ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x773ac000-0x773bbfff]
[    0.000000] PM: Registered nosave memory: [mem 0x7d2ce000-0x7ddaafff]
[    0.000000] PM: Registered nosave memory: [mem 0x7e186000-0x7e1cffff]
[    0.000000] PM: Registered nosave memory: [mem 0x7e1d0000-0x7f171fff]
[    0.000000] PM: Registered nosave memory: [mem 0x7f172000-0x7f23ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x7f800000-0x7fffffff]
[    0.000000] PM: Registered nosave memory: [mem 0x80000000-0xdfffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xe0000000-0xefffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xf0000000-0xfcffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfd000000-0xfe7fffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfe800000-0xfeffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]
[    0.000000] e820: [mem 0x80000000-0xdfffffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.000000] random: get_random_bytes called from start_kernel+0x99/0x4fd with crng_init=0
[    0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
[    0.000000] percpu: Embedded 46 pages/cpu @        (ptrval) s151552 r8192 d28672 u524288
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 2055214
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=net0:/tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel  imgurl=http://192.168.221.252://install/netboot/ubuntu18.04.1/x86_64/compute/rootimg.cpio.gz XCAT=192.168.221.252:3001  XCATHTTPPORT= console=tty0 console=ttyS1,115200n8r BOOTIF=01-ac-1f-6b-45-f8-84
[    0.000000] Memory: 7912024K/8351708K available (12300K kernel code, 2470K rwdata, 4240K rodata, 2408K init, 2416K bss, 439684K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 39092 entries in 153 pages
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4.
[    0.000000]  Tasks RCU enabled.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS1] enabled
[    0.000000] ACPI: Core revision 20170831
[    0.000000] ACPI: 2 ACPI AML tables successfully acquired and loaded
[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns
[    0.004000] APIC: Switch to symmetric I/O mode setup
[    0.008000] DMAR: Host address width 39
[    0.012000] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.016000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.020000] DMAR: RMRR base: 0x0000007f0fa000 end: 0x0000007f10afff
[    0.024000] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
[    0.028000] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.032000] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.036000] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.036000] x2apic enabled
[    0.040000] Switched APIC routing to cluster x2apic.
[    0.044000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.068000] tsc: PIT calibration matches HPET. 1 loops
[    0.072000] tsc: Detected 2200.002 MHz processor
[    0.076000] tsc: Detected 2200.000 MHz TSC
[    0.076000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4400.00 BogoMIPS (lpj=8800000)
[    0.084003] pid_max: default: 32768 minimum: 301
[    0.092055] Security Framework initialized
[    0.096003] Yama: becoming mindful.
[    0.099527] AppArmor: AppArmor initialized
[    0.106455] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.109254] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.112086] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.116043] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.120362] CPU: Physical Processor ID: 0
[    0.124005] CPU: Processor Core ID: 2
[    0.128022] mce: CPU supports 9 MCE banks
[    0.132031] CPU0: Thermal monitoring enabled (TM1)
[    0.136077] process: using mwait in idle threads
[    0.140004] Last level iTLB entries: 4KB 48, 2MB 0, 4MB 0
[    0.144002] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.148004] Spectre V2 : Mitigation: Full generic retpoline
[    0.152002] Spectre V2 : Spectre v2 mitigation: Enabling Indirect Branch Prediction Barrier
[    0.156001] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    0.160003] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.169962] Freeing SMP alternatives memory: 36K
[    0.178069] smpboot: CPU0: Intel(R) Atom(TM) CPU C3558 @ 2.20GHz (family: 0x6, model: 0x5f, stepping: 0x1)
[    0.180136] Performance Events: PEBS fmt3+, Goldmont events, 32-deep LBR, full-width counters, Intel PMU driver.
[    0.184010] ... version:                4
[    0.188003] ... bit width:              48
[    0.192002] ... generic registers:      4
[    0.196002] ... value mask:             0000ffffffffffff
[    0.200002] ... max period:             00007fffffffffff
[    0.204002] ... fixed-purpose events:   3
[    0.208002] ... event mask:             000000070000000f
[    0.212062] Hierarchical SRCU implementation.
[    0.217281] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[    0.220017] smp: Bringing up secondary CPUs ...
[    0.224185] x86: Booting SMP configuration:
[    0.228009] .... node  #0, CPUs:      #1 #2 #3
[    0.228608] smp: Brought up 1 node, 4 CPUs
[    0.236004] smpboot: Max logical packages: 1
[    0.240004] smpboot: Total of 4 processors activated (17600.00 BogoMIPS)
[    0.246424] devtmpfs: initialized
[    0.248092] x86/mm: Memory block size: 128MB
[    0.253126] evm: security.selinux
[    0.256006] evm: security.SMACK64
[    0.259316] evm: security.SMACK64EXEC
[    0.260005] evm: security.SMACK64TRANSMUTE
[    0.264002] evm: security.SMACK64MMAP
[    0.267664] evm: security.apparmor
[    0.268006] evm: security.ima
[    0.270971] evm: security.capability
[    0.272059] PM: Registering ACPI NVS region [mem 0x7e186000-0x7e1cffff] (303104 bytes)
[    0.276099] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.280018] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.284111] pinctrl core: initialized pinctrl subsystem
[    0.288170] RTC time: 16:23:23, date: 12/17/18
[    0.294515] NET: Registered protocol family 16
[    0.296131] audit: initializing netlink subsys (disabled)
[    0.300028] audit: type=2000 audit(1545063802.300:1): state=initialized audit_enabled=0 res=1
[    0.308008] cpuidle: using governor ladder
[    0.312009] cpuidle: using governor menu
[    0.316020] ACPI: bus type PCI registered
[    0.320004] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.324088] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.328006] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
[    0.332022] PCI: Using configuration type 1 for base access
[    0.337789] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.340008] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.344138] ACPI: Added _OSI(Module Device)
[    0.348004] ACPI: Added _OSI(Processor Device)
[    0.352009] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.356002] ACPI: Added _OSI(Processor Aggregator Device)
[    0.360003] ACPI: Added _OSI(Linux-Dell-Video)
[    0.372639] ACPI: Dynamic OEM Table Load:
[    0.376009] ACPI: SSDT 0xFFFF9FB535E73E00 00007A (v02 PmRef  Cpu0Cst  00003001 INTL 20061109)
[    0.380356] ACPI: Dynamic OEM Table Load:
[    0.384007] ACPI: SSDT 0xFFFF9FB535DA1000 0004B3 (v02 PmRef  Cpu0Ist  00003000 INTL 20061109)
[    0.388807] ACPI: Dynamic OEM Table Load:
[    0.392011] ACPI: SSDT 0xFFFF9FB535EFD000 00063B (v02 PmRef  ApIst    00003000 INTL 20061109)
[    0.396891] ACPI: Dynamic OEM Table Load:
[    0.400007] ACPI: SSDT 0xFFFF9FB535E8C000 000231 (v02 PmRef  ApCst    00003000 INTL 20061109)
[    0.405825] ACPI: Interpreter enabled
[    0.408028] ACPI: (supports S0 S4 S5)
[    0.411691] ACPI: Using IOAPIC for interrupt routing
[    0.412091] HEST: Table parsing has been initialized.
[    0.416006] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.420328] ACPI: Enabled 4 GPEs in block 00 to 7F
[    0.430814] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
[    0.438490] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.440011] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.444569] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
[    0.448690] PCI host bridge to bus 0000:00
[    0.452005] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.456004] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.460003] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.464003] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000dffff window]
[    0.468003] pci_bus 0000:00: root bus resource [mem 0x80000000-0xdfffffff window]
[    0.472003] pci_bus 0000:00: root bus resource [mem 0x280000000-0x7fffffffff window]
[    0.476003] pci_bus 0000:00: root bus resource [mem 0xfe010000-0xfe010fff window]
[    0.480003] pci_bus 0000:00: root bus resource [mem 0xfa000000-0xfa0fffff window]
[    0.484003] pci_bus 0000:00: root bus resource [mem 0xfb000000-0xfb7fffff window]
[    0.488003] pci_bus 0000:00: root bus resource [mem 0xfa100000-0xfa1007ff window]
[    0.492004] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.497077] pci 0000:00:11.0: enabling Extended Tags
[    0.502536] pci 0000:01:00.0: enabling Extended Tags
[    0.504074] pci 0000:01:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR0 for 16 VFs)
[    0.508012] pci 0000:01:00.0: VF(n) BAR2 space: [mem 0x00000000-0x0000ffff 64bit] (contains BAR2 for 16 VFs)
[    0.512120] pci 0000:00:06.0: PCI bridge to [bus 01]
[    0.516071] pci 0000:00:0a.0: PCI bridge to [bus 02]
[    0.520139] pci 0000:03:00.0: enabling Extended Tags
[    0.524154] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.528010] pci 0000:00:11.0: PCI bridge to [bus 03-04]
[    0.532203] pci 0000:04:00.0: BAR 0: assigned to efifb
[    0.536151] pci 0000:03:00.0: PCI bridge to [bus 04]
[    0.540250] pci 0000:05:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR0 for 64 VFs)
[    0.544020] pci 0000:05:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR3 for 64 VFs)
[    0.548362] pci 0000:05:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR0 for 64 VFs)
[    0.552020] pci 0000:05:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR3 for 64 VFs)
[    0.556238] pci 0000:00:16.0: PCI bridge to [bus 05]
[    0.560250] pci 0000:06:00.0: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR0 for 64 VFs)
[    0.564020] pci 0000:06:00.0: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR3 for 64 VFs)
[    0.568363] pci 0000:06:00.1: VF(n) BAR0 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR0 for 64 VFs)
[    0.572020] pci 0000:06:00.1: VF(n) BAR3 space: [mem 0x00000000-0x000fffff 64bit] (contains BAR3 for 64 VFs)
[    0.576237] pci 0000:00:17.0: PCI bridge to [bus 06]
[    0.580289] ACPI: PCI Interrupt Link [LNKA] (IRQs 6 7 10 *11 12 14 15)
[    0.584135] ACPI: PCI Interrupt Link [LNKB] (IRQs 6 7 *10 11 12 14 15)
[    0.588135] ACPI: PCI Interrupt Link [LNKC] (IRQs *6 7 10 11 12 14 15)
[    0.592134] ACPI: PCI Interrupt Link [LNKD] (IRQs 6 *7 10 11 12 14 15)
[    0.596134] ACPI: PCI Interrupt Link [LNKE] (IRQs 6 7 10 11 *12 14 15)
[    0.600134] ACPI: PCI Interrupt Link [LNKF] (IRQs 6 7 10 11 12 *14 15)
[    0.604134] ACPI: PCI Interrupt Link [LNKG] (IRQs 6 7 10 11 12 14 *15)
[    0.608134] ACPI: PCI Interrupt Link [LNKH] (IRQs 6 7 10 11 12 14 *15)
[    0.613582] SCSI subsystem initialized
[    0.616064] pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    0.620006] pci 0000:04:00.0: vgaarb: bridge control possible
[    0.624003] pci 0000:04:00.0: vgaarb: setting as boot device
[    0.628002] vgaarb: loaded
[    0.630744] ACPI: bus type USB registered
[    0.632022] usbcore: registered new interface driver usbfs
[    0.636016] usbcore: registered new interface driver hub
[    0.640022] usbcore: registered new device driver usb
[    0.644081] EDAC MC: Ver: 3.0.0
[    0.647279] Registered efivars operations
[    0.654676] PCI: Using ACPI for IRQ routing
[    0.664183] NetLabel: Initializing
[    0.667589] NetLabel:  domain hash size = 128
[    0.668002] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.672019] NetLabel:  unlabeled traffic allowed by default
[    0.676092] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.680004] hpet0: 8 comparators, 64-bit 24.000000 MHz counter
[    0.686041] clocksource: Switched to clocksource hpet
[    0.707362] VFS: Disk quotas dquot_6.6.0
[    0.711350] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.718398] AppArmor: AppArmor Filesystem Enabled
[    0.723157] pnp: PnP ACPI init
[    0.726468] system 00:00: [io  0x1800-0x18fe] has been reserved
[    0.732404] system 00:00: [mem 0xfed03000-0xfed03fff] has been reserved
[    0.739474] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    0.746096] system 00:01: [mem 0x7fc00000-0x7fffffff] has been reserved
[    0.752715] system 00:01: [mem 0xfed60000-0xfed60fff] has been reserved
[    0.759396] pnp 00:02: disabling [mem 0x000e0000-0x000fffff] because it overlaps 0000:05:00.0 BAR 7 [mem 0x00000000-0x000fffff 64bit]
[    0.771396] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:05:00.0 BAR 10 [mem 0x00000000-0x000fffff 64bit]
[    0.784264] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:05:00.1 BAR 7 [mem 0x00000000-0x000fffff 64bit]
[    0.797046] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:05:00.1 BAR 10 [mem 0x00000000-0x000fffff 64bit]
[    0.809908] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:06:00.0 BAR 7 [mem 0x00000000-0x000fffff 64bit]
[    0.822696] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:06:00.0 BAR 10 [mem 0x00000000-0x000fffff 64bit]
[    0.835561] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:06:00.1 BAR 7 [mem 0x00000000-0x000fffff 64bit]
[    0.848343] pnp 00:02: disabling [mem 0x000e0000-0x000fffff disabled] because it overlaps 0000:06:00.1 BAR 10 [mem 0x00000000-0x000fffff 64bit]
[    0.861235] system 00:02: [io  0x0680-0x069f] has been reserved
[    0.867167] system 00:02: [mem 0xff800000-0xffffffff] has been reserved
[    0.874390] system 00:04: [io  0x0a00-0x0a0f] has been reserved
[    0.880324] system 00:04: [io  0x0a10-0x0a1f] has been reserved
[    0.886257] system 00:04: [io  0x0a20-0x0a2f] has been reserved
[    0.892185] system 00:04: [io  0x0a30-0x0a3f] has been reserved
[    0.898112] system 00:04: [io  0x0a40-0x0a4f] has been reserved
[    0.905394] pnp: PnP ACPI: found 7 devices
[    0.917593] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.926523] pci 0000:01:00.0: BAR 7: assigned [mem 0xdf380000-0xdf38ffff 64bit]
[    0.933842] pci 0000:01:00.0: BAR 9: assigned [mem 0xdf390000-0xdf39ffff 64bit]
[    0.941164] pci 0000:00:06.0: PCI bridge to [bus 01]
[    0.946144] pci 0000:00:06.0:   bridge window [mem 0xdf300000-0xdf3fffff]
[    0.952939] pci 0000:00:0a.0: PCI bridge to [bus 02]
[    0.957917] pci 0000:03:00.0: PCI bridge to [bus 04]
[    0.962896] pci 0000:03:00.0:   bridge window [io  0xd000-0xdfff]
[    0.969000] pci 0000:03:00.0:   bridge window [mem 0xde000000-0xdf0fffff]
[    0.975807] pci 0000:00:11.0: PCI bridge to [bus 03-04]
[    0.981043] pci 0000:00:11.0:   bridge window [io  0xd000-0xdfff]
[    0.987144] pci 0000:00:11.0:   bridge window [mem 0xde000000-0xdf0fffff]
[    0.993944] pci 0000:05:00.0: BAR 7: no space for [mem size 0x00100000 64bit]
[    1.001088] pci 0000:05:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit]
[    1.008577] pci 0000:05:00.0: BAR 10: no space for [mem size 0x00100000 64bit]
[    1.015802] pci 0000:05:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit]
[    1.023379] pci 0000:05:00.1: BAR 7: no space for [mem size 0x00100000 64bit]
[    1.030519] pci 0000:05:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit]
[    1.038007] pci 0000:05:00.1: BAR 10: no space for [mem size 0x00100000 64bit]
[    1.045235] pci 0000:05:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit]
[    1.052811] pci 0000:00:16.0: PCI bridge to [bus 05]
[    1.057785] pci 0000:00:16.0:   bridge window [mem 0xdf200000-0xdf2fffff]
[    1.064579] pci 0000:00:16.0:   bridge window [mem 0xdda00000-0xddefffff 64bit pref]
[    1.072329] pci 0000:06:00.0: BAR 7: no space for [mem size 0x00100000 64bit]
[    1.079477] pci 0000:06:00.0: BAR 7: failed to assign [mem size 0x00100000 64bit]
[    1.086964] pci 0000:06:00.0: BAR 10: no space for [mem size 0x00100000 64bit]
[    1.094192] pci 0000:06:00.0: BAR 10: failed to assign [mem size 0x00100000 64bit]
[    1.101766] pci 0000:06:00.1: BAR 7: no space for [mem size 0x00100000 64bit]
[    1.108908] pci 0000:06:00.1: BAR 7: failed to assign [mem size 0x00100000 64bit]
[    1.116397] pci 0000:06:00.1: BAR 10: no space for [mem size 0x00100000 64bit]
[    1.123624] pci 0000:06:00.1: BAR 10: failed to assign [mem size 0x00100000 64bit]
[    1.131197] pci 0000:00:17.0: PCI bridge to [bus 06]
[    1.136174] pci 0000:00:17.0:   bridge window [mem 0xdf100000-0xdf1fffff]
[    1.142966] pci 0000:00:17.0:   bridge window [mem 0xdd400000-0xdd8fffff 64bit pref]
[    1.150723] pci 0000:00:17.0: PCI bridge to [bus 06]
[    1.155703] pci 0000:00:16.0: PCI bridge to [bus 05]
[    1.160716] pci 0000:00:16.0: BAR 14: assigned [mem 0x80000000-0x804fffff]
[    1.167600] pci 0000:00:17.0: BAR 14: assigned [mem 0x80500000-0x809fffff]
[    1.174488] pci 0000:00:06.0: PCI bridge to [bus 01]
[    1.179462] pci 0000:00:06.0:   bridge window [mem 0xdf300000-0xdf3fffff]
[    1.186259] pci 0000:00:0a.0: PCI bridge to [bus 02]
[    1.191243] pci 0000:03:00.0: PCI bridge to [bus 04]
[    1.196222] pci 0000:03:00.0:   bridge window [io  0xd000-0xdfff]
[    1.202325] pci 0000:03:00.0:   bridge window [mem 0xde000000-0xdf0fffff]
[    1.209123] pci 0000:00:11.0: PCI bridge to [bus 03-04]
[    1.214359] pci 0000:00:11.0:   bridge window [io  0xd000-0xdfff]
[    1.220463] pci 0000:00:11.0:   bridge window [mem 0xde000000-0xdf0fffff]
[    1.227263] pci 0000:05:00.0: BAR 6: assigned [mem 0x80000000-0x8007ffff pref]
[    1.234496] pci 0000:05:00.1: BAR 6: assigned [mem 0x80080000-0x800fffff pref]
[    1.241731] pci 0000:05:00.0: BAR 7: assigned [mem 0x80100000-0x801fffff 64bit]
[    1.249047] pci 0000:05:00.0: BAR 10: assigned [mem 0x80200000-0x802fffff 64bit]
[    1.256457] pci 0000:05:00.1: BAR 7: assigned [mem 0x80300000-0x803fffff 64bit]
[    1.263782] pci 0000:05:00.1: BAR 10: assigned [mem 0x80400000-0x804fffff 64bit]
[    1.271191] pci 0000:00:16.0: PCI bridge to [bus 05]
[    1.276171] pci 0000:00:16.0:   bridge window [mem 0x80000000-0x804fffff]
[    1.282966] pci 0000:00:16.0:   bridge window [mem 0xdda00000-0xddefffff 64bit pref]
[    1.290716] pci 0000:06:00.0: BAR 6: assigned [mem 0x80500000-0x8057ffff pref]
[    1.297950] pci 0000:06:00.1: BAR 6: assigned [mem 0x80580000-0x805fffff pref]
[    1.305179] pci 0000:06:00.0: BAR 7: assigned [mem 0x80600000-0x806fffff 64bit]
[    1.312495] pci 0000:06:00.0: BAR 10: assigned [mem 0x80700000-0x807fffff 64bit]
[    1.319905] pci 0000:06:00.1: BAR 7: assigned [mem 0x80800000-0x808fffff 64bit]
[    1.327229] pci 0000:06:00.1: BAR 10: assigned [mem 0x80900000-0x809fffff 64bit]
[    1.334640] pci 0000:00:17.0: PCI bridge to [bus 06]
[    1.339620] pci 0000:00:17.0:   bridge window [mem 0x80500000-0x809fffff]
[    1.346421] pci 0000:00:17.0:   bridge window [mem 0xdd400000-0xdd8fffff 64bit pref]
[    1.354335] NET: Registered protocol family 2
[    1.359053] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    1.366477] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.373408] TCP: Hash tables configured (established 65536 bind 65536)
[    1.380039] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    1.386180] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    1.392824] NET: Registered protocol family 1
[    1.397654] Unpacking initramfs...
[    3.904434] Freeing initrd memory: 159996K
[    3.908671] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    3.915127] software IO TLB [mem 0x7332b000-0x7732b000] (64MB) mapped at [        (ptrval)-        (ptrval)]
[    3.925031] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1fb633008a4, max_idle_ns: 440795292230 ns
[    3.935207] Scanning for low memory corruption every 60 seconds
[    3.941972] Initialise system trusted keyrings
[    3.946441] Key type blacklist registered
[    3.950531] workingset: timestamp_bits=36 max_order=21 bucket_order=0
[    3.958658] zbud: loaded
[    3.961888] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    3.967947] fuse init (API version 7.26)
[    3.975075] Key type asymmetric registered
[    3.979186] Asymmetric key parser 'x509' registered
[    3.984113] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[    3.991579] io scheduler noop registered
[    3.995514] io scheduler deadline registered
[    3.999852] io scheduler cfq registered (default)
[    4.005910] pcieport 0000:00:06.0: AER enabled with IRQ 25
[    4.011444] pcieport 0000:00:0a.0: AER enabled with IRQ 26
[    4.016979] pcieport 0000:00:11.0: AER enabled with IRQ 27
[    4.022505] pcieport 0000:00:16.0: AER enabled with IRQ 28
[    4.028044] pcieport 0000:00:17.0: AER enabled with IRQ 29
[    4.033559] pcieport 0000:00:06.0: Signaling PME with IRQ 25
[    4.039251] pcieport 0000:00:0a.0: Signaling PME with IRQ 26
[    4.044941] pcieport 0000:00:11.0: Signaling PME with IRQ 27
[    4.050629] pcieport 0000:00:16.0: Signaling PME with IRQ 28
[    4.056314] pcieport 0000:00:17.0: Signaling PME with IRQ 29
[    4.062008] dpc 0000:00:0a.0:pcie010: DPC error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
[    4.074980] dpc 0000:00:11.0:pcie010: DPC error containment capabilities: Int Msg #0, RPExt+ PoisonedTLP+ SwTrigger+ RP PIO Log 4, DL_ActiveErr+
[    4.088049] efifb: probing for efifb
[    4.091643] efifb: framebuffer at 0xde000000, using 1876k, total 1875k
[    4.098177] efifb: mode is 800x600x32, linelength=3200, pages=1
[    4.104102] efifb: scrolling: redraw
[    4.107678] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    4.123913] Console: switching to colour frame buffer device 100x37
[    4.140653] fb0: EFI VGA frame buffer device
[    4.145445] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    4.153142] ACPI: Power Button [PWRF]
[    4.157479] ERST: Error Record Serialization Table (ERST) support is initialized.
[    4.165209] pstore: using zlib compression
[    4.169447] pstore: Registered erst as persistent store backend
[    4.175919] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC.
[    4.183706] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    4.210948] 00:05: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    4.239279] 00:06: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
[    4.250045] Linux agpgart interface v0.103
[    4.256766] loop: module loaded
[    4.264931] libphy: Fixed MDIO Bus: probed
[    4.273722] tun: Universal TUN/TAP device driver, 1.6
[    4.283598] PPP generic driver version 2.4.2
[    4.292744] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    4.304337] ehci-pci: EHCI PCI platform driver
[    4.313682] ehci-platform: EHCI generic platform driver
[    4.323920] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    4.335251] ohci-pci: OHCI PCI platform driver
[    4.344925] ohci-platform: OHCI generic platform driver
[    4.355610] uhci_hcd: USB Universal Host Controller Interface driver
[    4.367629] xhci_hcd 0000:00:15.0: xHCI Host Controller
[    4.378391] xhci_hcd 0000:00:15.0: new USB bus registered, assigned bus number 1
[    4.392570] xhci_hcd 0000:00:15.0: hcc params 0x200077c1 hci version 0x100 quirks 0x01109810
[    4.407052] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    4.419987] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.433214] usb usb1: Product: xHCI Host Controller
[    4.443971] usb usb1: Manufacturer: Linux 4.15.0-29-generic xhci-hcd
[    4.456171] usb usb1: SerialNumber: 0000:00:15.0
[    4.466691] hub 1-0:1.0: USB hub found
[    4.476060] hub 1-0:1.0: 4 ports detected
[    4.486539] xhci_hcd 0000:00:15.0: xHCI Host Controller
[    4.497316] xhci_hcd 0000:00:15.0: new USB bus registered, assigned bus number 2
[    4.510358] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    4.522819] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    4.535686] usb usb2: Product: xHCI Host Controller
[    4.546149] usb usb2: Manufacturer: Linux 4.15.0-29-generic xhci-hcd
[    4.558071] usb usb2: SerialNumber: 0000:00:15.0
[    4.568422] hub 2-0:1.0: USB hub found
[    4.577523] hub 2-0:1.0: 4 ports detected
[    4.587682] i8042: PNP: No PS/2 controller found.
[    4.597800] mousedev: PS/2 mouse device common for all mice
[    4.608760] rtc_cmos 00:03: RTC can wake from S4
[    4.618749] rtc_cmos 00:03: rtc core: registered rtc_cmos as rtc0
[    4.629976] rtc_cmos 00:03: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
[    4.642873] i2c /dev entries driver
[    4.651653] device-mapper: uevent: version 1.0.3
[    4.661545] device-mapper: ioctl: 4.37.0-ioctl (2017-09-20) initialised: dm-devel@redhat.com
[    4.675535] ledtrig-cpu: registered to indicate activity on CPUs
[    4.686938] EFI Variables Facility v0.08 2004-May-17
[    4.705647] NET: Registered protocol family 10
[    4.719026] Segment Routing with IPv6
[    4.727969] NET: Registered protocol family 17
[    4.737808] Key type dns_resolver registered
[    4.748228] intel_rdt: Intel RDT L2 allocation detected
[    4.758722] RAS: Correctable Errors collector initialized.
[    4.769564] microcode: sig=0x506f1, pf=0x1, revision=0x24
[    4.780326] microcode: Microcode Update Driver: v2.2.
[    4.780340] sched_clock: Marking stable (4780313378, 0)->(5092781612, -312468234)
[    4.804073] registered taskstats version 1
[    4.813454] Loading compiled-in X.509 certificates
[    4.826252] Loaded X.509 cert 'Build time autogenerated kernel key: 1cc37f41791529df57507d3747a6804ab53f482b'
[    4.832053] usb 1-3: new high-speed USB device number 2 using xhci_hcd
[    4.859155] Couldn't get size: 0x800000000000000e
[    4.869307] MODSIGN: Couldn't get UEFI db list
[    4.879369] Couldn't get size: 0x800000000000000e
[    4.889587] MODSIGN: Couldn't get UEFI MokListRT
[    4.899872] Couldn't get size: 0x800000000000000e
[    4.910064] MODSIGN: Couldn't get UEFI dbx list
[    4.920021] zswap: loaded using pool lzo/zbud
[    4.932654] Key type big_key registered
[    4.941810] Key type trusted registered
[    4.952177] Key type encrypted registered
[    4.961351] AppArmor: AppArmor sha1 policy hashing enabled
[    4.961359] clocksource: Switched to clocksource tsc
[    4.982244] usb 1-3: New USB device found, idVendor=05e3, idProduct=0610
[    4.982246] usb 1-3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[    4.982248] usb 1-3: Product: USB2.0 Hub
[    4.982793] hub 1-3:1.0: USB hub found
[    4.983091] hub 1-3:1.0: 4 ports detected
[    5.034305] ima: No TPM chip found, activating TPM-bypass! (rc=-19)
[    5.045879] evm: HMAC attrs: 0x1
[    5.054683]   Magic number: 14:833:388
[    5.063535] thermal cooling_device3: hash matches
[    5.073404] rtc_cmos 00:03: setting system clock to 2018-12-17 16:23:29 UTC (1545063809)
[    5.087988] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[    5.099175] EDD information not available.
[    5.113382] Freeing unused kernel memory: 2408K
[    5.136057] Write protecting the kernel read-only data: 20480k
[    5.148278] Freeing unused kernel memory: 2008K
[    5.165695] Freeing unused kernel memory: 1904K
[    5.188117] x86/mm: Checked W+X mappings: passed, no W+X pages found.
mkdir: can't create directory '/dev/mapper': File exists
[    5.212436] dca service started, version 1.12.1
[    5.224475] pps_core: LinuxPPS API ver. 1 registered
[    5.234611] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    5.256694] PTP clock support registered
[    5.272022] usb 1-3.1: new high-speed USB device number 3 using xhci_hcd
[    5.298334] bnx2x: QLogic 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.712.30-0 (2014/02/10)
[    5.317686] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[    5.330033] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    5.348411] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.359593] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.377485] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.389901] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.401092] usb 1-3.1: New USB device found, idVendor=0557, idProduct=7000
insmod: can't insert '/lib/mlx_en.ko': No such file or directory[    5.413539] usb 1-3.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0

[    5.426910] hub 1-3.1:1.0: USB hub found
Creating device nodes with udev
/init: line 97: /sbin/udevd: not found
[    5.436550] hub 1-3.1:1.0: 4 ports detected
Couldn't find the proper booting device, switch to shell...

 Entering rescue/debug init shell.
 Exit shell to continue booting.
sh: cannot set terminal process group (-1): Inappropriate ioctl for device
sh: no job control in this shell
[    5.528054] usb 1-3.2: new high-speed USB device number 4 using xhci_hcd
sh-4.4# [    5.641538] usb 1-3.2: New USB device found, idVendor=05e3, idProduct=0610
[    5.654142] usb 1-3.2: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[    5.667145] usb 1-3.2: Product: USB2.0 Hub
[    5.677506] hound
[    5.687323] hub 1-3.2:1.0: 4 ports detected
[    5.776047] usb 1-3.1.1: new low-speed USB device number 5 using xhci_hcd
[    5.916405] usb 1-3.1.1: New USB device found, idVendor=0557, idProduct=2419
[    5.929174] usb 1-3.1.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
immarvin commented 5 years ago

oh, from the console, I find [ 0.000000] Kernel command line: BOOT_IMAGE=net0:/tftpboot/xcat/osimage/ubuntu18.04.1-x86_64-netboot-compute/kernel imgurl=http://192.168.221.252://install/netboot/ubuntu18.04.1/x86_64/compute/rootimg.cpio.gz XCAT=192.168.221.252:3001 XCATHTTPPORT= console=tty0 console=ttyS1,115200n8r BOOTIF=01-ac-1f-6b-45-f8-84, apparently, the imgurl=http://192.168.221.252://install/netboot/ubuntu18.04.1/x86_64/compute/rootimg.cpio.gz is incorrect. this is bug for xCAT, fixed in https://github.com/xcat2/xcat-core/pull/5902/files , you can apply the fix or use the latest xCAT development build on http://xcat.org/download.html#snapshot_builds, sorry for inconvenience

jamesfreeman959 commented 5 years ago

Hi there,

Thank you for this - I tried updating my xcat install from the snapshot builds, but the Debian.pm file referenced by https://github.com/xcat2/xcat-core/pull/5902/files did not change, so I simply patched it manually.

I then regenerated the image but booted the node, but it fails at the same place as before. The output right before the failure seems to be:

[    5.298334] bnx2x: QLogic 5771x/578xx 10/20-Gigabit Ethernet Driver bnx2x 1.712.30-0 (2014/02/10)
[    5.317686] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[    5.330033] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    5.348411] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    5.359593] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    5.377485] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k
[    5.389901] igb: Copyright (c) 2007-2014 Intel Corporation.
[    5.401092] usb 1-3.1: New USB device found, idVendor=0557, idProduct=7000
insmod: can't insert '/lib/mlx_en.ko': No such file or directory[    5.413539] usb 1-3.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0

[    5.426910] hub 1-3.1:1.0: USB hub found
Creating device nodes with udev
/init: line 97: /sbin/udevd: not found
[    5.436550] hub 1-3.1:1.0: 4 ports detected
Couldn't find the proper booting device, switch to shell...

I note there is an insmod failure, but mlx_en.ko is a Mellanox module as I understand it and there is no Mellanox hardware on my system. Can this error be safely ignored? All network adapters look to have been successfully enumerated before this point (e1000e, bnx2x) so I believe the network is sound - however the failure still relates to not finding a proper boot device - presumably this is because the the root image has not been downloaded.

Not sure how to proceed - I'm not clear where the failure is related to a driver load issue, or something else in the process.

Many thanks

James

bybai commented 5 years ago

Hi @jamesfreeman959 , "can't insert '/lib/mlx_en.ko': No such file or directory" , this error can be safely ignored here since you do not have Mellanox hardware.

immarvin commented 5 years ago

hi @jamesfreeman959 ,

Please provide the /proc/cmdline content in the emergency shell.

the fix for the issue I mentioned has been included in the latest development build http://xcat.org/files/xcat/xcat-core/devel/Ubuntu/core-snap/20181228.0615-core-debs-snap.tar.bz2, if you are ok, you can take a try.

jamesfreeman959 commented 5 years ago

Thanks for all your help everyone! I found the issue - my challenge was lack of familiarity with the xcat code, and confusion over the various error messages (again lack of familiarity as I don't know what can safely be ignored).

The mlx_en.ko error, and also the error regarding /sbin/udevd lead me to pull apart the initrd image. This in turn lead me to dissect the init script, and so I found my problem. In fact there were two - the first was resolved by pulling the latest snapshot as detailed previously. The next was actually rebuilding the image with the right network modules for my hardware. With these two fixes in place (the second entirely my fault!), I have a working stateless boot image.

One question - are there any plans to tidy up error messages? They make debugging problems difficult for someone new to xcat like me. For example /sbin/udevd hasn't existed since Ubuntu 14.04, so is there a way to remove it from the build of newer Ubuntu images, or do they all share the same script (and hence the line still exists?).

Thanks again!

immarvin commented 5 years ago

ok, we will try to tidy the init script according to the latest update in linux distributions, thanks for reminder