search

flatcar / Flatcar

Flatcar project repository for issue tracking, project documentation, etc.
https://www.flatcar.org/
Apache License 2.0
679 stars 29 forks source link

Fetching files from remote URL in Ignition does not work because DNS is not up in a Firecracker VM #255

Closed mazzy89 closed 3 years ago

mazzy89 commented 4 years ago

Description

Fetching files in Ignition does not work because DNS with systemd-networkd is not set up in a Firecracker VM

Impact

I cannot fetch files from remote URL in Ignition

Environment and steps to reproduce

  1. Set-up: I'm running Flatcar from the kernel and image cooked by this PR https://github.com/flatcar-linux/coreos-overlay/pull/485 and I'm running it with Firecracker
  2. Task:
[    4.551785] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:48040->[::1]:53: read: connection refused
  1. Action(s): [ sequence of actions that triggered the bug, see example below ] a. [ requested the start of a new pod or container ] b. [ container image downloaded ]
  2. Error: [describe the error that was triggered]

Expected behavior I would have expected that DNS is ready at that point and the URL can be fetched.

Additional information

I've also tried to have as kernel flags rd.driver.pre=virtio_net to force virtio_net to be loaded.

This is the Ignition used

{
  "ignition": {
    "version": "2.2.0"
  },
  "networkd": {
    "units": [{
      "contents": "[Match]\nName=eth0\n\n[Network]\nDNS=8.8.8.8\nAddress=172.66.0.2/24\nGateway=172.66.0.1",
      "name": "00-eth0.network"
    }]
  },
  "storage": {
    "files": [{
      "filesystem": "root",
      "path": "/etc/torcx/next-profile",
      "mode": 420,
      "contents": {
        "source": "data:,docker"
      }
    }, {
      "filesystem": "root",
      "path": "/var/lib/torcx/store/docker:19.03.torcx.tgz",
      "mode": 420,
      "contents": {
        "source": "http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz",
        "verification": {
          "hash": "sha512-75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932"
        }
      }
    }, {
      "filesystem": "root",
      "path": "/etc/torcx/profiles/docker.json",
      "mode": 420,
      "contents": {
        "source": "data:,%7B%22kind%22%3A%22profile-manifest-v0%22%2C%22value%22%3A%7B%22images%22%3A%5B%7B%22name%22%3A%22docker%22%2C%22reference%22%3A%2219.03%22%7D%5D%7D%7D"
      }
    }]
  }
}
mazzy89 commented 4 years ago

The boot that can help here

[    0.000000] Linux version 4.19.132-flatcar (jenkins@jenkins-worker-2) (gcc version 8.3.0 (Gentoo Hardened 8.3.0-r1 p1.1)) kinvolk/coreos-overlay#1 SMP Fri Jul 17 17:19:48 -00 2020
[    0.000000] Command line: rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=1 rd.driver.pre=virtio_net flatcar.autologin verity.usrhash=f35bbe57b88175cba7bc85017b1d51555481cee9942ab7c2d73859df04e7d704 console=ttyS0 reboot=k panic=1 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd virtio_mmio.device=4K@0xd0000000:5 virtio_mmio.device=4K@0xd0001000:6
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 5268001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 15295401 cycles
[    0.000032] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000064] tsc: Detected 2194.848 MHz processor
[    0.000714] last_pfn = 0x40000 max_arch_pfn = 0x400000000
[    0.000991] Disabled
[    0.001014] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.001060] CPU MTRRs all blank - virtualized system.
[    0.001085] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC
[    0.002076] ACPI: Early table checksum verification disabled
[    0.002234] ACPI BIOS Error (bug): A valid RSDP was not found (20180810/tbxfroot-210)
[    0.003221] No NUMA configuration found
[    0.003223] Faking a node at [mem 0x0000000000000000-0x000000003fffffff]
[    0.003237] NODE_DATA(0) allocated [mem 0x3fffa000-0x3fffffff]
[    0.003745] Zone ranges:
[    0.003768]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.003770]   DMA32    [mem 0x0000000001000000-0x000000003fffffff]
[    0.003771]   Normal   empty
[    0.003797] Movable zone start for each node
[    0.003798] Early memory node ranges
[    0.003800]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.003801]   node   0: [mem 0x0000000000100000-0x000000003fffffff]
[    0.003845] Zeroed struct page in unavailable ranges: 98 pages
[    0.003846] Initmem setup node 0 [mem 0x0000000000001000-0x000000003fffffff]
[    0.053278] TSC deadline timer available
[    0.053380] smpboot: Boot CPU (id 0) not listed by BIOS
[    0.053392] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.053530] [mem 0x40000000-0xffffffff] available for PCI devices
[    0.053532] Booting paravirtualized kernel on KVM
[    0.053545] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.146937] random: get_random_bytes called from start_kernel+0x93/0x525 with crng_init=0
[    0.146967] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
[    0.152553] percpu: Embedded 44 pages/cpu s143128 r8192 d28904 u2097152
[    0.152657] KVM setup async PF for cpu 0
[    0.152672] kvm-stealtime: cpu 0, msr 3ea161c0
[    0.152707] Built 1 zonelists, mobility grouping on.  Total pages: 257929
[    0.152708] Policy zone: DMA32
[    0.152720] Kernel command line: rootflags=rw mount.usrflags=ro rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=1 rd.driver.pre=virtio_net flatcar.autologin verity.usrhash=f35bbe57b88175cba7bc85017b1d51555481cee9942ab7c2d73859df04e7d704 console=ttyS0 reboot=k panic=1 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd virtio_mmio.device=4K@0xd0000000:5 virtio_mmio.device=4K@0xd0001000:6
[    0.155856] Memory: 961740K/1048184K available (10252K kernel code, 1218K rwdata, 6036K rodata, 45384K init, 1760K bss, 86444K reserved, 0K cma-reserved)
[    0.156553] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.156587] ftrace: allocating 29274 entries in 115 pages
[    0.184426] rcu: Hierarchical RCU implementation.
[    0.184428] rcu:     RCU event tracing is enabled.
[    0.184439] rcu:     RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[    0.184452] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.189240] NR_IRQS: 33024, nr_irqs: 32, preallocated irqs: 16
[    0.189743] Console: colour dummy device 80x25
[    0.189863] console [tty0] enabled
[    0.321732] console [ttyS0] enabled
[    0.322843] APIC: ACPI MADT or MP tables are not detected
[    0.324437] APIC: Switch to virtual wire mode setup with no configuration
[    0.326557] Not enabling interrupt remapping due to skipped IO-APIC setup
[    0.328632] KVM setup pv IPIs
[    0.329900] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1fa33006d59, max_idle_ns: 440795304862 ns
[    0.333103] Calibrating delay loop (skipped) preset value.. 4389.69 BogoMIPS (lpj=2194848)
[    0.335093] pid_max: default: 32768 minimum: 301
[    0.337136] Security Framework initialized
[    0.338105] SELinux:  Initializing.
[    0.342602] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.346132] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.348148] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.350134] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[    0.353622] x86/cpu: Activated the Intel User Mode Instruction Prevention (UMIP) CPU feature
[    0.355650] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.357094] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.359103] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.361113] Spectre V2 : Mitigation: Enhanced IBRS
[    0.362095] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.364103] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.366095] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[    0.380345] Freeing SMP alternatives memory: 28K
[    0.384643] smpboot: SMP disabled
[    0.386346] Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only.
[    0.388451] rcu: Hierarchical SRCU implementation.
[    0.391648] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.393300] smp: Bringing up secondary CPUs ...
[    0.395096] smp: Brought up 1 node, 1 CPU
[    0.396094] smpboot: Max logical packages: 1
[    0.398095] smpboot: Total of 1 processors activated (4389.69 BogoMIPS)
[    0.400533] devtmpfs: initialized
[    0.401176] x86/mm: Memory block size: 128MB
[    0.404188] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.406098] futex hash table entries: 256 (order: 2, 16384 bytes)
[    0.408208] pinctrl core: initialized pinctrl subsystem
[    0.410471] NET: Registered protocol family 16
[    0.412258] audit: initializing netlink subsys (disabled)
[    0.414373] cpuidle: using governor menu
[    0.415239] KVM setup pv remote TLB flush
[    0.417293] dca service started, version 1.12.1
[    0.421096] PCI: Fatal: No config space access function found
[    0.423191] audit: type=2000 audit(1595425154.445:1): state=initialized audit_enabled=0 res=1
[    0.426797] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.429974] ACPI: Interpreter disabled.
[    0.431173] vgaarb: loaded
[    0.432105] pps_core: LinuxPPS API ver. 1 registered
[    0.433105] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.435098] PTP clock support registered
[    0.437108] PCI: System does not support PCI
[    0.439583] clocksource: Switched to clocksource kvm-clock
[    0.471340] VFS: Disk quotas dquot_6.6.0
[    0.472722] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.475072] pnp: PnP ACPI: disabled
[    0.478650] NET: Registered protocol family 2
[    0.480438] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes)
[    0.482945] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[    0.485403] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[    0.487912] TCP: Hash tables configured (established 8192 bind 8192)
[    0.490554] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.492509] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.494695] NET: Registered protocol family 1
[    0.496133] NET: Registered protocol family 44
[    0.590152] random: fast init done
[    1.722689] virtio-mmio: Registering device virtio-mmio.0 at 0xd0000000-0xd0000fff, IRQ 5.
[    1.725966] virtio-mmio: Registering device virtio-mmio.1 at 0xd0001000-0xd0001fff, IRQ 6.
[    1.729235] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1fa33006d59, max_idle_ns: 440795304862 ns
[    1.732784] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    1.736386] Initialise system trusted keyrings
[    1.738047] workingset: timestamp_bits=39 max_order=18 bucket_order=0
[    1.991924] Key type asymmetric registered
[    1.993461] Asymmetric key parser 'x509' registered
[    1.995247] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    1.997893] io scheduler noop registered
[    1.999304] io scheduler deadline registered
[    2.000897] io scheduler cfq registered (default)
[    2.002585] io scheduler mq-deadline registered
[    2.004230] io scheduler kyber registered
[    2.005708] io scheduler bfq registered
[    2.007458] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    2.009683] virtio-mmio virtio-mmio.0: Failed to enable 64-bit or 32-bit DMA.  Trying to continue, but this might not work.
[    2.013640] virtio-mmio virtio-mmio.1: Failed to enable 64-bit or 32-bit DMA.  Trying to continue, but this might not work.
[    2.017534] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.044654] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a U6_16550A
[    2.072504] serial8250: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a U6_16550A
[    2.100087] serial8250: ttyS2 at I/O 0x3e8 (irq = 4, base_baud = 115200) is a U6_16550A
[    2.102928] i8042: PNP detection disabled
[    2.104695] serio: i8042 KBD port at 0x60,0x64 irq 1
[    2.106327] input: AT Raw Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
[    2.109056] rtc_cmos rtc_cmos: only 24-hr supported
[    2.110807] NET: Registered protocol family 10
[    2.112615] Segment Routing with IPv6
[    2.113829] NET: Registered protocol family 17
[    2.115235] Key type dns_resolver registered
[    2.116805] mce: Using 32 MCE banks
[    2.117933] sched_clock: Marking stable (1983449335, 134452350)->(2135166384, -17264699)
[    2.120537] registered taskstats version 1
[    2.121842] Loading compiled-in X.509 certificates
[    2.156653] Loaded X.509 cert 'Kinvolk GmbH: Module signing key for 4.19.132-flatcar: f23420621d11e00c6ec5978e3e304b5ddc755a2c'
[    2.160154] ima: No TPM chip found, activating TPM-bypass!
[    2.161854] ima: Allocated hash algorithm: sha1
[    2.163583] hctosys: unable to open rtc device (rtc0)
[    2.172981] Freeing unused kernel image memory: 45384K
[    2.176183] Write protecting the kernel read-only data: 18432k
[    2.184832] Freeing unused kernel image memory: 2016K
[    2.186802] Freeing unused kernel image memory: 108K
[    2.188332] Run /init as init process
[    2.195981] systemd[1]: systemd 241-53-gbe3cc54+ running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS -ACL +XZ +LZ4 +SECCOMP +BLKID -ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
[    2.202659] systemd[1]: Detected virtualization kvm.
[    2.204224] systemd[1]: Detected architecture x86-64.
[    2.205802] systemd[1]: Running in initial RAM disk.

Welcome to dracut-049 (Initramfs)!

[    2.211902] systemd[1]: No hostname configured.
[    2.213367] systemd[1]: Set hostname to <localhost>.
[    2.214950] systemd[1]: Initializing machine ID from random generator.
[    2.217210] systemd[1]: Failed to bump fs.file-max, ignoring: Invalid argument
[    2.332507] systemd[1]: File /usr/lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    2.337799] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[  OK  ] Listening on Journal Audit Socket.
[  OK  ] Listening on udev Control Socket.
[  OK  ] Reached target Swap.
[  OK  ] Created slice system-systemd\x2dfsck.slice.
[  OK  ] Reached target Timers.
[  OK  ] Listening on udev Kernel Socket.
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Reached target Local Encrypted Volumes.
[  OK  ] Listening on Journal Socket.
         Starting Apply Kernel Variables...
         Starting Setup Virtual Console...
[  OK  ] Reached target Sockets.
         Starting Create list of re…odes for the current kernel...
         Starting Journal Service...
[  OK  ] Reached target Paths.
[  OK  ] Reached target Slices.
[  OK  ] Started Apply Kernel Variables.
[  OK  ] Started Setup Virtual Console.
[  OK  ] Started Create list of req… nodes for the current kernel.
         Starting Create Static Device Nodes in /dev...
         Starting dracut cmdline hook...
[  OK  ] Started Create Static Device Nodes in /dev.
[  OK  ] Started dracut cmdline hook.
         Starting dracut pre-udev hook...
[    2.622540] device-mapper: uevent: version 1.0.3
[    2.625229] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com
[  OK  ] Started dracut pre-udev hook.
         Starting udev Kernel Device Manager...
[  OK  ] Started Journal Service.
[    2.945729] audit: type=1130 audit(1595425156.976:2): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-journald comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.817854] systemd-sysctl[146]: Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory
[    2.825276] systemd-vconsole-setup[147]: KD_FONT_OP_GET failed while trying to get the font metadata: Function not implemented
[    2.833956] systemd-vconsole-setup[147]: Fonts will not be copied to remaining consoles
[    2.837429] dracut-cmdline[155]: dracut-dracut-049
[    2.841777] dracut-cmdline[155]: Using kernel command line parameters: rootflags=rw mount.usrflags=ro rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=1 rd.driver.pre=virtio_net flatcar.autologin verity.usrhash=f35bbe57b88175cba7bc85017b1d51555481cee9942ab7c2d73859df04e7d704 console=ttyS0 reboot=k panic=1 i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd virtio_mmio.device=4K@0xd0000000:5 virtio_mmio.device=4K@0xd0001000:6
[    2.962228] systemd-udevd[251]: Specified group 'render' unknown
[  OK  ] Started udev Kernel Device Manager.[    2.968795] systemd[1]: Started udev Kernel Device Manager.
[    3.105481] audit: type=1130 audit(1595425157.136:3): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udevd comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'

         [    2.979116] systemd[1]: Starting Network Service...Starting Network Service...

         [    2.990113] systemd[1]: Starting dracut pre-trigger hook...Starting dracut pre-trigger hook...

[    3.012416] systemd-udevd[260]: Using default interface naming scheme 'v240'.
[    3.015469] systemd-udevd[260]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
[    3.042002] dracut-pre-trigger[258]: rd.md=0: removing MD RAID activation
[  OK  ] Started dracut pre-trigger hook.[    3.122067] systemd[1]: Started dracut pre-trigger hook.

[    3.259064] audit: type=1130 audit(1595425157.289:4): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-trigger comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.131783] systemd[1]: Starting udev Coldplug all Devices...         Starting udev Coldplug all Devices...

[    3.151760] systemd-udevd[251]: Specified group 'render' unknown
[  OK  ] Started udev Coldplug all Devices.[    3.202211] systemd[1]: Started udev Coldplug all Devices.

[    3.339245] audit: type=1130 audit(1595425157.370:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.212025] systemd[1]: Starting dracut initqueue hook...         Starting dracut initqueue hook...

[    3.299381] systemd-udevd[301]: Using default interface naming scheme 'v240'.
[    3.302296] systemd-udevd[301]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
[    3.448017] cryptd: max_cpu_qlen set to 1000
[    3.459262] virtio_blk virtio0: [vda] 20971520 512-byte logical blocks (10.7 GB/10.0 GiB)
[    3.333309] systemd-udevd[298]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
[    3.520611] GPT:Primary header thinks Alt. header is not at the end of the disk.
[    3.522970] GPT:9289727 != 20971519
[    3.524117] GPT:Alternate GPT header not at the end of the disk.
[    3.526006] GPT:9289727 != 20971519
[    3.527116] GPT: Use GNU Parted to correct GPT errors.
[    3.528778]  vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9
[    3.545947] AVX2 version of gcm_enc/dec engaged.
[    3.547511] AES CTR mode by8 optimization enabled
[    3.453582] systemd-vconsole-setup[319]: KD_FONT_OP_GET failed while trying to get the font metadata: Function not implemented
[    3.457946] systemd-vconsole-setup[319]: Fonts will not be copied to remaining consoles
[    3.588950] systemd[1]: Found device /dev/disk/by-partuuid/7130c94a-213a-4e5a-8e26-6cce9662f132.
[  OK  ] Found device /dev/disk/by-…a-213a-4e5a-8e26-6cce9662f132.
[    3.623472] systemd[1]: Found device /dev/disk/by-label/ROOT.
[  OK  ] Found device /dev/disk/by-label/ROOT.
[    3.648603] systemd[1]: Found device /dev/disk/by-label/OEM.
[  OK  ] Found device /dev/disk/by-label/OEM.
[    3.662423] systemd[1]: Started dracut initqueue hook.
[  OK  ] Started dracut initqueue hook.
[    3.806387] audit: type=1130 audit(1595425157.837:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.693162] systemd[1]: Starting dracut pre-mount hook...
         Starting dracut pre-mount hook...
[    3.703812] systemd[1]: Reached target Remote File Systems (Pre).
[  OK  [    3.708772] systemd-networkd[252]: Enumeration completed
] Reached target Remote File Systems (Pre).
[    3.713877] systemd[1]: Reached target Remote File Systems.[  OK
] Reached target Remote File Systems.[    3.721342] systemd-networkd[252]: lo: Configured

[    3.726483]          systemd-networkd[252]: Starting Ignition (setup)...eth0: Gained carrier
[    3.731568]
[  OK  systemd[1]: Starting Ignition (setup)...] Reached target Initrd Root Device.
[    3.745034] systemd
[  OK  [1]: Reached target Initrd Root Device.] Started Network Service.
[    3.756527] systemd[1]: Started Network Service.
[    3.893747] audit: type=1130 audit(1595425157.924:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-networkd comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  OK
[    3.769517] ] Started dracut pre-mount hook.systemd[1]: Started dracut pre-mount hook.

         Starting Network Name Resolution...[    3.779571] systemd[1]: Starting Network Name Resolution...[    3.915996] audit: type=1130 audit(1595425157.941:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-mount comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'

[    3.979506] EXT4-fs (vda6): mounted filesystem with ordered data mode. Opts: (null)
[    3.861945] systemd[1]: mnt-oem.mount: Succeeded.
[  OK  ] Started Ignition (setup).[    3.872339] systemd[1]: ignition-setup.service: Succeeded.
[    3.874937] systemd
[    4.010658] audit: type=1130 audit(1595425158.041:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=ignition-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[1]: Started Ignition (setup).
[    4.028178] audit: type=1131 audit(1595425158.048:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=ignition-setup comm="systemd" exe="/usr/lib64/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    4.100376] systemd-resolved[373]: Positive Trust Anchors:
[    4.103327] systemd-resolved[373]: . IN DS 19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
[    4.109778] systemd-resolved[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Host and Network Name Lookups.[373]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
[  OK  ] Reached target Network.
[    4.122206] systemd-resolved[373]:
         Negative trust anchors: 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private testStarting Ignition (disks)...

[    4.141767] systemd-resolved[373]: Defaulting to hostname 'linux'.
[    4.148817] ignition[384]: Ignition v0.34.0-22-g032f620[  OK  ] Started Ignition (disks).
         Starting Verity Setup for /dev/mapper/usr...
[    4.158573] systemd[1]: Started Network Name Resolution.
[    4.166398]
[  OK  ] Reached target Local File Systems (Pre).ignition[384]: Stage: disks[    4.311412] random: veritysetup: uninitialized urandom read (2 bytes read)

[    4.183333] systemd         [1]: Reached target Host and Network Name Lookups.Starting File System Check on /dev/disk/by-label/ROOT...
[    4.192919] ignition
[384]: reading system config file "/usr/lib/ignition/base.ign"
[    4.204274] systemd[1]: Reached target Network.
[    4.208595] ignition[384]: no config at "/usr/lib/ignition/base.ign"
[    4.223419] systemd[1]: [  OK  ] Started File System Check on /dev/disk/by-label/ROOT.
         Starting Ignition (disks)...
Mounting /sysroot...[    4.236932] systemd
[1]: ignition-disks.service: Succeeded.
[    4.247226] ignition[384]: no config URL provided
[    4.256002] systemd-fsck[392]: ROOT: clean, 577/553792 files, 37492/553472 blocks
[    4.261574] systemd[1]: [    4.398420] EXT4-fs (vda9): mounted filesystem with ordered data mode. Opts: (null)
Started Ignition (disks).
[    4.275462] [  OK  ignition[384]: reading system config file "/usr/lib/ignition/user.ign"] Mounted /sysroot.
[  OK
[    4.287143] ] Reached target Initrd Root File System.systemd[1]: Starting Verity Setup for /dev/mapper/usr...

[    4.301153] systemd[1]: Reached target Local File Systems (Pre).
[    4.307145] ignition[384]: disks: disks passed
[    4.311212] systemd[1]: Starting File System Check on /dev/disk/by-label/ROOT...
[    4.316040] ignition[384]: Ignition finished successfully
[    4.325420] systemd[1]: Started File System Check on /dev/disk/by-label/ROOT.
[    4.335775] systemd[1]: Mounting /sysroot...
[    4.339816] systemd[1]: Mounted /sysroot.
[    4.343816] systemd[  OK  ] Started Verity Setup for /dev/mapper/usr.
[1]: Reached target Initrd Root File System.
[    4.357190] systemd[1]: Started Verity Setup for /dev/mapper/usr.[  OK  ] Found device /dev/mapper/usr.
[    4.370467]
systemd[1]:          Found device /dev/mapper/usr.
Mounting /sysroot/usr...[    4.377257] systemd[1]:
Condition check resulted in File System Check on /dev/mapper/usr being skipped.
[    4.384263] systemd[1]: Mounting /sysroot/usr...
[    4.538430] EXT4-fs (dm-0): mounted filesystem without journal. Opts: (null)
[    4.407548] systemd[1]: Mounted /sysroot/usr.
[  OK  ] Mounted /sysroot/usr.
[    4.414355] systemd[1]: Starting Root filesystem setup...
         Starting Root filesystem setup...
[    4.420806] systemd[1]: Reached target Local File Systems.
[  OK  ] Reached target Local File Systems.
[    4.431510] systemd[1]: Reached target System Initialization.
[  OK  ] Reached target System Initialization.
[    4.443231] systemd[1]: Reached target Basic System.
[  OK  ] Reached target Basic System.
[  OK  ] Started Root filesystem setup.[    4.496332] systemd[1]: initrd-setup-root.service: Succeeded.
[    4.499014] systemd
[1]: Started Root filesystem setup.         Starting Ignition (files)...
[    4.504362] systemd[1]: Starting Ignition (files)...

[    4.513451] ignition[424]: Ignition v0.34.0-22-g032f620
[    4.521152] ignition[424]: Stage: files
[    4.523482] ignition[424]: reading system config file "/usr/lib/ignition/base.ign"
[    4.526629] ignition[424]: no config at "/usr/lib/ignition/base.ign"
[    4.529378] ignition[424]: files: createFilesystemsFiles: createFiles: op(1): [started]  writing file "/etc/torcx/next-profile"
[    4.533761] ignition[424]: files: createFilesystemsFiles: createFiles: op(1): [finished] writing file "/etc/torcx/next-profile"
[    4.538068] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): [started]  writing file "/var/lib/torcx/store/docker:19.03.torcx.tgz"
[    4.542870] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/coreos-overlay#1
[    4.551785] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:48040->[::1]:53: read: connection refused
[    4.720492] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/Flatcar#253
[    4.730015] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:45337->[::1]:53: read: connection refused
[    5.121507] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/coreos-overlay#3
[    5.130004] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:50614->[::1]:53: read: connection refused
[    5.663877] systemd-networkd[252]: eth0: Gained IPv6LL
[    5.922871] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/coreos-overlay#4
[    5.931029] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:36820->[::1]:53: read: connection refused
[    7.523359] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/coreos-overlay#5
[    7.532029] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:48373->[::1]:53: read: connection refused
[**    ] A start job is running for Ignition (files) (8s / no limit)
[   10.724848] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: attempt kinvolk/coreos-overlay#6
[   10.732913] ignition[424]: files: createFilesystemsFiles: createFiles: op(2): GET error: Get http://storage-download.googleapis.com/flatcar-jenkins/torcx/pkgs/amd64-usr/docker/75a121853333c5a7b242cf4bd713210f8cf65fd418b2c7fb7a9cf18f3514f3d9d24141fef22a5119d4803f72e05395f7a8968088192d1c3bb42e91cd058bd932/docker:19.03.torcx.tgz: dial tcp: lookup storage-download.googleapis.com on [::1]:53: read udp [::1]:52230->[::1]:53: read: connection refus[    **] A start job is running for Ignition (files) (11s / no limit)
[  *** ] A start job is running for Ignition (files) (13s / no limit)

and it shows that

[    3.726483]          systemd-networkd[252]: Starting Ignition (setup)...eth0: Gained carrier

is executed before the files are fetched so I do not get why the DNS is not set it up at that point.

pothos commented 4 years ago

The networkd config you created in Ignition is not applied during the initramfs but only later. If you don't have a DHCP server, you can provide a static IP address configuration with these parameters https://docs.flatcar-linux.org/ignition/network-configuration/#using-static-ip-addresses-with-ignition ip=ADDR::GATEWAY:255.255.255.0::eth0:none:8.8.8.8:8.8.4.4 (Address without a /24)

mazzy89 commented 4 years ago

I see 😊 thanks a lot Kai as always

mazzy89 commented 4 years ago

Tested. We can close this. it works as expected also on Firecracker.