[server panic] stopmaster

woonjjang commented 1 year ago

타겟

Host stopmaster HostName ec2-3-34-25-166.ap-northeast-2.compute.amazonaws.com User ubuntu IdentityFile ~/.ssh/stopmaster-key.pem

서버, db 관련 취약점 작업들을 진행함.
서버 리부트가 필요해서 진행 / 리부트 후 AWS console 및 서버가 제대로 구성되지 않는 것으로 보임
ssh 접근이 어려움

해결하려고 시도해 본 것

console에서 관련 서버 이미지 생성되는 로그를 "EC2 직렬 콘솔"로 볼 수 밖에 없는데, id /pw 부분에서 막임
key로 로그인을 했어서, 직접 로그인 하는 방법이 없음
접근만 하면 리부트 시켜보면서 시도할 수 있는데 방법을 찾지 못함.
인스턴트 오류시 보통 생기는 문제에 관해서
- https://docs.aws.amazon.com/ko_kr/AWSEC2/latest/UserGuide/instance-console.html

woonjjang commented 1 year ago

[  OK  ] Finished Initial cloud-init job (pre-networking).
[  OK  ] Reached target Network (Pre).
         Starting Network Service...
[  OK  ] Started Network Service.
         Starting Wait for Network to be Configured...
         Starting Network Name Resolution...
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Network.
[  OK  ] Reached target Host and Network Name Lookups.
[ ***  ] A start job is running for Wait for… to be Configured (19s / no limit)

마지막 상태에서 멈춰있는 상태가 지속됩니다. ec2 리부트하고 보고 있을 때

woonjjang commented 1 year ago

인스턴스 삭제한거 아니몀 호스트가 바꼈을수도 있어요

[  OK  ] Started Dispatcher daemon for systemd-networkd.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots.
[  OK  ] Finished EC2 Instance Connect Host Key Harvesting.
[  OK  ] Started Network Time Service.
[  OK  ] Finished Permit User Sessions.
[  OK  ] Started The PHP 7.3 FastCGI Process Manager.
         Starting Hold until boot process finishes up...
         Starting Terminate Plymouth Boot Screen...
         Starting OpenBSD Secure Shell server...
[  OK  ] Started System Logging Service.
[  OK  ] Finished Record successful boot for GRUB.
[  OK  ] Finished Hold until boot process finishes up.
[  OK  ] Finished Terminate Plymouth Boot Screen.
[  OK  ] Started Login Service.
[  OK  ] Started Accounts Service.
[  OK  ] Started OpenBSD Secure Shell server.
[  OK  ] Started LSB: automatic crash report generation.
[  OK  ] Finished Clean php session files.
[  OK  ] Started Authorization Manager.
[  OK  ] Started Disk Manager.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
         Starting GRUB failed boot detection...
         Starting Modem Manager...
[  OK  ] Started Serial Getty on ttyS0.
         Starting Set console scheme...
[  OK  ] Started Unattended Upgrades Shutdown.
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Finished Set console scheme.
[  OK  ] Created slice system-getty.slice.
[  OK  ] Started Getty on tty1.
[  OK  ] Reached target Login Prompts.
[  OK  ] Finished GRUB failed boot detection.
[  OK  ] Started Modem Manager.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Started MariaDB 10.3.38 database server.
[  OK  ] Stopped Deferred execution scheduler.
[FAILED] Failed to start Deferred execution scheduler.
See 'systemctl status atd.service' for details.

Ubuntu 20.04.5 LTS ip-10-50-11-11 ttyS0

ip-10-50-11-11 login: [  152.729958] cloud-init[974]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:config' at Tue, 14 Mar 2023 12:41:09 +0000. Up 152.63 seconds.
[  153.425799] cloud-init[981]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:final' at Tue, 14 Mar 2023 12:41:09 +0000. Up 153.25 seconds.
[  153.428334] cloud-init[981]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 finished at Tue, 14 Mar 2023 12:41:09 +0000. Datasource DataSourceEc2Local.  Up 153.42 seconds

마지막 로그 내용

woonjjang commented 1 year ago

현재는 접근이 되어야 서버 내부에서 해당 파일을 수정해서 reboot 하거나 할 수 있는데 접근이 안 돼는 상황

https://docs.aws.amazon.com/ko_kr/AWSEC2/latest/UserGuide/connect-to-serial-console.html#sc-connect-SSH

woonjjang commented 1 year ago

[ 4919.911759] reboot: Power down
[    0.000000] Linux version 5.15.0-1031-aws (buildd@lcy02-amd64-045) (gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #35~20.04.1-Ubuntu SMP Sat Feb 11 16:19:06 UTC 2023 (Ubuntu 5.15.0-1031.35~20.04.1-aws 5.15.85)
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-1031-aws root=PARTUUID=093eb684-01 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 panic=-1
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Hygon HygonGenuine
[    0.000000]   Centaur CentaurHauls
[    0.000000]   zhaoxin   Shanghai  
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x008: 'MPX bounds registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x010: 'MPX CSR'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[3]:  832, xstate_sizes[3]:   64
[    0.000000] x86/fpu: xstate_offset[4]:  896, xstate_sizes[4]:   64
[    0.000000] x86/fpu: xstate_offset[5]:  960, xstate_sizes[5]:   64
[    0.000000] x86/fpu: xstate_offset[6]: 1024, xstate_sizes[6]:  512
[    0.000000] x86/fpu: xstate_offset[7]: 1536, xstate_sizes[7]: 1024
[    0.000000] x86/fpu: xstate_offset[9]: 2560, xstate_sizes[9]:    8
[    0.000000] x86/fpu: Enabled xstate features 0x2ff, context size is 2568 bytes, using 'compacted' format.
[    0.000000] signal: max sigframe size: 3632
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003e3e9fff] usable
[    0.000000] BIOS-e820: [mem 0x000000003e3ea000-0x000000003fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000e03fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: Amazon EC2 t3.micro/, BIOS 1.0 10/16/2017
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 2c001001, primary cpu clock
[    0.000002] kvm-clock: using sched offset of 11915862885 cycles
[    0.000004] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000007] tsc: Detected 2499.998 MHz processor
[    0.000537] last_pfn = 0x3e3ea max_arch_pfn = 0x400000000
[    0.000583] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.007924] Using GB pages for direct mapping
[    0.008057] RAMDISK: [mem 0x372b3000-0x37950fff]
[    0.008071] ACPI: Early table checksum verification disabled
[    0.008079] ACPI: RSDP 0x00000000000F8F40 000014 (v00 AMAZON)
[    0.008085] ACPI: RSDT 0x000000003E3EE350 000044 (v01 AMAZON AMZNRSDT 00000001 AMZN 00000001)
[    0.008091] ACPI: FACP 0x000000003E3EFF80 000074 (v01 AMAZON AMZNFACP 00000001 AMZN 00000001)
[    0.008097] ACPI: DSDT 0x000000003E3EE3A0 0010E9 (v01 AMAZON AMZNDSDT 00000001 AMZN 00000001)
[    0.008100] ACPI: FACS 0x000000003E3EFF40 000040
[    0.008104] ACPI: SSDT 0x000000003E3EF6C0 00087A (v01 AMAZON AMZNSSDT 00000001 AMZN 00000001)
[    0.008108] ACPI: APIC 0x000000003E3EF5D0 000076 (v01 AMAZON AMZNAPIC 00000001 AMZN 00000001)
[    0.008112] ACPI: SRAT 0x000000003E3EF530 0000A0 (v01 AMAZON AMZNSRAT 00000001 AMZN 00000001)
[    0.008115] ACPI: SLIT 0x000000003E3EF4C0 00006C (v01 AMAZON AMZNSLIT 00000001 AMZN 00000001)
[    0.008119] ACPI: WAET 0x000000003E3EF490 000028 (v01 AMAZON AMZNWAET 00000001 AMZN 00000001)
[    0.008124] ACPI: HPET 0x00000000000C9000 000038 (v01 AMAZON AMZNHPET 00000001 AMZN 00000001)
[    0.008128] ACPI: SSDT 0x00000000000C9040 00007B (v01 AMAZON AMZNSSDT 00000001 AMZN 00000001)
[    0.008131] ACPI: Reserving FACP table memory at [mem 0x3e3eff80-0x3e3efff3]
[    0.008132] ACPI: Reserving DSDT table memory at [mem 0x3e3ee3a0-0x3e3ef488]
[    0.008134] ACPI: Reserving FACS table memory at [mem 0x3e3eff40-0x3e3eff7f]
[    0.008135] ACPI: Reserving SSDT table memory at [mem 0x3e3ef6c0-0x3e3eff39]
[    0.008136] ACPI: Reserving APIC table memory at [mem 0x3e3ef5d0-0x3e3ef645]
[    0.008137] ACPI: Reserving SRAT table memory at [mem 0x3e3ef530-0x3e3ef5cf]
[    0.008138] ACPI: Reserving SLIT table memory at [mem 0x3e3ef4c0-0x3e3ef52b]
[    0.008139] ACPI: Reserving WAET table memory at [mem 0x3e3ef490-0x3e3ef4b7]
[    0.008141] ACPI: Reserving HPET table memory at [mem 0xc9000-0xc9037]
[    0.008142] ACPI: Reserving SSDT table memory at [mem 0xc9040-0xc90ba]
[    0.008259] SRAT: PXM 0 -> APIC 0x00 -> Node 0
[    0.008262] SRAT: PXM 0 -> APIC 0x01 -> Node 0
[    0.008266] ACPI: SRAT: Node 0 PXM 0 [mem 0x00000000-0x3fffffff]
[    0.008289] NODE_DATA(0) allocated [mem 0x3e3bf000-0x3e3e8fff]
[    0.008790] Zone ranges:
[    0.008791]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.008794]   DMA32    [mem 0x0000000001000000-0x000000003e3e9fff]
[    0.008796]   Normal   empty
[    0.008797]   Device   empty
[    0.008798] Movable zone start for each node
[    0.008801] Early memory node ranges
[    0.008802]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.008804]   node   0: [mem 0x0000000000100000-0x000000003e3e9fff]
[    0.008806] Initmem setup node 0 [mem 0x0000000000001000-0x000000003e3e9fff]
[    0.008813] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.008842] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.010653] On node 0, zone DMA32: 7190 pages in unavailable ranges
[    0.011099] ACPI: PM-Timer IO Port: 0xb008
[    0.011111] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.011175] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.011178] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.011181] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.011183] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.011184] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.011188] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.011190] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.011193] TSC deadline timer available
[    0.011195] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[    0.011222] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.011225] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.011226] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.011228] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.011230] [mem 0x40000000-0xdfffffff] available for PCI devices
[    0.011232] Booting paravirtualized kernel on KVM
[    0.011234] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.011241] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[    0.011576] percpu: Embedded 60 pages/cpu s208896 r8192 d28672 u1048576
[    0.011609] kvm-guest: stealtime: cpu 0, msr 3d032080
[    0.011613] kvm-guest: PV spinlocks enabled
[    0.011615] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.011625] Built 1 zonelists, mobility grouping on.  Total pages: 250714
[    0.011627] Policy zone: DMA32
[    0.011629] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.15.0-1031-aws root=PARTUUID=093eb684-01 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 panic=-1
[    0.011701] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-5.15.0-1031-aws", will be passed to user space.
[    0.011850] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.011926] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.011968] mem auto-init: stack:off, heap alloc:on, heap free:off
[    0.014222] Memory: 948008K/1019424K available (16393K kernel code, 4382K rwdata, 10852K rodata, 3240K init, 6560K bss, 71156K reserved, 0K cma-reserved)
[    0.014516] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.014531] Kernel/User page tables isolation: enabled
[    0.014545] ftrace: allocating 50241 entries in 197 pages
[    0.046691] ftrace: allocated 197 pages with 4 groups
[    0.046833] rcu: Hierarchical RCU implementation.
[    0.046835] rcu:     RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=2.
[    0.046837]  Rude variant of Tasks RCU enabled.
[    0.046838]  Tracing variant of Tasks RCU enabled.
[    0.046839] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.046840] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.051116] NR_IRQS: 524544, nr_irqs: 440, preallocated irqs: 16
[    0.051309] random: crng init done
[    0.196425] Console: colour VGA+ 80x25
[    0.641130] printk: console [tty1] enabled
[    4.639206] printk: console [ttyS0] enabled
[    4.642885] ACPI: Core revision 20210730
[    4.646593] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 30580167144 ns
[    4.744092] APIC: Switch to symmetric I/O mode setup
[    4.748549] x2apic enabled
[    4.751976] Switched APIC routing to physical x2apic.
[    4.757947] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x240937b9988, max_idle_ns: 440795218083 ns
[    4.844870] Calibrating delay loop (skipped) preset value.. 4999.99 BogoMIPS (lpj=9999992)
[    4.848865] pid_max: default: 32768 minimum: 301
[    4.848865] LSM: Security Framework initializing
[    4.848865] landlock: Up and running.
[    4.848865] Yama: becoming mindful.
[    4.848865] AppArmor: AppArmor initialized
[    4.848865] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    4.848865] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    4.848865] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    4.848865] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    4.848865] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    4.848865] Spectre V2 : Mitigation: Retpolines
[    4.848865] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    4.848865] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[    4.848865] RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to RETBleed attacks, data leaks possible!
[    4.848865] RETBleed: Vulnerable
[    4.848865] Speculative Store Bypass: Vulnerable
[    4.848865] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    4.848865] MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
[    4.848865] Freeing SMP alternatives memory: 40K
[    4.848865] smpboot: CPU0: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz (family: 0x6, model: 0x55, stepping: 0x7)
[    4.849044] Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only.
[    4.852924] rcu: Hierarchical SRCU implementation.
[    4.857314] NMI watchdog: Perf NMI watchdog permanently disabled
[    4.860936] smp: Bringing up secondary CPUs ...
[    4.865016] x86: Booting SMP configuration:
[    4.868618] .... node  #0, CPUs:      #1
[    4.797564] kvm-clock: cpu 1, msr 2c001041, secondary cpu clock
[    4.870816] kvm-guest: stealtime: cpu 1, msr 3d132080
[    4.960954] MDS CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/mds.html for more details.
[    4.964871] MMIO Stale Data CPU bug present and SMT on, data leak possible. See https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/processor_mmio_stale_data.html for more details.
[    4.968905] smp: Brought up 1 node, 2 CPUs
[    4.972529] smpboot: Max logical packages: 1
[    4.972869] smpboot: Total of 2 processors activated (9999.99 BogoMIPS)
[    4.977104] devtmpfs: initialized
[    4.977104] x86/mm: Memory block size: 128MB
[    4.984937] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    4.988878] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[    4.992931] pinctrl core: initialized pinctrl subsystem
[    4.997146] PM: RTC time: 14:20:57, date: 2023-03-14
[    5.001019] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    5.004991] DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations
[    5.008952] DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    5.012891] DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    5.016884] audit: initializing netlink subsys (disabled)
[    5.020905] audit: type=2000 audit(1678803651.467:1): state=initialized audit_enabled=0 res=1
[    5.021065] thermal_sys: Registered thermal governor 'fair_share'
[    5.024865] thermal_sys: Registered thermal governor 'bang_bang'
[    5.108873] thermal_sys: Registered thermal governor 'step_wise'
[    5.112868] thermal_sys: Registered thermal governor 'user_space'
[    5.116868] thermal_sys: Registered thermal governor 'power_allocator'
[    5.208886] EISA bus registered
[    5.216900] cpuidle: using governor ladder
[    5.220874] cpuidle: using governor menu
[    5.224448] ACPI: bus type PCI registered
[    5.224872] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    5.229015] PCI: Using configuration type 1 for base access
[    5.234173] Kprobes globally optimized
[    5.236918] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    5.320879] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    5.328980] ACPI: Added _OSI(Module Device)
[    5.332870] ACPI: Added _OSI(Processor Device)
[    5.336590] ACPI: Added _OSI(3.0 _SCP Extensions)
[    5.336870] ACPI: Added _OSI(Processor Aggregator Device)
[    5.340869] ACPI: Added _OSI(Linux-Dell-Video)
[    5.344560] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    5.344868] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    5.350239] ACPI: 3 ACPI AML tables successfully acquired and loaded
[    5.353940] ACPI: Interpreter enabled
[    5.356887] ACPI: PM: (supports S0 S4 S5)
[    5.360424] ACPI: Using IOAPIC for interrupt routing
[    5.360886] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    5.364869] PCI: Using E820 reservations for host bridge windows
[    5.369236] ACPI: Enabled 16 GPEs in block 00 to 0F
[    5.375466] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    5.376878] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[    5.380881] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    5.385245] acpiphp: Slot [3] registered
[    5.388758] acpiphp: Slot [4] registered
[    5.388885] acpiphp: Slot [5] registered
[    5.392398] acpiphp: Slot [6] registered
[    5.392884] acpiphp: Slot [7] registered
[    5.396894] acpiphp: Slot [8] registered
[    5.400365] acpiphp: Slot [9] registered
[    5.400883] acpiphp: Slot [10] registered
[    5.404371] acpiphp: Slot [11] registered
[    5.404884] acpiphp: Slot [12] registered
[    5.408891] acpiphp: Slot [13] registered
[    5.412522] acpiphp: Slot [14] registered
[    5.412884] acpiphp: Slot [15] registered
[    5.416419] acpiphp: Slot [16] registered
[    5.416889] acpiphp: Slot [17] registered
[    5.420669] acpiphp: Slot [18] registered
[    5.420891] acpiphp: Slot [19] registered
[    5.424481] acpiphp: Slot [20] registered
[    5.424884] acpiphp: Slot [21] registered
[    5.428410] acpiphp: Slot [22] registered
[    5.428888] acpiphp: Slot [23] registered
[    5.432896] acpiphp: Slot [24] registered
[    5.436500] acpiphp: Slot [25] registered
[    5.436884] acpiphp: Slot [26] registered
[    5.440459] acpiphp: Slot [27] registered
[    5.440884] acpiphp: Slot [28] registered
[    5.444894] acpiphp: Slot [29] registered
[    5.448524] acpiphp: Slot [30] registered
[    5.448884] acpiphp: Slot [31] registered
[    5.452439] PCI host bridge to bus 0000:00
[    5.452870] pci_bus 0000:00: root bus resource [bus 00-ff]
[    5.456870] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    5.460868] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    5.464868] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    5.468872] pci_bus 0000:00: root bus resource [mem 0x80000000-0xfebfffff window]
[    5.472968] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[    5.477712] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[    5.482075] pci 0000:00:01.3: [8086:7113] type 00 class 0x000000
[    5.485826] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    5.488885] pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
[    5.492929] pci 0000:00:01.3: PIIX4 devres E PIO at fff0-ffff
[    5.496884] pci 0000:00:01.3: PIIX4 devres F MMIO at ffc00000-ffffffff
[    5.500882] pci 0000:00:01.3: PIIX4 devres G PIO at fff0-ffff
[    5.504884] pci 0000:00:01.3: PIIX4 devres H MMIO at ffc00000-ffffffff
[    5.508887] pci 0000:00:01.3: PIIX4 devres I PIO at fff0-ffff
[    5.512882] pci 0000:00:01.3: PIIX4 devres J PIO at fff0-ffff
[    5.516868] pci 0000:00:01.3: quirk_piix4_acpi+0x0/0x180 took 31250 usecs
[    5.521803] pci 0000:00:03.0: [1d0f:1111] type 00 class 0x030000
[    5.525487] pci 0000:00:03.0: reg 0x10: [mem 0xfe400000-0xfe7fffff pref]
[    5.530978] pci 0000:00:03.0: reg 0x30: [mem 0xfebe0000-0xfebeffff pref]
[    5.532962] pci 0000:00:03.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    5.537522] pci 0000:00:04.0: [1d0f:8061] type 00 class 0x010802
[    5.542156] pci 0000:00:04.0: reg 0x10: [mem 0xfebf0000-0xfebf3fff]
[    5.549793] pci 0000:00:05.0: [1d0f:ec20] type 00 class 0x020000
[    5.553829] pci 0000:00:05.0: reg 0x10: [mem 0xfebf4000-0xfebf7fff]
[    5.568945] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[    5.572986] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[    5.577006] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[    5.580986] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[    5.584927] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[    5.589150] iommu: Default domain type: Translated 
[    5.672881] iommu: DMA domain TLB invalidation policy: lazy mode 
[    5.677045] SCSI subsystem initialized
[    5.680906] pci 0000:00:03.0: vgaarb: setting as boot VGA device
[    5.684865] pci 0000:00:03.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    5.684880] pci 0000:00:03.0: vgaarb: bridge control possible
[    5.688869] vgaarb: loaded
[    5.691891] ACPI: bus type USB registered
[    5.692885] usbcore: registered new interface driver usbfs
[    5.696911] usbcore: registered new interface driver hub
[    5.700886] usbcore: registered new device driver usb
[    5.704907] pps_core: LinuxPPS API ver. 1 registered
[    5.708871] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    5.712887] PTP clock support registered
[    5.716918] EDAC MC: Ver: 3.0.0
[    5.720294] NetLabel: Initializing
[    5.720378] NetLabel:  domain hash size = 128
[    5.804872] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    5.808887] NetLabel:  unlabeled traffic allowed by default
[    5.816899] PCI: Using ACPI for IRQ routing
[    5.822377] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    5.824872] hpet0: 8 comparators, 32-bit 62.500000 MHz counter
[    5.830988] clocksource: Switched to clocksource kvm-clock
[    5.843967] VFS: Disk quotas dquot_6.6.0
[    5.847843] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    5.922848] AppArmor: AppArmor Filesystem Enabled
[    5.926739] pnp: PnP ACPI init
[    5.930391] pnp: PnP ACPI: found 5 devices
[    5.941446] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    6.026262] NET: Registered PF_INET protocol family
[    6.030444] IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    6.046366] tcp_listen_portaddr_hash hash table entries: 512 (order: 1, 8192 bytes, linear)
[    6.125028] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    6.133305] TCP established hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    6.141522] TCP bind hash table entries: 8192 (order: 5, 131072 bytes, linear)
[    6.226833] TCP: Hash tables configured (established 8192 bind 8192)
[    6.231718] MPTCP token hash table entries: 1024 (order: 2, 24576 bytes, linear)
[    6.238967] UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
[    6.322873] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
[    6.328012] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    6.332382] NET: Registered PF_XDP protocol family
[    6.563305] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    6.568943] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    6.573883] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    6.579216] pci_bus 0000:00: resource 7 [mem 0x80000000-0xfebfffff window]
[    6.623316] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    6.627857] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    6.639175] pci 0000:00:01.0: quirk_isa_dma_hangs+0x0/0x20 took 11051 usecs
[    6.724729] PCI: CLS 0 bytes, default 64
[    6.728564] Trying to unpack rootfs image as initramfs...
[    6.728610] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x240937b9988, max_idle_ns: 440795218083 ns
[    6.837260] Freeing initrd memory: 6776K
[    6.846639] clocksource: Switched to clocksource tsc
[    6.859368] Initialise system trusted keyrings
[    6.865644] Key type blacklist registered
[    6.873736] workingset: timestamp_bits=36 max_order=18 bucket_order=0
[    6.924056] zbud: loaded
[    6.928391] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    6.933286] fuse: init (API version 7.34)
[    6.937673] integrity: Platform Keyring initialized
[    7.028742] Key type asymmetric registered
[    7.032394] Asymmetric key parser 'x509' registered
[    7.036368] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 243)
[    7.043396] io scheduler mq-deadline registered
[    7.121693] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    7.126599] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    7.133412] ACPI: button: Power Button [PWRF]
[    7.137226] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
[    7.224895] ACPI: button: Sleep Button [SLPF]
[    7.229030] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    7.258068] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    7.325938] Linux agpgart interface v0.103
[    7.331031] loop: module loaded
[    7.334710] nvme nvme0: pci function 0000:00:04.0
[    7.339036] tun: Universal TUN/TAP device driver, 1.6
[    7.339325] ACPI: \_SB_.LNKD: Enabled at IRQ 11
[    7.422333] PPP generic driver version 2.4.2
[    7.430268] VFIO - User Level meta-driver version: 0.3
[    7.434549] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    7.522385] ehci-pci: EHCI PCI platform driver
[    7.526290] ehci-platform: EHCI generic platform driver
[    7.530466] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    7.530909] nvme nvme0: 2/0/0 default/read/poll queues
[    7.535200] ohci-pci: OHCI PCI platform driver
[    7.626008] ohci-platform: OHCI generic platform driver
[    7.626803]  nvme0n1: p1
[    7.630597] uhci_hcd: USB Universal Host Controller Interface driver
[    7.722021] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    7.729207] i8042: Warning: Keylock active
[    7.733952] serio: i8042 KBD port at 0x60,0x64 irq 1
[    7.738180] serio: i8042 AUX port at 0x60,0x64 irq 12
[    7.821943] mousedev: PS/2 mouse device common for all mice
[    7.826499] rtc_cmos 00:00: RTC can wake from S4
[    7.831524] rtc_cmos 00:00: registered as rtc0
[    7.835415] rtc_cmos 00:00: setting system clock to 2023-03-14T14:21:02 UTC (1678803662)
[    7.924569] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[    7.928946] i2c_dev: i2c /dev entries driver
[    7.932579] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[    8.025769] device-mapper: uevent: version 1.0.3
[    8.029915] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    8.037094] platform eisa.0: Probing EISA bus 0
[    8.121991] platform eisa.0: EISA: Cannot allocate resource for mainboard
[    8.126853] platform eisa.0: Cannot allocate resource for EISA slot 1
[    8.132785] platform eisa.0: Cannot allocate resource for EISA slot 2
[    8.222025] platform eisa.0: Cannot allocate resource for EISA slot 3
[    8.226846] platform eisa.0: Cannot allocate resource for EISA slot 4
[    8.231783] platform eisa.0: Cannot allocate resource for EISA slot 5
[    8.243916] platform eisa.0: Cannot allocate resource for EISA slot 6
[    8.322847] platform eisa.0: Cannot allocate resource for EISA slot 7
[    8.327593] platform eisa.0: Cannot allocate resource for EISA slot 8
[    8.332242] platform eisa.0: EISA: Detected 0 cards
[    8.421881] intel_pstate: CPU model not supported
[    8.425977] ledtrig-cpu: registered to indicate activity on CPUs
[    8.430980] drop_monitor: Initializing network drop monitor service
[    8.435781] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
[    8.435829] NET: Registered PF_INET6 protocol family
[    8.531101] Segment Routing with IPv6
[    8.534641] In-situ OAM (IOAM) with IPv6
[    8.538286] NET: Registered PF_PACKET protocol family
[    8.622225] Key type dns_resolver registered
[    8.626334] No MBM correction factor available
[    8.630321] IPI shorthand broadcast: enabled
[    8.634057] sched_clock: Marking stable (3840468962, 4793564155)->(11954134005, -3320100888)
[    8.726208] registered taskstats version 1
[    8.730890] Loading compiled-in X.509 certificates
[    8.735639] Loaded X.509 cert 'Build time autogenerated kernel key: 9661695d80a98f99cb2df8c24334d09c0c25f561'
[    8.825119] Loaded X.509 cert 'Canonical Ltd. Live Patch Signing: 14df34d1a87cf37625abec039ef2bf521249b969'
[    8.834249] Loaded X.509 cert 'Canonical Ltd. Kernel Module Signing: 88f752e560a1e0737e31163a466ad7b70a850c19'
[    8.843244] blacklist: Loading compiled-in revocation X.509 certificates
[    8.924085] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing: 61482aa2830d0ab2ad5af10b7250da9033ddcef0'
[    8.932479] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2017): 242ade75ac4a15e50d50c84b0d45ff3eae707a03'
[    9.025798] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (ESM 2018): 365188c1d374d6b07c3c8f240f8ef722433d6a8b'
[    9.033963] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2019): c0746fd6c5da3ae827864651ad66ae47fe24b3e8'
[    9.125763] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v1): a8d54bbb3825cfb94fa13c9f8a594a195c107b8d'
[    9.133888] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v2): 4cf046892d6fd3c9a5b03f98d845f90851dc6a8c'
[    9.225728] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (2021 v3): 100437bb6de6e469b581e61cd66bce3ef4ed53af'
[    9.233890] Loaded X.509 cert 'Canonical Ltd. Secure Boot Signing (Ubuntu Core 2019): c1d57b8f6b743f23ee41f4f7ee292f06eecadfb9'
[    9.326496] zswap: loaded using pool lzo/zbud
[    9.330474] Key type .fscrypt registered
[    9.334020] Key type fscrypt-provisioning registered
[    9.422068] Key type encrypted registered
[    9.425734] AppArmor: AppArmor sha1 policy hashing enabled
[    9.429900] ima: No TPM chip found, activating TPM-bypass!
[    9.434111] Loading compiled-in module X.509 certificates
[    9.438991] Loaded X.509 cert 'Build time autogenerated kernel key: 9661695d80a98f99cb2df8c24334d09c0c25f561'
[    9.526111] ima: Allocated hash algorithm: sha1
[    9.529968] ima: No architecture policies found
[    9.533676] evm: Initialising EVM extended attributes:
[    9.622062] evm: security.selinux
[    9.625415] evm: security.SMACK64
[    9.628597] evm: security.SMACK64EXEC
[    9.631996] evm: security.SMACK64TRANSMUTE
[    9.635603] evm: security.SMACK64MMAP
[    9.639226] evm: security.apparmor
[    9.721743] evm: security.ima
[    9.724877] evm: security.capability
[    9.728279] evm: HMAC attrs: 0x1
[    9.731781] PM:   Magic number: 15:787:333
[    9.735395] tty tty54: hash matches
[    9.739288] RAS: Correctable Errors collector initialized.
[    9.823375] md: Waiting for all devices to be available before autodetect
[    9.828081] md: If you don't use raid, use raid=noautodetect
[    9.832329] md: Autodetecting RAID arrays.
[    9.921403] md: autorun ...
[    9.924458] md: ... autorun DONE.
[    9.930288] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[    9.938238] VFS: Mounted root (ext4 filesystem) readonly on device 259:1.
[   10.022625] devtmpfs: mounted
[   10.027028] Freeing unused decrypted memory: 2036K
[   10.031655] Freeing unused kernel image (initmem) memory: 3240K
[   10.040967] Write protecting the kernel read-only data: 30720k
[   10.122430] Freeing unused kernel image (text/rodata gap) memory: 2036K
[   10.127424] Freeing unused kernel image (rodata/data gap) memory: 1436K
[   10.211917] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[   10.216505] x86/mm: Checking user space page tables
[   10.299820] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[   10.304556] Run /sbin/init as init process
[   10.434894] systemd[1]: Inserted module 'autofs4'
[   10.455114] systemd[1]: systemd 245.4-4ubuntu3.20 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
[   10.532068] systemd[1]: Detected virtualization kvm.
[   10.536046] systemd[1]: Detected architecture x86-64.

Welcome to Ubuntu 20.04.5 LTS!

[   10.593309] systemd[1]: Set hostname to <ip-10-50-11-11>.
[   10.875347] systemd[1]: Created slice system-modprobe.slice.
[  OK  ] Created slice system-modprobe.slice.
[   10.882075] systemd[1]: Created slice system-serial\x2dgetty.slice.
[  OK  ] Created slice system-serial\x2dgetty.slice.
[   10.919613] systemd[1]: Created slice User and Session Slice.
[  OK  ] Created slice User and Session Slice.
[   10.925313] systemd[1]: Started ntp-systemd-netif.path.
[  OK  ] Started ntp-systemd-netif.path.
[   11.019644] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Started Forward Password R…uests to Wall Directory Watch.
[   11.028228] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[  OK  ] Set up automount Arbitrary…s File System Automount Point.
[   11.127629] systemd[1]: Reached target User and Group Name Lookups.
[  OK  ] Reached target User and Group Name Lookups.
[   11.223679] systemd[1]: Reached target Slices.
[  OK  ] Reached target Slices.
[   11.228457] systemd[1]: Reached target Mounting snaps.
[  OK  ] Reached target Mounting snaps.
[   11.323296] systemd[1]: Reached target System Time Set.
[  OK  ] Reached target System Time Set.
[   11.429808] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[  OK  ] Listening on Device-mapper event daemon FIFOs.
[   11.435824] systemd[1]: Listening on LVM2 poll daemon socket.
[  OK  ] Listening on LVM2 poll daemon socket.
[   11.523921] systemd[1]: Listening on multipathd control socket.
[  OK  ] Listening on multipathd control socket.
[   11.619313] systemd[1]: Listening on Syslog Socket.
[  OK  ] Listening on Syslog Socket.
[   11.624433] systemd[1]: Listening on initctl Compatibility Named Pipe.
[  OK  ] Listening on initctl Compatibility Named Pipe.
[   11.719601] systemd[1]: Listening on Journal Audit Socket.
[  OK  ] Listening on Journal Audit Socket.
[   11.725330] systemd[1]: Listening on Journal Socket (/dev/log).
[  OK  ] Listening on Journal Socket (/dev/log).
[   11.819835] systemd[1]: Listening on Journal Socket.
[  OK  ] Listening on Journal Socket.
[   11.825100] systemd[1]: Listening on Network Service Netlink Socket.
[  OK  ] Listening on Network Service Netlink Socket.
[   11.923662] systemd[1]: Listening on udev Control Socket.
[  OK  ] Listening on udev Control Socket.
[   11.929167] systemd[1]: Listening on udev Kernel Socket.
[  OK  ] Listening on udev Kernel Socket.
[   12.024727] systemd[1]: Mounting Huge Pages File System...
         Mounting Huge Pages File System...
[   12.031489] systemd[1]: Mounting POSIX Message Queue File System...
         Mounting POSIX Message Queue File System...
[   12.125367] systemd[1]: Mounting Kernel Debug File System...
         Mounting Kernel Debug File System...
[   12.220118] systemd[1]: Mounting Kernel Trace File System...
         Mounting Kernel Trace File System...
[   12.227257] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[   12.234023] systemd[1]: Starting Set the console keyboard layout...
         Starting Set the console keyboard layout...
[   12.326223] systemd[1]: Starting Create list of static device nodes for the current kernel...
         Starting Create list of st…odes for the current kernel...
[   12.420951] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
         Starting Monitoring of LVM…meventd or progress polling...
[   12.520288] systemd[1]: Starting Load Kernel Module chromeos_pstore...
         Starting Load Kernel Module chromeos_pstore...
[   12.528276] systemd[1]: Starting Load Kernel Module drm...
         Starting Load Kernel Module drm...
[   12.620589] systemd[1]: Starting Load Kernel Module efi_pstore...
         Starting Load Kernel Module efi_pstore...
[   12.627882] systemd[1]: Starting Load Kernel Module pstore_blk...
         Starting Load Kernel Module pstore_blk...
[   12.724330] systemd[1]: Starting Load Kernel Module pstore_zone...
         Starting Load Kernel Module pstore_zone...
[   12.731397] systemd[1]: Starting Load Kernel Module ramoops...
         Starting Load Kernel Module ramoops...
[   12.823891] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[   12.832925] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
[   12.927491] systemd[1]: Starting Load Kernel Modules...
         Starting Load Kernel Modules...
[   12.934324] systemd[1]: Starting Remount Root and Kernel File Systems...
         Starting Remount Root and Kern[   13.024666] EXT4-fs (nvme0n1p1): re-mounted. Opts: discard. Quota mode: none.
el File Systems...
[   13.028653] systemd[1]: Starting udev Coldplug all Devices...
         Startin[   13.041101] IPMI message handler: version 39.2
g udev Coldplug all Devices...
[   13.124368] systemd[1]: Starting Uncomplicated firewall...
         Starting Uncomplicated firewall...
[   13.132620] ipmi device interface
[   13.133507] systemd[1]: Started Journal Service.
[  OK  ] Started Journal Service.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Kernel Debug File System.
[  OK  ] Mounted Kernel Trace File System.
[  OK  ] Finished Set the console keyboard layout.
[  OK  ] Finished Create list of st… nodes for the current kernel.
[  OK  ] Finished Load Kernel Module chromeos_pstore.
[  OK  ] Finished Load Kernel Module drm.
[  OK  ] Finished Load Kernel Module efi_pstore.
[  OK  ] Finished Load Kernel Module pstore_blk.
[  OK  ] Finished Load Kernel Module pstore_zone.
[  OK  ] Finished Load Kernel Module ramoops.
[  OK  ] Finished Load Kernel Modules.
[  OK  ] Finished Remount Root and Kernel File Systems.
[  OK  ] Finished Uncomplicated firewall.
         Activating swap    13.647645] Adding 2097148k swap on /swapfile.  Priority:-2 extents:11 across:2433024k SSFS
9m/swapfile...
         Mounting FUSE Control File System...
         Mounting Kernel Configuration File System...
         Starting Flush Journal to Persistent Storage...
[   13.821114] systemd-journald[165]: Received client request to flush runtime journal.
         Starting Load/Save Random Seed...
         Starting Apply Kernel Variables...
         Starting Create System Users...
[  OK  ] Activated swap /swapfile.
[  OK  ] Finished udev Coldplug all Devices.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Mounted Kernel Configuration File System.
[  OK  ] Finished Load/Save Random Seed.
[  OK  ] Finished Apply Kernel Variables.
[  OK  ] Reached target Swap.
         Starting udev Wait for Complete Device Initialization...
[  OK  ] Finished Create System Users.
         Starting Create Static Device Nodes in /dev...
[  OK  ] Finished Flush Journal to Persistent Storage.
[  OK  ] Finished Create Static Device Nodes in /dev.
         Starting udev Kernel Device Manager...
[  OK  ] Started udev Kernel Device Manager.
[  OK  ] Started Dispatch Password …ts to Console Directory Watch.
[  OK  ] Reached target Local Encrypted Volumes.
[  OK  ] Found device /dev/ttyS0.
[  OK  ] Listening on Load/Save RF …itch Status /dev/rfkill Watch.
         Mounting Arbitrary Executable File Formats File System...
[  OK  ] Mounted Arbitrary Executable File Formats File System.
[  OK  ] Finished udev Wait for Complete Device Initialization.
         Starting Device-Mapper Multipath Device Controller...
[  OK  ] Started Device-Mapper Multipath Device Controller.
[  OK  ] Finished Monitoring of LVM… dmeventd or progress polling.
[  OK  ] Reached target Local File Systems (Pre).
         Mounting Mount unit for am…on-ssm-agent, revision 5656...
         Mounting Mount unit for am…on-ssm-agent, revision 6312...
         Mounting Mount unit for core, revision 14447...
         Mounting Mount unit for core, revision 14784...
         Mounting Mount unit for core18, revision 2708...
         Mounting Mount unit for core18, revision 2714...
         Mounting Mount unit for core20, revision 1822...
         Mounting Mount unit for core20, revision 1828...
         Mounting Mount unit for lxd, revision 24323...
         Mounting Mount unit for lxd, revision 24483...
[  OK  ] Mounted Mount unit for amazon-ssm-agent, revision 5656.
[  OK  ] Mounted Mount unit for amazon-ssm-agent, revision 6312.
[  OK  ] Mounted Mount unit for core, revision 14447.
[  OK  ] Mounted Mount unit for core, revision 14784.
[  OK  ] Mounted Mount unit for core18, revision 2708.
[  OK  ] Mounted Mount unit for core18, revision 2714.
[  OK  ] Mounted Mount unit for core20, revision 1822.
[  OK  ] Mounted Mount unit for core20, revision 1828.
[  OK  ] Mounted Mount unit for lxd, revision 24323.
[  OK  ] Mounted Mount unit for lxd, revision 24483.
[  OK  ] Reached target Mounted snaps.
[  OK  ] Reached target Local File Systems.
         Starting Load AppArmor profiles...
         Starting Set console font and keymap...
         Starting Create final runt…dir for shutdown pivot root...
         Starting Tell Plymouth To Write Out Runtime Data...
         Starting Create Volatile Files and Directories...
[  OK  ] Finished Load AppArmor profiles.
[  OK  ] Finished Set console font and keymap.
[  OK  ] Finished Create final runt…e dir for shutdown pivot root.
[  OK  ] Finished Tell Plymouth To Write Out Runtime Data.
[  OK  ] Finished Create Volatile Files and Directories.
         Starting Load Kernel Module chromeos_pstore...
         Starting Load Kernel Module pstore_blk...
         Starting Load Kernel Module pstore_zone...
         Starting Load Kernel Module ramoops...
         Starting Tell Plymouth To Write Out Runtime Data...
         Starting Load AppArmor pro…managed internally by snapd...
         Starting Update UTMP about System Boot/Shutdown...
         Starting Initial cloud-init job (pre-networking)...
[  OK  ] Finished Load Kernel Module chromeos_pstore.
[  OK  ] Finished Load Kernel Module pstore_blk.
[  OK  ] Finished Load Kernel Module pstore_zone.
[  OK  ] Finished Load Kernel Module ramoops.
[  OK  ] Finished Tell Plymouth To Write Out Runtime Data.
[  OK  ] Finished Update UTMP about System Boot/Shutdown.
[  OK  ] Finished Load AppArmor pro…s managed internally by snapd.
[   12.848701] cloud-init[394]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'init-local' at Tue, 14 Mar 2023 14:21:11 +0000. Up 12.51 seconds.
[  OK  ] Finished Initial cloud-init job (pre-networking).
[  OK  ] Reached target Network (Pre).
         Starting Network Service...
[  OK  ] Started Network Service.
         Starting Wait for Network to be Configured...
         Starting Network Name Resolution...
[  OK  ] Started Network Name Resolution.
[  OK  ] Reached target Network.
[  OK  ] Reached target Host and Network Name Lookups.
[FAILED] Failed to start Wait for Network to be Configured.
See 'systemctl status systemd-networkd-wait-online.service' for details.
         Starting Initial cloud-ini… (metadata service crawler)...
[  133.766642] cloud-init[424]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'init' at Tue, 14 Mar 2023 14:23:12 +0000. Up 133.52 seconds.
[  133.770136] cloud-init[424]: ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++
[  133.829712] cloud-init[424]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[  133.835748] cloud-init[424]: ci-info: | Device |   Up  |  Address  |    Mask   | Scope |     Hw-Address    |
[  133.839750] cloud-init[424]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[  133.931105] cloud-init[424]: ci-info: |  ens5  | False |     .     |     .     |   .   | 02:51:b1:44:7b:c4 |
[  133.933242] cloud-init[424]: ci-info: |   lo   |  True | 127.0.0.1 | 255.0.0.0 |  host |         .         |
[  OK  ] Finished Initial cloud-ini…ob (metadata service crawler).
[  134.031146] cloud-init[424]: ci-info: |   lo   |  True |  ::1/128  |     .     |  host |         .         |
[  OK  ] Reached target Cloud-config availability.
[  134.130261] cloud-init[424]: ci-info: +--------+-------+-----------+-----------+-------+-------------------+
[  OK  ] Reached target Network is Online.
[  134.132819] cloud-init[424]: ci-info: +++++++++++++++++++Route IPv6 info+++++++++++++++++++
[  OK  ] Reached target System Initialization.
[  134.231865] cloud-init[424]: ci-info: +-------+-------------+---------+-----------+-------+
[  OK  ] Started ACPI Events Check.
[  134.330620] cloud-init[424]: ci-info: | Route | Destination | Gateway | Interface | Flags |
[  134.332656] cloud-init[424]: ci-info: +-------+-------------+---------+-----------+-------+
[  OK  ] Started Daily apt download activities.
[  134.333580] cloud-init[424]: ci-info: +-------+-------------+---------+-----------+-------+
[  OK  ] Started Daily apt upgrade and clean activities.
[  OK  ] Started Periodic ext4 Onli…ata Check for All Filesystems.
[  OK  ] Started Discard unused blocks once a week.
[  OK  ] Started Refresh fwupd metadata regularly.
[  OK  ] Started Daily rotation of log files.
[  OK  ] Started Daily man-db regeneration.
[  OK  ] Started Message of the Day.
[  OK  ] Started Clean PHP session files every 30 mins.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Started Ubuntu Advantage Timer for running repeated jobs.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Timers.
[  OK  ] Listening on ACPID Listen Socket.
[  OK  ] Listening on cloud-init hotplug hook socket.
[  OK  ] Listening on D-Bus System Message Bus Socket.
[  OK  ] Listening on Open-iSCSI iscsid Socket.
[  OK  ] Listening on Socket unix for snap application lxd.daemon.
[  OK  ] Listening on Socket unix f…p application lxd.user-daemon.
         Starting Socket activation for snappy daemon.
[  OK  ] Listening on UUID daemon activation socket.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
         Starting Availability of block devices...
[  OK  ] Listening on Socket activation for snappy daemon.
[  OK  ] Finished Availability of block devices.
[  OK  ] Reached target Sockets.
[  OK  ] Reached target Basic System.
         Starting Accounts Service...
[  OK  ] Started ACPI event daemon.
         Starting The Apache HTTP Server...
         Starting LSB: automatic crash report generation...
         Starting Deferred execution scheduler...
[  OK  ] Started Regular background program processing daemon.
[  OK  ] Started D-Bus System Message Bus.
[  OK  ] Started Save initial kernel messages after boot.
         Starting Remove Stale Onli…t4 Metadata Check Snapshots...
         Starting EC2 Instance Connect Host Key Harvesting...
         Starting Record successful boot for GRUB...
[  OK  ] Started irqbalance daemon.
         Starting MariaDB 10.3.38 database server...
         Starting Dispatcher daemon for systemd-networkd...
[  OK  ] Started ntp-systemd-netif.service.
         Starting Network Time Service...
         Starting The PHP 7.3 FastCGI Process Manager...
         Starting Authorization Manager...
         Starting System Logging Service...
[  OK  ] Started Service for snap a…on-ssm-agent.amazon-ssm-agent.
         Starting Service for snap application lxd.activate...
[  OK  ] Reached target Login Prompts (Pre).
         Starting Snap Daemon...
         Starting Login Service...
         Starting Permit User Sessions...
         Starting Disk Manager...
[  OK  ] Started Dispatcher daemon for systemd-networkd.
[  OK  ] Started The Apache HTTP Server.
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Finished Remove Stale Onli…ext4 Metadata Check Snapshots.
[  OK  ] Finished EC2 Instance Connect Host Key Harvesting.
[  OK  ] Started Network Time Service.
[  OK  ] Finished Permit User Sessions.
[  OK  ] Started The PHP 7.3 FastCGI Process Manager.
         Starting Hold until boot process finishes up...
         Starting Terminate Plymouth Boot Screen...
         Starting OpenBSD Secure Shell server...
[  OK  ] Started System Logging Service.
[  OK  ] Finished Record successful boot for GRUB.
[  OK  ] Finished Hold until boot process finishes up.
[  OK  ] Finished Terminate Plymouth Boot Screen.
[  OK  ] Started Login Service.
[  OK  ] Started Authorization Manager.
[  OK  ] Started Disk Manager.
[  OK  ] Started OpenBSD Secure Shell server.
[  OK  ] Started LSB: automatic crash report generation.
[  OK  ] Started Accounts Service.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
         Starting GRUB failed boot detection...
         Starting Modem Manager...
[  OK  ] Started Serial Getty on ttyS0.
         Starting Set console scheme...
[  OK  ] Started Unattended Upgrades Shutdown.
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Finished Set console scheme.
[  OK  ] Started Modem Manager.
[  OK  ] Created slice system-getty.slice.
[  OK  ] Started Getty on tty1.
[  OK  ] Reached target Login Prompts.
[  OK  ] Finished GRUB failed boot detection.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Stopped Deferred execution scheduler.
         Starting Deferred execution scheduler...
[  OK  ] Started Deferred execution scheduler.
[  OK  ] Started MariaDB 10.3.38 database server.
[  OK  ] Stopped Deferred execution scheduler.
[FAILED] Failed to start Deferred execution scheduler.
See 'systemctl status atd.service' for details.

Ubuntu 20.04.5 LTS ip-10-50-11-11 ttyS0

ip-10-50-11-11 login: [  152.881374] cloud-init[890]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:config' at Tue, 14 Mar 2023 14:23:32 +0000. Up 152.77 seconds.
[  153.490409] cloud-init[897]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:final' at Tue, 14 Mar 2023 14:23:32 +0000. Up 153.36 seconds.
[  153.492606] cloud-init[897]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 finished at Tue, 14 Mar 2023 14:23:32 +0000. Datasource DataSourceEc2Local.  Up 153.48 seconds

woonjjang commented 1 year ago

host file 변경과 관련된 이슈 링크

woonjjang commented 1 year ago

Welcome to Ubuntu 20.04.5 LTS!

[   10.539933] systemd[1]: Set hostname to <ip-10-50-11-11>.

woonjjang commented 1 year ago

안되면 https://aws.amazon.com/ko/premiumsupport/knowledge-center/ec2-linux-status-check-failure-os-errors/

저장장치를 다른 ec2에 붙여서 복원시도 하는 방법

woonjjang commented 1 year ago

1번 이슈

최종적으로 서버 리스타트가 되었는데, ssh 접근은 안 되는 상황
EC2 직렬콘솔 통해서 접근했는데, ubuntu 계정에 비번이나 루트 계정을 모르는 상황...
```
Ubuntu 20.04.5 LTS ip-10-50-11-11 ttyS0
```

ip-10-50-11-11 login: [ 155.760122] cloud-init[886]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:config' at Tue, 14 Mar 2023 23:31:16 +0000. Up 155.65 seconds. [ 156.377029] cloud-init[893]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'modules:final' at Tue, 14 Mar 2023 23:31:16 +0000. Up 156.24 seconds. [ 156.380204] cloud-init[893]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 finished at Tue, 14 Mar 2023 23:31:16 +0000. Datasource DataSourceEc2Local. Up 156.37 seconds

ip-10-50-11-11 login: ip-10-50-11-11 login: ip-10-50-11-11 login: ip-10-50-11-11 login:


## 2번이슈
- @mozodev 언급했던데로 host 문제가 있어서, 중간에 오류가 나는 것으로 보임
- 관련해서 ec2는 가상설정에 대해 조정하거 할 수 있는 영역이 서버 접근하지 않으면 불가능해서, 관련 호스트를 수정하거나 할 수 없음.

Finished Initial cloud-init job (pre-networking). [ OK ] Reached target Network (Pre). Starting Network Service... [ OK ] Started Network Service. Starting Wait for Network to be Configured... Starting Network Name Resolution... [ OK ] Started Network Name Resolution. [ OK ] Reached target Network. [ OK ] Reached target Host and Network Name Lookups. [FAILED] Failed to start Wait for Network to be Configured. See 'systemctl status systemd-networkd-wait-online.service' for details. Starting Initial cloud-ini… (metadata service crawler)... [ 134.664899] cloud-init[423]: Cloud-init v. 22.4.2-0ubuntu0~20.04.2 running 'init' at Tue, 14 Mar 2023 23:30:54 +0000. Up 134.53 seconds. [ 134.668429] cloud-init[423]: ci-info: +++++++++++++++++++++++++++Net device info++++++++++++++++++++++++++++



## 3번 이슈
- 관련해서 전부 다시 밀고 다른 ec2로 붙이거나 AMI 통해서 시도해보고 싶으나, 관련된 경험이 없어서
- data나 이런 것이 날아갈 것이 걱정됨. 
- files 와 db 백업 최종본이 local에 없어서, 쉽게 ec2 자체를 날리지 못 함.

woonjjang commented 1 year ago

EC2

EFS 볼륨을 스냅샷을 떠서 거기 저장되어 있는 소스, 백업 디비를 통해서 소스 / DB 확보
새로운 EC2를 실행해서, 관련되 provisioning 과정을 거침
provision https://github.com/team-durumi/kyeol-composer/tree/main/provision 를 따라서 최대한 설치
composer / drush 까지 설치 완료

drush rs 단계에서 오류를 넘어가지 못 하고 있음.

ubuntu@ip-172-31-11-243:/var/www/kyeol-composer$ drush rs
Command runserver needs a higher bootstrap level to run - you will need to invoke drush from a more functional Drupal environment to run this  [error]
command.
The drush command 'rs' could not be executed.                                                                                                  [error]
Drush was not able to start (bootstrap) the Drupal database.                                                                                   [error]
Hint: This may occur when Drush is trying to:
 * bootstrap a site that has not been installed or does not have a configured database. In this case you can select another site with a working
database setup by specifying the URI to use with the --uri parameter on the command line. See `drush topic docs-aliases` for details.
 * connect the database through a socket. The socket file may be wrong or the php-cli may have no access to it in a jailed shell. See
http://drupal.org/node/1428638 for details.
 * connect to the database through a cli command using the --defaults-extra-file parameter to pass credentials through a tmp file. This method
can break if other credentials are specified in a ~/.my.cnf file in your home directory. You may have to delete or rename the ~/.my.cnf file in
your home directory.

Drush was attempting to connect to: 
 Drupal version         :  7.82                                                          
 Site URI               :  http://default                                                
 PHP configuration      :  /etc/php/7.4/cli/php.ini                                      
 PHP OS                 :  Linux                                                         
 Drush script           :  /usr/local/bin/drush                                          
 Drush version          :  8.4.8                                                         
 Drush temp directory   :  /tmp                                                          
 Drush configuration    :  /var/www/kyeol-composer/drush/drushrc.php                     
 Drush alias files      :  /var/www/kyeol-composer/web/../drush/self.aliases.drushrc.php 
 Drupal root            :  /var/www/kyeol-composer/web                                   
 Drupal Settings File   :  sites/default/settings.php                                    
 Site path              :  sites/default

mozodev commented 1 year ago

drush status 실행하면 마지막 정보를 확인할 수 있는데 database 정보가 없는 걸로 보아서 settings.php 에 데이터베이스 정보가 없는 것 같네요. 디비 생성 및 복원하고 데이터베이스 사용자 비번으로 디비 접근이 되는 거 확인한 후에 settings.php에 해당 정보를 기입한 후에 시도해보세요. 굉장히 느린 chat-GPT가 된 느낌이네요...

woonjjang commented 1 year ago

죄송해요...

settings.php 와 local.settings.php 두 가지 설정으로 되어 있었는데, local.settings.php에 데이터베이스 정보가 붙어있고, settings.php 에는 local.settings.php을 바라보라고 해서 연결하면, 비슷한 오류 상태를 반복하고 있습니다. DB 생성 및 복원은 마치고 데이터베이스도 Tableplus로 접근을 하고 있습니다.

좀 더 해보겠습니다. !!

woonjjang commented 1 year ago

https://kyeol.kr/

메모를 따라서 ssl config 설정을 마치고 나서 접근 완료. 기존 파일이 안 불러와줘서 folder 권한 문제로 생각했었으나, 저장된 파일의 경로들도 https로 저장되어있는지 오래되어,

kyeol-ssl.conf 구성에 따라 ssl 작동하는 것을 구성했음.

[테스트]

글쓰기, 파일업로드 기본적인 테스트 완료
서버 reboot 시에 ssh 접근 관련 오류는 계속 생기는 것으로 보임.

woonjjang commented 1 year ago

라이브 이후 이슈는 종료. 화요일까지 서버 상태 커버 및 취약조치 대응

team-durumi / kyeol-composer