fosslinux / live-bootstrap

Use of a Linux initramfs to fully automate the bootstrapping process
286 stars 26 forks source link

working qemu checkout? kexec-fiwix: jumping to trampoline... #448

Closed pyy closed 3 months ago

pyy commented 4 months ago

Hello everyone, Firstly, thanks a lot for this great project! I tried to use --qemu to run this program, but got this, and it forever stops here. have tried different checkout, but it is still same or similar.

Could someone give me a working checkout hint for --qemu?

./rootfs.py --qemu --cores 10 --qemu-ram 8192 --force-timestamps --external-sources ./rootfs.py --qemu results same

copy_file: /kaem.x86
copy_file: /aarch64.answers
copy_file: /riscv32.answers
copy_file: /amd64.answers
copy_file: /CHANGELOG.org
copy_file: /preseeded.kaem
copy_file: /makefile
copy_file: /kaem.aarch64
copy_file: /kaem.riscv64
copy_file: /x86.answers
copy_file: /README.md
copy_file: /cleanup.sh
copy_file: /after.kaem
copy_file: /e820
Fiwix ext2 initrd created successfully.
 +> if match xFalse xTrue
kexec-fiwix: starting kernel /boot/fiwix...

kexec-fiwix: Fiwix kernel file length: 375816
kexec-fiwix: Reading kernel...
ELF virtual entry point       : 0x80100030
ELF program header offset     : 0x34
ELF number of program  headers: 2
ELF size of program  headers  : 32
kexec-fiwix: Placing kernel in memory...
ELF physical entry point      : 0x100030
header 0:
    p_offset: 0x00001000
    p_paddr : 0x00100000
    p_filesz: 0x0004b5c0
    p_memsz : 0x0004b5c0
header 1:
    p_offset: 0x0004c5c0
    p_paddr : 0x0014c5c0
    p_filesz: 0x0000f4c4
    p_memsz : 0x00037df0
Preparing multiboot info for kernel...
Found image at filenum 8835
initrd_src: 0x67c7dc66
initrd_len: 0x54000000
kexec-fiwix: Memory map file length: 192
kexec-fiwix: Reading memory map file...
Preparing trampoline...
kexec-fiwix: jumping to trampoline..

for checkout version 493ddfa8293fc158342dc796ab6021dae2a491aa it got Kernel panic - not syncing: Attempted to kill init!

Googulator commented 4 months ago

That version is extremely old; try a more recent one.

Also, is this on a 13th or 14th generation Intel CPU on the host?

pyy commented 4 months ago

Thanks for your quick reply.

It's on Intel i5-12500 12th generation. Latest checkout 4737ab7481817e2bccab42669571cff7cca712d3 same problem.

Also tried on i7 4th generation, with this latest checkout. same problem.

pder commented 4 months ago

I have been in the same boat with hanging on the same kexec-fiwix: jumping to trampoline.. and I have an i7 14700K

The root cause is still unknown but I have had success updating my Fiwix tarball to use newer sources and it boots successfully consistently.

Could you try testing the following? cd live-bootstrap git remote add pder https://github.com/pder/live-bootstrap.git git fetch pder git checkout -b newer-fiwix pder/newer-fiwix

./rootfs.py -q

pyy commented 4 months ago

@pder Thanks for your code.

May I ask which checkout commit of Fiwix do you use for this file? https://github.com/mikaku/Fiwix/releases/download/v1.5.0-lb1/fiwix-1.5.0-lb1.tar.gz e049c8cab5d36eac6bbf58915e7c064ec9b3a54c21bf47d6db293ae938af1ae1

I've tried your https://github.com/pder/live-bootstrap/tree/newer-fiwix with Fiwix's latest master checkout (https://github.com/mikaku/Fiwix/tree/f997f3144612743d341f7dd62f01c7d07a387a88). I got this.

...
s.o kernel/syscalls/getrlimit.o kernel/syscalls/sigreturn.o kernel/syscalls/ioct
l.o kernel/syscalls/umask.o kernel/syscalls/geteuid.o kernel/syscalls/munmap.o k
ernel/syscalls/alarm.o kernel/syscalls/nanosleep.o kernel/syscalls/close.o kerne
l/syscalls/chroot.o kernel/syscalls/ftruncate64.o kernel/syscalls/truncate64.o k
ernel/syscalls/msgrcv.o kernel/syscalls/umount2.o kernel/syscalls/time.o kernel/
syscalls/stime.o kernel/syscalls/semop.o kernel/syscalls/newlstat.o kernel/sysca
lls/sigpending.o kernel/syscalls/reboot.o kernel/syscalls/setuid.o kernel/syscal
ls/wait4.o kernel/syscalls/chown32.o kernel/syscalls/fcntl.o kernel/syscalls/fst
at64.o kernel/syscalls/shmctl.o kernel/syscalls/dup2.o kernel/syscalls/setpgid.o
 kernel/syscalls/getdents.o kernel/syscalls/getpgrp.o kernel/syscalls/fchown.o k
ernel/syscalls/execve.o kernel/syscalls/shmget.o kernel/syscalls/lchown.o mm/bio
s_map.o mm/buddy_low.o mm/memory.o mm/page.o mm/alloc.o mm/fault.o mm/mmap.o mm/
swapper.o fs/filesystems.o fs/devices.o fs/buffer.o fs/fd.o fs/locks.o fs/super.
o fs/inode.o fs/namei.o fs/elf.o fs/script.o fs/minix/super.o fs/minix/bitmaps.o
 fs/minix/inode.o fs/minix/namei.o fs/minix/symlink.o fs/minix/dir.o fs/minix/fi
le.o fs/minix/v1_inode.o fs/minix/v2_inode.o fs/ext2/inode.o fs/ext2/super.o fs/
ext2/namei.o fs/ext2/symlink.o fs/ext2/dir.o fs/ext2/file.o fs/ext2/bitmaps.o fs
/pipefs/super.o fs/pipefs/fifo.o fs/pipefs/pipe.o fs/iso9660/inode.o fs/iso9660/
super.o fs/iso9660/namei.o fs/iso9660/dir.o fs/iso9660/file.o fs/iso9660/rrip.o
fs/iso9660/symlink.o fs/procfs/super.o fs/procfs/inode.o fs/procfs/namei.o fs/pr
ocfs/dir.o fs/procfs/file.o fs/procfs/symlink.o fs/procfs/tree.o fs/procfs/data.
o drivers/char/console.o drivers/char/tty.o drivers/char/tty_queue.o drivers/cha
r/vt.o drivers/char/defkeymap.o drivers/char/keyboard.o drivers/char/memdev.o dr
ivers/char/serial.o drivers/char/lp.o drivers/char/fb.o drivers/char/sysrq.o dri
vers/block/dma.o drivers/block/floppy.o drivers/block/part.o drivers/block/ata.o
 drivers/block/ata_hd.o drivers/block/ata_pci.o drivers/block/atapi.o drivers/bl
ock/atapi_cd.o drivers/block/ramdisk.o drivers/pci/pci.o drivers/video/video.o d
rivers/video/vgacon.o drivers/video/fbcon.o drivers/video/fonts.o drivers/video/
bga.o lib/ctype.o lib/strings.o lib/printk.o lib/sysconsole.o -o /boot/fiwix
 +> cd ../..
 +> if match xTrue xTrue
 +> then
 +> sha256sum -o fiwix-1.5.0-lb1.checksums /boot/fiwix
 +> cp fiwix-1.5.0-lb1.checksums /steps
 +> else
 +> sha256sum -c fiwix-1.5.0-lb1.checksums
Unexpected EOF, improperly terminated if statement.
Subprocess error 1
ABORTING HARD
Subprocess error 1
ABORTING HARD
Subprocess error 1
ABORTING HARD
Subprocess error
ABORTING HARD
Build finished.
ERROR: no hda writes!
pder commented 4 months ago

I actually included distfiles/fiwix-1.5.0-lb1.tar.gz in my newer-fiwix branch for convenience. This tarball was created with Fiwix commit f997f3144612743d341f7dd62f01c7d07a387a88 which is what you mentioned above. If you created your own tarball of the source you can compare it to the one I included in the commit.

I created it using this command:

cd Fiwix
tar czvf ../fiwix-1.5.0-lb1.tar.gz --transform 's,\.,fiwix-1.5.0-lb1,' --exclude-vcs .
pyy commented 4 months ago

@pder diff -qr dir1/ dir2/ shows we are using the same fiwix-1.5.0-lb1.tar.gz file. I also get /boot/fiwix caf7b68ee2322be72b9f2c8e0841588e3cc37fabf3316843368ca75475322551 this same result. So I think fiwix compiles successfully.

I don't know what's wrong here, anyone has a clue? Thank you.

Unexpected EOF, improperly terminated if statement.
Subprocess error 1
ABORTING HARD
Subprocess error 1
ABORTING HARD
Subprocess error 1
ABORTING HARD
Subprocess error
ABORTING HARD
Build finished.
ERROR: no hda writes!
pder commented 4 months ago

Could you run git status and git submodule status?

Should look like this: $ git status On branch newer-fiwix nothing to commit, working tree clean

$ git submodule status a2781242d19e6be891b453d8fa827137ab5db31a builder-hex0 (heads/main-18-ga278124) 63fd7767db0651b703b8942cfc0c92989a3e4b25 seed/stage0-posix (Release_1.6.0-3-g63fd776)

ajherchenroder commented 4 months ago

What is your host distro? I have had this same problem running on arch. My current workaround is to use nested virtualization. I run a Debian VM and then run the bootstrap in Debian. That seems to work for me.

pyy commented 4 months ago

During the weekends, I have made different tests. And it works in below way. I really gratefully welcome everyone to comment on my content. Thank you.

git clone https://github.com/fosslinux/live-bootstrap
cd live-bootstrap
git submodule update --init --recursive
./download-distfiles.sh

git remote add pder https://github.com/pder/live-bootstrap.git
git fetch pder
git checkout -b newer-fiwix pder/newer-fiwix

sha256sum distfiles/fiwix-1.5.0-lb1.tar.gz 
e049c8cab5d36eac6bbf58915e7c064ec9b3a54c21bf47d6db293ae938af1ae1  distfiles/fiwix-1.5.0-lb1.tar.gz

nano steps/improve/get_network.sh
see below.

./rootfs.py -q --external-sources

I have changed steps/improve/get_network.sh file to delete verification internet step, only left dhcpcd --waitip=4. Otherweise sometimes it will fail in different ways. Perhaps it conflicts with --external-sources? I need make more tests to confirm it.

When I use bigger cpu cores or ram, it will fail to kexec-fiwix: jumping to trampoline... again. For example, --cores 8 or --qemu-ram 8192 or --qemu-ram 16384 are all will fail. Secondly, --qemu-cmd qemu-system-i386 will also fail no matter the core or ram setting.

Also, if I make multiple qemu tests at the same time, using different folders and different arguments, it will faill to kexec-fiwix: jumping to trampoline... or other Kernel panic inside qemu. Perhaps because of i5-12500 ? Or host os settings?

For bare-metal test, after compiling this new fiwix kernel, the computer will reboot, and it comes to very begin step again, and finally become a loop running. Anyone know how to fix it?

Next step, I will try to run this script fully offline, like -nic none, -nic user,ipv6=off,model=e1000,restrict=on

@pder Thanks for your code. Yes, I have the same git submodule status results as yours. I found that, my re-packed tar.gz (https://github.com/mikaku/Fiwix/archive/f997f3144612743d341f7dd62f01c7d07a387a88.tar.gz) of latested Fiwix code doesn't work, it has the same inside contents as yours. It compiles fiwix successfully but it will stop at Unexpected EOF, improperly terminated if statement. in later steps. I have no idea. So I use your tar.gz file.

@ajherchenroder I have tried both refresh minmial Debian 12 and OpenSUSE Leap 15.5 as host distro on an Intel i5-12500 dedicated server for this live-bootstrap project, and finally get above results.

ajherchenroder commented 4 months ago

I looked at my Debian VM and it’s actually Debian 11 without back ports. I don’t know if that will make a difference but it may be worth a try.