mobinmob / void-66-services

66 service frontends for voidlinux
Other
30 stars 11 forks source link

rpi4 - didn't boot. How to switch on debugging infos? #154

Open dreamcat4 opened 2 years ago

dreamcat4 commented 2 years ago

Hi there @mobinmob & thanks for working on this project!

Tried having a go on rpi4 yesterday.

Any way I can provide a better feedback on this? Assuming that you don't have an rpi device yourself. Then it's necessary to help debug the problem.

I also tried to look for any specific customizations for rpi in void linux itself. However mostly it's just a few service that comes up later on in boot stages. Such as to configure networking etc. But we are nowhere near that when the boot failed. It's still like trying to mount early stuff (for example like /proc), or boot the initramfs.

Then it does nothing for several minutes. And complains 'out of memory error' starts culling some task. Then a few minutes later it does out of memory again.

mobinmob commented 2 years ago

Hi there @mobinmob & thanks for working on this project!

Thank you for testing :)

Tried having a go on rpi4 yesterday.

* Followed the default instructions written in `conf/void-66-conf.md` file.

  * this seemed to look generally OK. No obvious error msgs or anything

* At the end of the steps I add the kernel argument to switch over to 66. And then reboot it

* However the booting fails

Any way I can provide a better feedback on this? Assuming that you don't have an rpi device yourself. Then it's necessary to help debug the problem.

Increase the VERBOSITY setting in /etc/66/init.conf to 4. If you can get past very early boot , check if there is the early getty in tty12, even if the boot process has crashed. You can see the early boot messages there in /run/66/log/0/current .You can also save the log if you remount root rw.

I also tried to look for any specific customizations for rpi in void linux itself. However mostly it's just a few service that comes up later on in boot stages. Such as to configure networking etc. But we are nowhere near that when the boot failed. It's still like trying to mount early stuff (for example like /proc), or boot the initramfs.

66 is not associated to the initramfs in any way - I use the same initramfs for runit and 66. If a crash occurs after displaying Booting Void Linux then it is an issue with 66 - probably with something in the boot@system modules service and its components. But, please try to save a verbose log and upload it ito a pastebin service, along with the environment file of the boot@systemd service. The environment file is linked to /etc/66rc.conf if you run the66boot-initial-setup script.

Networking configuration is usually done outside of boot@system - I will gladly accept a frontend for rpi if there is something special needed :)

Then it does nothing for several minutes. And complains 'out of memory error' starts culling some task. Then a few minutes later it does out of memory again.

That is a first. I have tested 66/boot@ in constrained setups but I have never seen such behavior. It merits some investigation...

dreamcat4 commented 2 years ago

66rc.conf:

https://gist.github.com/dreamcat4/3b80bbf902fc6e74d599c5359a2caae0

check if there is the early getty in tty12

unfortunately no, there is no way to change the TTY, F1-F12 nothing happens. Perhaps is there any other logfile of previous failed boot been recorded on the disk?

dreamcat4 commented 2 years ago

It seems i forgot to mention that i manually tried to change MNT_SYS=!no. Because after seeing the place where it got stuck trying to mount /sys. However that was the only intervention i tried from the default config which was generated for me (but it didn't seem to make any difference!) hehe

dreamcat4 commented 2 years ago

after some extra minutes... out of memory error void-66-rpi4-out-of-memory-error

mobinmob commented 2 years ago

66rc.conf:

https://gist.github.com/dreamcat4/3b80bbf902fc6e74d599c5359a2caae0

It is not possible to boot with this environment file, as it contains only a fraction of the needed keys. Check this one: https://termbin.com/deo4

Νο idea how it was produced as 66boot-initial-setup should never remove keys :(

dreamcat4 commented 2 years ago

ok great news! thank you. i shall look into this tomorrow morning. to try to get a further progress

dreamcat4 commented 2 years ago

It is not possible to boot with this environment file, as it contains only a fraction of the needed keys. Check this one: https://termbin.com/deo4

ok i checked - it's the same exact on my system. Sorry it was just copy / pasted wrong from the terminal. It cut off the file. All is there

https://gist.github.com/dreamcat4/e2629ecce8f66dc22abddc37b88884a3

mobinmob commented 2 years ago

unfortunately no, there is no way to change the TTY, F1-F12 nothing happens. Perhaps is there any other logfile of previous failed boot been recorded on the disk?

The early getty always starts in tty12. It starts very early in order to function as an emergency/rescue shell. It should be up at that stage. The other ttys start much later in the boot process. There is an issue with having an early log saved to disk. Without root mounted rw, there is no permanent storage available to save the log file. The log file is saved to the tmpfs mounted under /run/66. Ι plan to have a service that will move the log for tmpfs to the disk, but in your case it will not make a difference - the crash happens very early...

It seems i forgot to mention that i manually tried to change MNT_SYS=!no. Because after seeing the place where it got stuck trying to mount /sys. However that was the only intervention i tried from the default config which was generated for me (but it didn't seem to make any difference!) hehe

Judging by the screenshot in the next post, you have MNT_SYS=!no but you have not re-enabled the boot@system service. The configuration takes effect with 66-enable, not just by changing the environment file. Also, the conf file/environment file is not auto-configured in any way. You have to taylor the conf for your system. I have done some work to ease the process with 66boot-storage-autoconf and 66boot-rcdotconf, but I have not released them yet to the PR and repository. Having said that, I do not think MNT_SYS is the problem there :( It is almost certain that you need it on. It may be a good idea to enable SWAP - it is off by default. Please check your configuration, re-enable boot@system as suggested in the doc. If you encounter the same behavior after that, you may need to send a report to the 66 mailing list, as it may be a bug in the sw :(

dreamcat4 commented 2 years ago

thanks for these details. it is helpful and i shall try my best to follow these new instructions. will come back with further details in coming days