debian-pi / raspbian-ua-netinst

Raspbian (minimal) unattended netinstaller
Other
1.17k stars 153 forks source link

is this installer still working? #538

Open oupala opened 2 years ago

oupala commented 2 years ago

It seems that something is getting wrong in installing a fresh install of raspbian-ua-netinst.

Here are the logs from raspbian-ua-netinst-<yyyymmddThhmmss>.log:

=================================================
raspbian-ua-netinst
=================================================
Release: v1.1.7
Built on 2021-07-17 23:59:49+02:00
Running on Raspberry Pi version 1
=================================================
https://github.com/debian-pi/raspbian-ua-netinst/
=================================================
Starting HWRNG succeeded!
Copying boot files... OK

Network configuration:
  ip4_addr = dhcp
  ip6_addr = disable
  online_config = 

Loading drivers.
Finished loading drivers.
Waiting for eth0... OK
Configuring eth0 for IPv4 using DHCP... 192.168.1.11/24
Time set using ntpdate to... Mon Sep 12 21:06:44 UTC 2022

Installation started at Mon Sep 12 21:06:28 UTC 2022.

Installer configuration:
  preset = server
  packages = 
  mirror = http://mirrordirector.raspbian.org/raspbian/
  release = buster
  init_system = systemd
  hostname = pi
  domainname = 
  rootpw = raspbian
  root_ssh_pubkey = 
  disable_root = 
  username = 
  userpw = 
  user_ssh_pubkey = 
  user_is_admin = 
  cdebootstrap_cmdline = --flavour=minimal --include=linux-image-rpi-rpfv,cpufrequtils,kmod,systemd-sysv,libpam-systemd,rng-tools,dosfstools,e2fsprogs,fake-hwclock,ifupdown,net-tools,ntp,openssh-server,resolvconf,vim-tiny,iputils-ping,wget,ca-certificates,rsyslog,cron,dialog,locales,less,man-db,isc-dhcp-client
  boot_volume_label = 
  bootsize = +128M
  bootoffset = 8192
  rootsize = 
  timeserver = time.nist.gov
  timezone = Etc/UTC
  locales = 
  system_default_locale = 
  cmdline = dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 elevator=deadline
  usbroot = 
  rootdev = /dev/mmcblk0
  rootpartition = /dev/mmcblk0p2
  rootfstype = ext4
  rootfs_mkfs_options = 
  rootfs_install_mount_options = noatime,data=writeback,nobarrier,noinit_itable
  rootfs_mount_options = errors=remount-ro,noatime
  final_action = reboot
  hardware_versions = 1
  gpu_mem = 

Waiting 5 seconds.....
Waiting for /dev/mmcblk0... OK
Applying new partition table... OK
Initializing /boot as vfat... OK
Copying /boot files in... OK
Model memory in MB: '463' is a low mem model: 'false'
Keeping backup copy in /bootfs
Initializing / as ext4... OK
Mounting new filesystems... OK
Starting install process...
P: Retrieving Release
E: Couldn't download Release!

Oh noes, something went wrong!

When I check the website, I can see that the raspbian mirror still exists on http://mirrordirector.raspbian.org/raspbian/dists/buster/!

What could possibly go wrong?

diederikdehaas commented 2 years ago

Can you check whether the same issue occurs with installing Bullseye?
It may be that the key(s) from raspbian-archive-keyring need to be updated. In that case it would also fail with Bullseye

diederikdehaas commented 2 years ago

If you're comfortable building the installer from source, then adding --verbose to https://github.com/debian-pi/raspbian-ua-netinst/blob/v1.1.x/scripts/etc/init.d/rcS#L1150 should give more insight as to why it is failing

oupala commented 2 years ago

I get the same behavior with bullseye.

Here are the logs from raspbian-ua-netinst-<yyyymmddThhmmss>.log:

=================================================
raspbian-ua-netinst
=================================================
Release: v1.1.7
Built on 2021-07-17 23:59:49+02:00
Running on Raspberry Pi version 1
=================================================
https://github.com/debian-pi/raspbian-ua-netinst/
=================================================
Starting HWRNG succeeded!
Copying boot files... OK
=================================================
=== Start executing installer-config.txt. ===
=== Finished executing installer-config.txt. ===
=================================================

Network configuration:
  ip4_addr = dhcp
  ip6_addr = disable
  online_config = 

Loading drivers.
Finished loading drivers.
Waiting for eth0... OK
Configuring eth0 for IPv4 using DHCP... 192.168.1.11/24
Time set using ntpdate to... Tue Sep 13 20:55:49 UTC 2022

Installation started at Tue Sep 13 20:55:32 UTC 2022.

Installer configuration:
  preset = server
  packages = 
  mirror = http://mirrordirector.raspbian.org/raspbian/
  release = bullseye
  init_system = systemd
  hostname = pi
  domainname = 
  rootpw = raspbian
  root_ssh_pubkey = 
  disable_root = 
  username = 
  userpw = 
  user_ssh_pubkey = 
  user_is_admin = 
  cdebootstrap_cmdline = --flavour=minimal --include=linux-image-rpi-rpfv,cpufrequtils,kmod,systemd-sysv,libpam-systemd,rng-tools,dosfstools,e2fsprogs,fake-hwclock,ifupdown,net-tools,ntp,openssh-server,resolvconf,vim-tiny,iputils-ping,wget,ca-certificates,rsyslog,cron,dialog,locales,less,man-db,isc-dhcp-client
  boot_volume_label = 
  bootsize = +128M
  bootoffset = 8192
  rootsize = 
  timeserver = time.nist.gov
  timezone = Etc/UTC
  locales = 
  system_default_locale = 
  cmdline = dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 elevator=deadline
  usbroot = 
  rootdev = /dev/mmcblk0
  rootpartition = /dev/mmcblk0p2
  rootfstype = ext4
  rootfs_mkfs_options = 
  rootfs_install_mount_options = noatime,data=writeback,nobarrier,noinit_itable
  rootfs_mount_options = errors=remount-ro,noatime
  final_action = reboot
  hardware_versions = 1
  gpu_mem = 

Waiting 5 seconds.....
Waiting for /dev/mmcblk0... OK
Applying new partition table... OK
Initializing /boot as vfat... OK
Copying /boot files in... OK
Model memory in MB: '463' is a low mem model: 'false'
Keeping backup copy in /bootfs
Initializing / as ext4... OK
Mounting new filesystems... OK
Starting install process...
P: Retrieving Release
E: Couldn't download Release!

Oh noes, something went wrong!

I don't feel confortable enough to build de debug installer (but that could be an idea to release a debug installer in addition to a default installer).

Would it be long to release an installer with an updated key from raspbian-archive-keyring?

raspbian has recently been renamed raspberry pi os. Does this change has an impact on the install process?

diederikdehaas commented 2 years ago

I build an installer with the --verbose flag and did an install and that succeeded.

# grep -A30 "Starting install process..." /var/log/raspbian-ua-netinst.log 
Starting install process...
P: Retrieving Release
P: Retrieving Release.gpg
P: Validating Release
gpgv: Signature made Wed Sep 14 10:23:01 2022 UTC

gpgv:                using RSA key A0DA38D0D76E8B5D638872819165938D90FDDD2E

I: Good signature from "Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <mpthompson@gmail.com>"
gpgv: Good signature from "Mike Thompson (Raspberry Pi Debian armhf ARMv6+VFP) <mpthompson@gmail.com>"

P: Parsing Release
P: Retrieving Packages.xz
P: Validating Packages.xz
P: Parsing Packages
P: Retrieving gcc-8-base
P: Validating gcc-8-base
...

Could you try again? It looks like just a temporary failue.

And if it fails again, could you create an installer-config.txt with bootsize=+256M and try it with that? This setting shouldn't matter (for this issue), but it's the only change I made which might be relevant

oupala commented 2 years ago

I'll try that.

Is it possible for you to share the installer with the --debug flag?

diederikdehaas commented 2 years ago

Max attachment size (here) is 25MB, but the installer is 38MB. So it looks like that isn't going to work

oupala commented 2 years ago

You can share it on plik.

diederikdehaas commented 2 years ago

https://plik.root.gg/file/Q5UHLgVy1hES7ujB/ykZ4cDggoTD1jLxD/raspbian-ua-netinst-20220914-gita820581.img.xz

oupala commented 2 years ago

Is there a way to enable debugging? I still have the same error, and no additional verbose log.

Here are the logs from raspbian-ua-netinst-<yyyymmddThhmmss>.log:

=================================================
raspbian-ua-netinst
=================================================
Revision: git~a820581 (Also put 'options inet6' on its own line., 2021-08-30)
Built on 2022-09-14 13:39:40+02:00
Running on Raspberry Pi version 1
=================================================
https://github.com/debian-pi/raspbian-ua-netinst/
=================================================
Starting HWRNG succeeded!
Copying boot files... OK

Network configuration:
  ip4_addr = dhcp
  ip6_addr = disable
  online_config = 

Loading drivers.
Finished loading drivers.
Waiting for eth0... OK
Configuring eth0 for IPv4 using DHCP... 192.168.1.11/24
Time set using ntpdate to... Wed Sep 14 20:08:23 UTC 2022

Installation started at Wed Sep 14 20:08:07 UTC 2022.

Installer configuration:
  preset = server
  packages = 
  mirror = http://mirrordirector.raspbian.org/raspbian/
  release = buster
  init_system = systemd
  hostname = pi
  domainname = 
  rootpw = raspbian
  root_ssh_pubkey = 
  disable_root = 
  username = 
  userpw = 
  user_ssh_pubkey = 
  user_is_admin = 
  cdebootstrap_cmdline = --flavour=minimal --include=linux-image-rpi-rpfv,cpufrequtils,kmod,systemd-sysv,libpam-systemd,rng-tools,dosfstools,e2fsprogs,fake-hwclock,ifupdown,net-tools,ntp,openssh-server,resolvconf,vim-tiny,iputils-ping,wget,ca-certificates,rsyslog,cron,dialog,locales,less,man-db,isc-dhcp-client
  boot_volume_label = 
  bootsize = +128M
  bootoffset = 8192
  rootsize = 
  timeserver = time.nist.gov
  timezone = Etc/UTC
  locales = 
  system_default_locale = 
  cmdline = dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 elevator=deadline
  usbroot = 
  rootdev = /dev/mmcblk0
  rootpartition = /dev/mmcblk0p2
  rootfstype = ext4
  rootfs_mkfs_options = 
  rootfs_install_mount_options = noatime,data=writeback,nobarrier,noinit_itable
  rootfs_mount_options = errors=remount-ro,noatime
  final_action = reboot
  hardware_versions = 1
  gpu_mem = 

Waiting 5 seconds.....
Waiting for /dev/mmcblk0... OK
Applying new partition table... OK
Initializing /boot as vfat... OK
Copying /boot files in... OK
Model memory in MB: '463' is a low mem model: 'false'
Keeping backup copy in /bootfs
Initializing / as ext4... OK
Mounting new filesystems... OK
Starting install process...
P: Retrieving Release
E: Couldn't download Release!

Oh noes, something went wrong!

I will now make another try with an installer-config.txt with bootsize=+256M.

oupala commented 2 years ago

Exactly the same result with an installer-config.txt with bootsize=+256M.

Here are the logs from raspbian-ua-netinst-<yyyymmddThhmmss>.log:


=================================================
raspbian-ua-netinst
=================================================
Revision: git~a820581 (Also put 'options inet6' on its own line., 2021-08-30)
Built on 2022-09-14 13:39:40+02:00
Running on Raspberry Pi version 1
=================================================
https://github.com/debian-pi/raspbian-ua-netinst/
=================================================
Starting HWRNG succeeded!
Copying boot files... OK
=================================================
=== Start executing installer-config.txt. ===
=== Finished executing installer-config.txt. ===
=================================================

Network configuration:
  ip4_addr = dhcp
  ip6_addr = disable
  online_config = 

Loading drivers.
Finished loading drivers.
Waiting for eth0... OK
Configuring eth0 for IPv4 using DHCP... 192.168.1.11/24
Failed to set time using ntpdate!
Failed to set time using timeserver 'time.nist.gov'.
Failed to set time using timeserver 'time.nist.gov'.
Failed to set time using timeserver 'nist1.symmetricom.com'.
Failed to set time using timeserver 'nist-time-server.eoni.com'.
Failed to set time using timeserver 'utcnist.colorado.edu'.
Failed to set time using timeserver 'nist1-pa.ustiming.org'.
Failed to set time using timeserver 'nist.expertsmi.com'.
Failed to set time using timeserver 'nist1-macon.macon.ga.us'.
Failed to set time using timeserver 'wolfnisttime.com'.
Failed to set time using timeserver 'nist.time.nosc.us'.
Failed to set time using timeserver 'nist.netservicesgroup.com'.
Failed to set time using timeserver 'nisttime.carsoncity.k12.mi.us'.
Failed to set time using timeserver 'nist1-lnk.binary.net'.
Failed to set time using timeserver 'ntp-nist.ldsbc.edu'.
Failed to set time using timeserver 'utcnist2.colorado.edu'.
Failed to set time using timeserver 'nist1-ny2.ustiming.org'.
Failed to set time using timeserver 'wwv.nist.gov'.
FAILED to set the date, so things are likely to fail now ...
Make sure that rdate port 37 is not blocked by your firewall.

Installation started at Thu Jan  1 00:00:00 UTC 1970.

Installer configuration:
  preset = server
  packages = 
  mirror = http://mirrordirector.raspbian.org/raspbian/
  release = buster
  init_system = systemd
  hostname = pi
  domainname = 
  rootpw = raspbian
  root_ssh_pubkey = 
  disable_root = 
  username = 
  userpw = 
  user_ssh_pubkey = 
  user_is_admin = 
  cdebootstrap_cmdline = --flavour=minimal --include=linux-image-rpi-rpfv,cpufrequtils,kmod,systemd-sysv,libpam-systemd,rng-tools,dosfstools,e2fsprogs,fake-hwclock,ifupdown,net-tools,ntp,openssh-server,resolvconf,vim-tiny,iputils-ping,wget,ca-certificates,rsyslog,cron,dialog,locales,less,man-db,isc-dhcp-client
  boot_volume_label = 
  bootsize = +128M
  bootoffset = 8192
  rootsize = 
  timeserver = time.nist.gov
  timezone = Etc/UTC
  locales = 
  system_default_locale = 
  cmdline = dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 elevator=deadline
  usbroot = 
  rootdev = /dev/mmcblk0
  rootpartition = /dev/mmcblk0p2
  rootfstype = ext4
  rootfs_mkfs_options = 
  rootfs_install_mount_options = noatime,data=writeback,nobarrier,noinit_itable
  rootfs_mount_options = errors=remount-ro,noatime
  final_action = reboot
  hardware_versions = 1
  gpu_mem = 

Waiting 5 seconds.....
Waiting for /dev/mmcblk0... OK
Applying new partition table... OK
Initializing /boot as vfat... OK
Copying /boot files in... OK
Model memory in MB: '463' is a low mem model: 'false'
Keeping backup copy in /bootfs
Initializing / as ext4... OK
Mounting new filesystems... OK
Starting install process...
P: Retrieving Release
E: Couldn't download Release!

Oh noes, something went wrong!
oupala commented 2 years ago

I'm having some doubt about the hardware: maybe the raspberry board, or the power supply.

But it is strange because I successfully installed raspberry pi os...

I need some further testing...

Any suggestion is more than welcomed!

diederikdehaas commented 2 years ago

failed to set time using ntpdate!

(and the other time setting failures) indicate a network problem and then it makes sense that the network-installer also fails.
But on the other attempts, setting the time did succeed...

Is there a way to enable debugging?

cdebootstrap also has a --debug flag

I'm having some doubt about the hardware

Why do you think that? I've seen no indication of that. It really looks like a network failure when retrieving the Release file, but I don't understand why that would fail, when retrieving the time worked (most of the attempts).

diederikdehaas commented 2 years ago

https://plik.root.gg/file/TrAKauSxoT90RJ5Q/s0MJfjdDtl7Y52Tg/raspbian-ua-netinst-20220915-gita820581.img.xz is a build with --debug

oupala commented 2 years ago

Thanks for all the effort that you do to provide me with debug flag.

I think it is now time to hold this issue because I need more time to invetigate.

I changed the SD card and the USB power supply and it appears that I succeded in installing raspbian.

We should maybe read "E: Couldn't download Release!" as "E: Cannot write to downloaded release to the SD card!".

In fact, now that I know that the problem does not come from the installer, and now that I have an installer with the debug flag enabled, I need more time to find out what was the root cause of the problem: USB power supply? corrupted SD card? failure on the router? network interruption during install?

Once I get more information, I'll come here to tell you what was happening.