PREP_SYSTEM_FOR_DIETPI.sh | Automate

[Fourdee]

Status: Beta

What is this?

• This script will install and convert any 'bloated' Debian/Raspbian installation, into a lightweight DietPi system.

What this script does NOT do:

• Support converting existing installed software (eg: nextcloud, Plex Media Server) over to the DietPi system. All existing software (APT) and user data will be deleted.

---

Step 1: Ensure a Debian/Raspbian OS is running on the system

• For best results, we recommend a fresh/clean minimal Debian/Raspbian installation
• Native PC users: please install Debian stable before hand: https://www.debian.org/distrib/netinst
• Desktop images should work, however, the minimal the image, the quicker the installation, as less packages will need to be removed.
• NB: We do not support Ubuntu, or have any plans to do so.

Step 2: Pre-req packages

• Critical packages These should already exist on most systems, however pure minimal images may require the following installations) apt-get update; apt-get install -y systemd-sysv ca-certificates sudo wget locales --reinstall

Step 3: Run DietPi installer

• Must be run from SSH or local term eg: Outside of desktop environment
• In case of SSH, to prevent connection loss, an Ethernet connection is required.

Ensure you have elevated privileges (eg: login as root, or use sudo su).

Copy and paste all into term.

wget https://raw.githubusercontent.com/MichaIng/DietPi/master/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/MichaIng/DietPi/jessie-support/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/MichaIng/DietPi/beta/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
#wget https://raw.githubusercontent.com/MichaIng/DietPi/dev/PREP_SYSTEM_FOR_DIETPI.sh -O PREP_SYSTEM_FOR_DIETPI.sh
chmod +x PREP_SYSTEM_FOR_DIETPI.sh
./PREP_SYSTEM_FOR_DIETPI.sh

Follow the onscreen prompts.

[MichaIng]

Just in theory: If we really get this automated secure for at least some devices, this means we do not necessarily need to create images for those. We could just provide the preparation script to upgrade any Debian system into DietPi. By use of apt-mark, as suggested here: https://github.com/Fourdee/DietPi/pull/1266, we would be sure that possibly disturbing software is removed and provide our own config files/adjustments for our dependencies. Kernel/bootleader apt packages are generally excluded from autoremove on every device I checked. As running systems are, yeah running, we could extract boot critical information (especially fstab/boot.ini/network details) from the running system in case and create our configs around those.

Hehe just a little optimistic view into the feature 😆.

[Fourdee]

@MichaIng

We could just provide the preparation script to upgrade any Debian system into DietPi. As running systems are, yeah running, we could extract boot critical information (especially fstab/boot.ini/network details) from the running system in case and create our configs around those. Hehe just a little optimistic view into the feature

I like where this is going 👍

[Fourdee]

@MichaIng

I've not looked into manual holding packages yet. I'am simply converting the old script into automated. Then we can work on it.

[MichaIng]

@Fourdee Great work so far, like the idea of setting options by whiptail, first steps into end user direction 😄.

Just consider: https://github.com/Fourdee/DietPi/pull/1266

If you generally like the idea, I will recreate the PR for the automated script. Other small adjustments based on https://github.com/Fourdee/DietPi/issues/1219#issuecomment-347734540 and following as well.

Just tell, after you implemented you ideas, then I will create PR.

[Fourdee]

~Ok, so the new package removal system:~

Scraped, in favor of @MichaIng APT removal method.

• ~We provide a list of essential minimal packages that must be installed for DietPi~
• ~The script then finds all the dependencies for each package we need and stores it into an array~
• ~Package removal list, is generated by comparing the currently installed, to the array we generated. If any packages exist that we dont need, DietPi will remove them.~

~This allows us to exclude the old system of manually removing "known" packages we dont want. Instead, having everything generated automatically, based on the current installed system.~

/DietPi/dietpi/PREP_SYSTEM_FOR_DIETPI.sh                         

 [Ok] Generating list of minimal packages required for DietPi installation

 [Ok] Obtaining list of currently installed packages

 [Info] (0): Passed

 [Ok] Generating a list of deps, required for the DietPi packages
This may take some time, please wait...

Checking deps: acl
 - Adding deps: libacl1
 - Adding deps: libattr1
 - Adding deps: libc6
Checking deps: adduser
 - Adding deps: passwd
 - Adding deps: debconf
Checking deps: apt
 - Adding deps: adduser
 - Adding deps: gpgv
 - Adding deps: debian-archive-keyring
 - Adding deps: init-system-helpers
 - Adding deps: libapt-pkg5.0
 - Adding deps: libgcc1
 - Adding deps: libstdc++6
Checking deps: apt-transport-https
 - Adding deps: libcurl3-gnutls
Checking deps: apt-utils
 - Adding deps: apt
 - Adding deps: libapt-inst2.0
 - Adding deps: libdb5.3
Checking deps: base-files
Checking deps: base-passwd
 - Adding deps: libdebconfclient0
Checking deps: bash
 - Adding deps: dash
 - Adding deps: libtinfo5
 - Adding deps: base-files
 - Adding deps: debianutils
Checking deps: bash-completion
 - Adding deps: dpkg
 - Adding deps: bash
Checking deps: bc
 - Adding deps: libreadline7
Checking deps: bsdmainutils
 - Adding deps: libbsd0
 - Adding deps: libncurses5
 - Adding deps: bsdutils
Checking deps: bsdutils
 - Adding deps: libsystemd0
Checking deps: bzip2
 - Adding deps: libbz2-1.0
Checking deps: ca-certificates
 - Adding deps: openssl
Checking deps: console-common
 - Adding deps: console-data
 - Adding deps: kbd
 - Adding deps: lsb-base
Checking deps: console-data
Checking deps: console-setup
 - Adding deps: console-setup-linux
 - Adding deps: console-setup-freebsd
 - Adding deps: xkb-data
 - Adding deps: keyboard-configuration
Checking deps: console-setup-linux
 - Adding deps: initscripts
Checking deps: coreutils
 - Adding deps: libselinux1
Checking deps: cpio
Checking deps: crda
 - Adding deps: libnl-3-200
 - Adding deps: libnl-genl-3-200
 - Adding deps: libssl1.1
 - Adding deps: wireless-regdb
 - Adding deps: iw
Checking deps: cron
 - Adding deps: libpam0g
 - Adding deps: libpam-runtime
Checking deps: curl
 - Adding deps: libcurl3
 - Adding deps: zlib1g
Checking deps: dash
Checking deps: dbus
 - Adding deps: libapparmor1
 - Adding deps: libaudit1
 - Adding deps: libcap-ng0
 - Adding deps: libdbus-1-3
 - Adding deps: libexpat1
Checking deps: debconf
 - Adding deps: perl-base
Checking deps: debian-archive-keyring
Checking deps: debianutils
 - Adding deps: sensible-utils
Checking deps: diffutils
Checking deps: dmidecode
Checking deps: dmsetup
 - Adding deps: libdevmapper1.02.1
Checking deps: dosfstools
 - Adding deps: libudev1
Checking deps: dphys-swapfile
 - Adding deps: dc
Checking deps: dpkg
 - Adding deps: liblzma5
 - Adding deps: tar
Checking deps: e2fslibs
Checking deps: e2fsprogs
 - Adding deps: e2fslibs
 - Adding deps: libblkid1
 - Adding deps: libcomerr2
 - Adding deps: libss2
 - Adding deps: libuuid1
 - Adding deps: util-linux
Checking deps: ethtool
Checking deps: fake-hwclock
Checking deps: fbset
 - Adding deps: udev
 - Adding deps: makedev
Checking deps: findutils
Checking deps: firmware-atheros
Checking deps: firmware-brcm80211
Checking deps: firmware-ralink
Checking deps: firmware-realtek
Checking deps: fuse
 - Adding deps: libfuse2
 - Adding deps: mount
 - Adding deps: sed
Checking deps: gnupg
 - Adding deps: gnupg-agent
 - Adding deps: libassuan0
 - Adding deps: libgcrypt20
 - Adding deps: libgpg-error0
 - Adding deps: libksba8
 - Adding deps: libsqlite3-0
Checking deps: gpgv
Checking deps: grep
 - Adding deps: libpcre3
 - Adding deps: install-info
Checking deps: groff-base
Checking deps: gzip
Checking deps: hdparm
Checking deps: hfsplus
 - Adding deps: libhfsp0
Checking deps: hostname
Checking deps: htop
 - Adding deps: libncursesw5
Checking deps: ifupdown
 - Adding deps: iproute2
Checking deps: init
 - Adding deps: systemd-sysv
 - Adding deps: sysvinit-core
Checking deps: init-system-helpers
Checking deps: initramfs-tools
 - Adding deps: initramfs-tools-core
 - Adding deps: linux-base
Checking deps: initscripts
 - Adding deps: sysvinit-utils
 - Adding deps: sysv-rc
 - Adding deps: file-rc
 - Adding deps: openrc
 - Adding deps: coreutils
Checking deps: insserv
Checking deps: iproute2
 - Adding deps: libelf1
 - Adding deps: libmnl0
Checking deps: iputils-ping
 - Adding deps: libcap2
 - Adding deps: libidn11
 - Adding deps: libnettle6
Checking deps: isc-dhcp-client
 - Adding deps: libdns-export162
 - Adding deps: libisc-export160
Checking deps: isc-dhcp-common
Checking deps: iw
Checking deps: kbd
Checking deps: keyboard-configuration
 - Adding deps: liblocale-gettext-perl
Checking deps: klibc-utils
 - Adding deps: libklibc
Checking deps: kmod
 - Adding deps: libkmod2
Checking deps: less
Checking deps: locales
 - Adding deps: libc-bin
 - Adding deps: libc-l10n
Checking deps: login
 - Adding deps: libpam-modules
Checking deps: lsb-base
Checking deps: mawk
Checking deps: mount
 - Adding deps: libmount1
 - Adding deps: libsmartcols1
Checking deps: multiarch-support
Checking deps: nano
Checking deps: ncurses-base
Checking deps: ncurses-bin
Checking deps: net-tools
Checking deps: ntfs-3g
 - Adding deps: fuse
 - Adding deps: libgnutls30
 - Adding deps: libntfs-3g871
Checking deps: ntp
 - Adding deps: netbase
 - Adding deps: libedit2
 - Adding deps: libopts25
Checking deps: p7zip-full
 - Adding deps: p7zip
Checking deps: parted
 - Adding deps: libparted2
Checking deps: passwd
 - Adding deps: libsemanage1
Checking deps: perl-base
Checking deps: procps
 - Adding deps: libprocps6
Checking deps: psmisc
Checking deps: readline-common
Checking deps: resolvconf
 - Adding deps: ifupdown
Checking deps: sed
Checking deps: sensible-utils
Checking deps: startpar
Checking deps: sudo
Checking deps: systemd
 - Adding deps: libcryptsetup4
 - Adding deps: libip4tc0
 - Adding deps: liblz4-1
 - Adding deps: libseccomp2
 - Adding deps: procps
Checking deps: systemd-sysv
 - Adding deps: systemd
Checking deps: sysvinit-utils
Checking deps: tar
Checking deps: tzdata
Checking deps: udev
Checking deps: unzip
Checking deps: usbutils
 - Adding deps: libusb-1.0-0
Checking deps: util-linux
 - Adding deps: libfdisk1
Checking deps: wget
 - Adding deps: libpsl5
Checking deps: whiptail
 - Adding deps: libnewt0.52
 - Adding deps: libpopt0
 - Adding deps: libslang2
Checking deps: wireless-regdb
Checking deps: wireless-tools
 - Adding deps: libiw30
Checking deps: wpasupplicant
 - Adding deps: libpcsclite1
 - Adding deps: libssl1.0.2
Checking deps: wput
 - Adding deps: libgnutls-openssl27
Checking deps: zip
Checking deps: intel-microcode
 - Adding deps: iucode-tool
Checking deps: amd64-microcode
Checking deps: firmware-linux-nonfree
 - Adding deps: firmware-misc-nonfree
 - Adding deps: firmware-amd-graphics

 [Ok] Generating a list of packages, not required by DietPi, to be removed from system.
This may take some time, please wait...

 [Info] The following packages will be removed
adwaita-icon-theme alsa-utils bind9-host binutils build-essential checkinstall cifs-utils cmake cmake-data cpp cpp-6 cryptsetup cryptsetup-bin dh-python dhcpcd5 dialog discover discover-data distro-info-data dnsmasq dnsmasq-base dnsutils dropbear dropbear-bin dropbear-initramfs dropbear-run efibootmgr fakeroot file fontconfig fontconfig-config fonts-dejavu-core g++ g++-6 galculator gcc gettext-base git git-man glib-networking:amd64 glib-networking-common glib-networking-services gparted gpicview grub2-common gsettings-desktop-schemas gtk-update-icon-cache hicolor-icon-theme installation-report iptables irqbalance laptop-detect leafpad light-locker lightdm lightdm-gtk-greeter lighttpd linux-libc-dev:amd64 lsb-release lsof lxappearance lxappearance-obconf lxde lxde-common lxde-core lxde-icon-theme lxhotkey-core lxhotkey-gtk lxinput lxlock lxmenu-data lxpanel lxpanel-data lxpolkit lxrandr lxsession lxsession-data lxsession-edit lxsession-logout lxterminal make manpages manpages-dev mime-support mysql-common netcat netcat-traditional openbox openbox-lxde-session os-prober patch pciutils pcmanfm perl perl-modules-5.24 php-apcu php-apcu-bc php-cgi php-common php-curl php-fpm php-gd php-mbstring php-mcrypt php-xml php-zip php7.0-cgi php7.0-cli php7.0-common php7.0-curl php7.0-fpm php7.0-gd php7.0-json php7.0-mbstring php7.0-mcrypt php7.0-opcache php7.0-readline php7.0-xml php7.0-zip pinentry-curses policykit-1 python python-minimal python-talloc python2.7 python2.7-minimal python3 python3-minimal python3.5 python3.5-minimal samba-common samba-libs:amd64 sgml-base shared-mime-info smbclient spawn-fcgi sysbench tcl tcl8.6 tigervnc-common tigervnc-standalone-server tk tk8.6 ucf upower vnc4server wamerican x11-common x11-utils x11-xkb-utils x11-xserver-utils x11vnc x11vnc-data xarchiver xauth xbitmaps xcompmgr xdg-user-dirs xfonts-base xfonts-encodings xfonts-utils xinit xml-core xserver-common xserver-xorg xserver-xorg-core xserver-xorg-input-all xserver-xorg-input-libinput xserver-xorg-video-all xserver-xorg-video-amdgpu xserver-xorg-video-ati xserver-xorg-video-fbdev xserver-xorg-video-nouveau xserver-xorg-video-radeon xserver-xorg-video-vesa xserver-xorg-video-vmware xterm xz-utils zerofree

 [Info] The following packages will be installed
acl adduser apt apt-transport-https apt-utils base-files base-passwd bash bash-completion bc bsdmainutils bsdutils bzip2 ca-certificates console-common console-data console-setup console-setup-linux coreutils cpio crda cron curl dash dbus debconf debian-archive-keyring debianutils diffutils dmidecode dmsetup dosfstools dphys-swapfile dpkg e2fslibs e2fsprogs ethtool fake-hwclock fbset findutils firmware-atheros firmware-brcm80211 firmware-ralink firmware-realtek fuse gnupg gpgv grep groff-base gzip hdparm hfsplus hostname htop ifupdown init init-system-helpers initramfs-tools initscripts insserv iproute2 iputils-ping isc-dhcp-client isc-dhcp-common iw kbd keyboard-configuration klibc-utils kmod less locales login lsb-base mawk mount multiarch-support nano ncurses-base ncurses-bin net-tools ntfs-3g ntp p7zip-full parted passwd perl-base procps psmisc readline-common resolvconf sed sensible-utils startpar sudo systemd systemd-sysv sysvinit-utils tar tzdata udev unzip usbutils util-linux wget whiptail wireless-regdb wireless-tools wpasupplicant wput zip intel-microcode amd64-microcode firmware-linux-nonfree

[Fourdee]

Buster notes:

• I've disabled option for now.

RPI:

W: The repository 'https://archive.raspberrypi.org/debian buster Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch https://archive.raspberrypi.org/debian/dists/buster/main/binary-armhf/Packages  404  Not Found

C2:

W: The repository 'http://ftp.debian.org/debian buster-backports Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch http://ftp.debian.org/debian/dists/buster-backports/main/binary-arm64/Packages  404  Not Found [IP: 130.89.148.12 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.

[Fourdee]

Note to self:

• Check over: https://github.com/Fourdee/DietPi/issues/1219#issuecomment-347734540

ToDo:

• Automate TZ/Locale??? Had issues with this in the past. Need to research it again.
• Add ARMbian linux kernel to install list, or, add a general "do not remove" linux-* to script?

[Fourdee]

Ready for early internal testing to fix bugs and improve.

Not stable enough for public users, yet.

Tested:

• 🈯️ Odroid C2 Stretch

[MichaIng]

@Fourdee

Buster notes:

I've disabled option for now.

• Yes, the raspberrypi.org repo was always behind: https://archive.raspberrypi.org/debian/dists/ I remember stretch distro to appear there just around it became release for raspbian, long after it became release for debian and long after I switched already to stretch with my RPi. Jessie distro worked fine there, as it mainly provides just firmware/kernel/bootloader. I will upgrade my RPi to buster after doing some tests on my buster VM to then tell more reliably if buster + raspberrypi.org stretch repos work together. We need to further implement buster for this into DietPi, but let's first get the automated script ready for public testing.
• About buster debian repo: Of course no buster-backport exists, as it is already the testing repo with up-to-date packages 😆: http://ftp.debian.org/debian/dists/ Okay sid also exists, but so far this looks just like a total experimental developer playground, with also many old (pre-)jessie packages (like php5...) inside. So for this, we can just comment out buster-backports and choose the main repo always inside our software scripts.
• About debian repo generally: Found this: https://deb.debian.org/ It is already default on new debian installations and seems to be kind of mirror director. Works well, even is reachable via https, as the link shows. Only on Jessie I got the following errors on apt update, if https was used. With http everything works fine:

  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_main_binary-amd64_Packages.diff_Index is not what the server reported 9376 345
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_non-free_binary-amd64_Packages.diff_Index is not what the server reported 736 349
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_main_i18n_Translation-en.diff_Index is not what the server reported 3688 343
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-updates_non-free_i18n_Translation-en.diff_Index is not what the server reported 736 347
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_main_binary-amd64_Packages.diff_Index is not what the server reported 27796 347
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_contrib_binary-amd64_Packages.diff_Index is not what the server reported 24964 15799
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_non-free_binary-amd64_Packages.diff_Index is not what the server reported 22858 351
  W: Size of file /var/lib/apt/lists/partial/deb.debian.org_debian_dists_jessie-backports_main_i18n_Translation-en.diff_Index is not what the server reported 27796 345

• Found just a few entries about this in g***le, seems to be known as error with apt+https on Jessie: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802617)
• I suggest we use this also as default source, with https for stretch (and buster)? As can be seen, even security updates are inside: https://deb.debian.org/debian-security/dists/

[Fourdee]

@MichaIng

Legend 👍

In regards to mirror:

root@DietPi:~# cat /etc/apt/sources.list
deb http://deb.debian.org/debian stretch main contrib non-free
deb http://deb.debian.org/security stretch main contrib non-free
deb http://ftp.debian.org/debian/ stretch-backports main contrib non-free

1st run

Ign:28 http://deb.debian.org/security stretch/non-free Translation-en
Err:14 http://deb.debian.org/security stretch/main amd64 Packages
  404  Not Found [IP: 151.101.0.204 80]
Ign:15 http://deb.debian.org/security stretch/main all Packages

2nd run

Ign:20 http://deb.debian.org/security stretch/non-free Translation-en_GB
Err:6 http://deb.debian.org/security stretch/main i386 Packages
  404  Not Found [IP: 151.101.0.204 80]
Ign:7 http://deb.debian.org/security stretch/main amd64 Packages

This is why I dislike using mirror services. They are unreliable.

[Fourdee]

Moved:

@Fourdee About the dependency list:

• I just wrote a little script that shows installed dependants of all installed packages. With this we can further reduce our dependency list to the final packages that are really directly used by dietpi / usual system operations. Here your dependency list where I added 🈯️ > dependant, if one within our anyway installed packages exist: 'acl' 🈺 > systemd on Jessie, but not on Stretch+Buster, so we really need this, getfacl or setfacl 'adduser' 🈯️ > systemd+apt+cron+udev+several other packages, depending on distro 'apt' 🈯️ > apt-utils+apt-transport-https 'apt-transport-https' 'apt-utils' 'base-files' 🈯️ > bash 'base-passwd' 🈯️ essential package will be never autoremoved 'bash' 🈯️ > bash-completion + essential 'bash-completion' 'bc' 'bsdmainutils' 🈺 Do we need this? Was not in the list before! 'bsdutils' 🈯️ essential 'bzip2' 'ca-certificates' 'console-common' 🈺 console-setup supersedes or depends on other console related packages 'console-data' 🈺 console-setup supersedes or depends on other console related packages 'console-setup' 'console-setup-linux' 🈺 console-setup supersedes or depends on other console related packages 'coreutils' 🈯️ essential 'cpio' 🈯️ (> initramfs-tools-core) > initramfs-tools 'crda' 'cron' 'curl' 'dash' 🈯️ > bash > bash-completion 'dbus' 'debconf' 🈯️ > debconf-utils 'debian-archive-keyring' 🈯️ > apt and wrong on Raspbian, where raspbian-archive-keyring takes place! 'debianutils' 🈯️ > cron+bash+dash+some others, depending on distro 'diffutils' 🈯️ essential 'dmidecode' 🈺 Do we need this? Was not in the list before! 'dmsetup' 🈯️ > > > parted 'dosfstools' 🈺 Could be left out on VMs, as we do not have fat boot partition there. 'dphys-swapfile' 'dpkg' 🈯️ > cron+bash-completion+others+anyway essential 'e2fslibs' 🈯️ > e2fsprogs 'e2fsprogs' 🈯️ essential 'ethtool' 'fake-hwclock' 'fbset' 'findutils' 🈯️ essential 'firmware-atheros' 🈺 was x86 only before? 'firmware-brcm80211' 🈺 was x86 only before? 'firmware-ralink' 🈺 was x86 only before? 'firmware-realtek' 🈺 was x86 only before? 'firmware-misc-nonfree' 🈺 was x86 only before? 'fuse' 🈺 Didn't see this before, do we need it? https://packages.debian.org/de/stretch/fuse 'gnupg' 🈺 We used this before, but gpgv seems to be a lightweight version of it. 'gpgv' 🈺 Superseded by gnupg, but more lightweight, maybe we need to switch? 'grep' 🈯️ essential 'groff-base' 🈺 Do we need this? Was not in the list before! 'gzip' 🈯️ essential 'hdparm' 'hfsplus' 'hostname' 🈯️ essential 'htop' 'ifupdown' 🈯️ > resolvconf 'init' 🈯️ essential 'init-system-helpers' 🈯️ > init and essential 'initramfs-tools' 'initscripts' 🈺 Was dependency of systemd on Jessie, but since Stretch not anymore. I guess systemd now supersedes that package. 'insserv' 🈺 same than initscripts, don't have any of them on my systems (besides Jessie VM) and absolutely no issues. 'iproute2' 🈯️ > ifupdown > resolvconf 'iputils-ping' 🈺 Helpful indeed sometimes, but not necessary? Also superseded by curl, wget etc? 'isc-dhcp-client' 'isc-dhcp-common' 🈯️ This is just man pages! 'iw' 'kbd' 🈺 console-setup supersedes or depends on other console related packages 'keyboard-configuration' 🈺 console-setup supersedes or depends on other console related packages 'klibc-utils' 🈯️ initramfs-tools-core > initramfs-tools 'kmod' 🈯️ (> initramfs-tools-core) > initramfs-tools 'less' 🈺 Helpful indeed sometimes, but not necessary? 'locales' 'login' 🈯️ essential 'lsb-base' 🈯️ > cron+udev+swapfile+resolvconf+others 'mawk' 🈯️ > dash > bash 'mount' 🈯️ > systemd 'multiarch-support' 🈯️ > hundrets of libs, also e2fslibs > e2fsprogs, which is essential 'nano' not necessary, but yeah I use this several times a day, everybody needs a simple editor! 'ncurses-base' 🈯️ essential 'ncurses-bin' 🈯️ essential 'net-tools' 'ntfs-3g' 'ntp' 'p7zip-full' 'parted' 'passwd' 🈯️ > adduser 'perl-base' 🈯️ > init-system-helpers+debconf > debconf-utils 'procps' 🈯️ > udev+systemd 'psmisc' 'readline-common' 🈯️ > parted+gnu(pg) 'resolvconf' 'rfkill' #Used by some onboard WiFi chipsets 'rsync' 🈯️ Not necessary here, as it will be installed on demand by dietpi-backup/sync 'sed' 🈯️ essential 'sensible-utils' 🈺 Dependency of debianutils until buster. Just wondered that it is not autoremoved on my buster VM (marked as auto), but no dependency anymore and not essential. Just purged it on my Buster VM without any warnings or additional removed packages. Let's see what happens 😄. 'startpar' 🈺 Was dependency of sysv-rc > systemd on Jessie, but since Stretch, systemd does not depend on sysv-rc anymore. Seems to solve the tasks differently now (same as insserv)? 'sudo' 'systemd' 🈯️ > systemd-sysv > init 'systemd-sysv' 🈯️ > init 'sysvinit-utils' 🈯️ essential 'tar' 🈯️ essential 'tzdata' 'udev' 🈯️ > initramfs-tools-core > initramfs-tools 'unzip' 'usbutils' 'util-linux' 🈯️ > udev+systemd+others+essential 'wget' 'whiptail' 'wireless-regdb' 🈯️ > crda 'wireless-tools' 'wpasupplicant' 'wput' 'zip'

By removing the green ones from the list, there should be no additional packages removed. By removing also some of the orange ones, you get the list I used in my PR: https://github.com/Fourdee/DietPi/pull/1266/files#diff-e9e0c6c64b4937739e13ffac58f4a888R74

[MichaIng]

@Fourdee

Err:14 http://deb.debian.org/security stretch/main amd64 Packages 404 Not Found [IP: 151.101.0.204 80]

Hehe this is just the wrong URL, use: deb https://deb.debian.org/debian-security/ <distro>/updates main contrib non-free

---

Going on through the script:

• On RPi you just need raspberrypi-sys-mods, raspi-copies-and-fills, libraspberrypi-bin, everything else is solved via dependencies.
• https://github.com/Fourdee/DietPi/blob/testing/PREP_SYSTEM_FOR_DIETPI.sh#L572 dosfstools is already standard dependency.

  ---

But yeah everything not much more than cosmetic. I can do PR if you want. Otherwise REALLY great work!! 👍 👍 👍 Also found the new fstab creation. Looks really professional! I will test the script tomorrow on my VMs.

---

If we are already touching this:

• Perhaps we could also allow choosing whether to install wifi related packages, like for bt. We could actually move them (Wifi and Bt) into an own software stack to allow removing and installing them.
• Would love to see nfs/samba/ftp log files and mount folders created on demand.

But both definitely no high priority 😉

[Fourdee]

@MichaIng

Epic list https://github.com/Fourdee/DietPi/issues/1285#issuecomment-351888843, many thanks for that 👍

---

In regards to relying on deps to pull in packages we manually listed (eg: sed) In general, there is no harm is us manually listing additional packages that DietPi uses in its scripts, instead of relying on other packages to pull it in.

Benefits of listing (example sed):

• We can see what packages DietPi fully requires/uses to function
• By not relying on deps, if Debian were to change sed essential deps overnight, no risk of breaking script.

---

Firmware:

Covers us for all WiFi chipsets etc across all devices. We really should have it in for all images.

firmware-misc-nonfree can be x86_64 only, mostly intel/nvidia stuff https://packages.debian.org/stretch/firmware-misc-nonfree

---

Fuse

Required for ntfs-3g https://packages.debian.org/de/stretch/ntfs-3g:

root@DietPi:~# apt-get purge fuse
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  fuse* ntfs-3g*
0 upgraded, 0 newly installed, 2 to remove and 1 not upgraded.
After this operation, 1,241 kB disk space will be freed.
Do you want to continue? [Y/n]

[Fourdee]

with changes, quick test on DietPi RPi with only openssh-server installed:

The following packages will be REMOVED:
  bluetooth* bluez* bluez-firmware* bsdmainutils* busybox* console-common*
  console-data* cryptsetup* cryptsetup-bin* debconf-utils* dmidecode* ed*
  file* firmware-libertas* firmware-misc-nonfree* gdb* gettext-base*
  groff-base* hardlink* info* iptables* isc-dhcp-common* less*
  libbabeltrace-ctf1* libbabeltrace1* libboost-filesystem1.62.0*
  libboost-system1.62.0* libc6-dbg* libcwidget3v5* libdrm2* libdw1* libestr0*
  libfastjson4* libgdbm3* libglib2.0-0* libglib2.0-data* libident* libip6tc0*
  libiptc0* liblogging-stdlog0* liblognorm5* libluajit-5.1-common*
  libmagic-mgc* libmagic1* libmpdec2* libnetfilter-conntrack3* libnfnetlink0*
  libnih-dbus1* libnih1* libpam-chksshpwd* libpam-systemd* libperl5.24*
  libpipeline1* libpython3.5* libpython3.5-minimal* libpython3.5-stdlib*
  libsasl2-modules* libsigc++-2.0-0v5* libsysfs2* libudev0* libusb-0.1-4*
  libwrap0* libxapian30* libxtables12* luajit* mime-support* openssh-client*
  openssh-server* openssh-sftp-server* paxctld* perl* perl-modules-5.24*
  pi-bluetooth* rename* ucf*
0 upgraded, 0 newly installed, 75 to remove and 4 not upgraded.
After this operation, 138 MB disk space will be freed.

@MichaIng

Legend 👍 Much lighter install =))

[Fourdee]

dietpi-drive_manager not setting fstab correctly, will sort it.

• 🈯️ fixed:

[Fourdee]

NTS:

• RPi boot loop. install stage is stuck on -3
• Disable x-systemd.automount for RootFS, breaks FS_partition??

  Command (m for help): The partition table has been altered.
  Calling ioctl() to re-read partition table.
  Re-reading the partition table failed.: Device or resource busy

[MichaIng]

Offer some adjustments and further ideas here: https://github.com/Fourdee/DietPi/pull/1307

[Fourdee]

Setting this back to High Priority. This will vastly benefit image creation, and, offer users another installation method.

[Fourdee]

Jessie > Stretch:

need to check ordering

• 🈯️ Fixed with commit below

  
  [Ok] Step 8: Finalise system for first run of DietPi:
  
  [Info] Installing Dropbear by default

Reading package lists... Done Building dependency tree Reading state information... Done dropbear is already the newest version (2016.74-5). The following packages were automatically installed and are no longer required: acl busybox-initramfs isc-dhcp-common libapt-inst1.5 libapt-pkg4.12 libcap2-bin libdns-export100 libgnutls-deb0-28 libhogweed2 libicu52 libirs-export91 libisc-export95 libisccfg-export90 libnettle4 libprocps3 libpsl0 libreadline6 libssl1.0.0 libusb-0.1-4 Use 'sudo apt autoremove' to remove them. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

[Fourdee]

Tests:

• 🈺 C2 Stretch > Jessie : Works fine, but I believe there may be some issues with downgrading and packages left over ? https://pastebin.com/raw/6tXfajvF
• 🈯️ RPi Stretch > Stretch

~Script looks fine, however, devices stuck in boot loop (again).~ Resolved I've been meaning to shuffle FS_part FS_expand into systemd services to allow for earlier running of those tasks, outside of DietPi's boot script, may as well do that at same time.

Note to self:

• Allo GUI image, add swapfile generation if [ -d /var/www/allo ]; then during 1st run.
• clean this up

  
  [Info] Generating DietPi /etc/fstab

/boot/dietpi/dietpi-drive_manager: line 139: /DietPi/dietpi/func/dietpi-notify: No such file or directory /boot/dietpi/dietpi-drive_manager: line 388: /DietPi/dietpi/func/dietpi-set_userdata: No such file or directory

[Fourdee]

@MichaIng i'll re-enable the RPi packages (eg: kernel etc) due to below, for now to allow testing:

DietPi puts them on hold, after our kernel is installed.

The following packages have unmet dependencies:
 libraspberrypi-bin : Depends: libraspberrypi0 (= 1.20171029-1) but 1.20170811-1 is to be installed
E: Unable to correct problems, you have held broken packages.

[MichaIng]

@Fourdee Ah okay, makes sense. So we initially install kernel packages, also to at least temporarily resolve apt update messages about them, and then install our own kernel version on top of it? Otherwise we could also directly do the second and just ignore RPi packages completely. I was even thinking, if it would not be possible to completely remove the RPi kernel packages and "just" install it by ourself. This would make accidental overwriting by package and irritating available update messages impossible 😄.

---

Just copied from my PR for consideration:

ToDo:

• 🈯️ Put deb.debian.org repos as default in place. Use deb.debian.org/debian-security as replacement for @security.debian.org.
• 🈯️ Preserve current network interfaces, otherwise internet and/or SSH access can break after reboot.
• Recognize and preserve user SSH server choice? That way we also preserve potential key authentication, port etc. settings and other stuff, user configured on his SSH client. Otherwise SSH connection potentially fails after reboot.
• 🈯️ Cleanup openssh + dropbear handling in finalise script part, after applying the above.
• Check if we need libpam-systemd, which looks necessary for logind functionality. Otherwise logind could be just disabled: https://packages.debian.org/de/stretch/libpam-systemd
• 🈺 Make WiFi related packages optional, like Bluetooth? https://github.com/Fourdee/DietPi/pull/1331
• Make wrapper and file system packages optional, if not explicitly needed by certain DietPi script/software?
• 🈺 Choose right microcode package based on architecture? https://github.com/Fourdee/DietPi/pull/1331
• Remove (all?) firmware packages for VM images. No real processor seems to be emulated and no other usual hardware, where firmware is needed for. Works well, tested on all distro versions on VirtualBox.

[Fourdee]

Quick test after all my recent changes/commits:

• 🈯️ RPi 3 DietPi-Jessie > DietPi-Stretch -- 🈺 Edit: issue with dietpi-global hanging bash, due to dietpi-notify in dietpi-globals causing 100% processing loop. removed.

I need to setup some VM's with stock Debian and run further tests. If successful, we can open this up to public alpha testing.

[Fourdee]

Note to self,

• When upgrading from Jessie > Stretch, some autoremove packages remain.

Clean these up:

 [Info] Creating DietPi core environment

/boot/dietpi/func/dietpi-notify: line 19: /DietPi/dietpi/func/dietpi-globals: No such file or directory

 [Info] Generating DietPi /etc/fstab

/boot/dietpi/dietpi-drive_manager: line 249: /DietPi/dietpi/func/dietpi-notify: No such file or directory
/boot/dietpi/dietpi-drive_manager: line 370: /DietPi/dietpi/func/dietpi-set_userdata: No such file or directory

[MichaIng]

Test of current script on fresh Debian Jessie, installed via mini.iso with efi partition.

• Debian disc installation correctly chose grub-efi-amd64 package by itself.
• 🈯️ DietPi script chose however grub2 => fixed with https://github.com/Fourdee/DietPi/pull/1318
• 🈴 Several times during step 7: /boot/dietpi/func/dietpi-notify: line 19: /DietPi/dietpi/func/dietpi-globals: No such file or directory Ah as mentioned above, sry
• 🈴 During step 7:

  [Info] Generating DietPi /etc/fstab
  /boot/dietpi/dietpi-drive_manager: line 121: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 190: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 203: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 190: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 198: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 249: /DietPi/dietpi/func/dietpi-notify: No such file or directory
  /boot/dietpi/dietpi-drive_manager: line 370: /DietPi/dietpi/func/dietpi-set_userdata: No such file or directory

  Ah as mentioned above, sry

• Step 8: Minor, but we should get rid of any expected error message:

  [Info] Resetting boot.ini, config.txt, cmdline.txt etc
  sed: can't read /boot/cmdline.txt: No such file or directory

  fixed with https://github.com/Fourdee/DietPi/pull/1318

  DietPi-Set_Hardware
  ─────────────────────────────────────────────────────
  Mode: bluetooth (disable)
  Please wait...
  
  [Info] Bluetooth service failed to stop. Is it installed?
  [Failed] bluetooth disable | An issue has occured

  
  DietPi-Set_Hardware
  ─────────────────────────────────────────────────────
  Mode: preferipversion (auto)
  Please wait...
  
  [Ok] preferipversion auto | Completed
  
  [Info] Configuring kernels

rm: cannot remove '/etc/dietpi/.': Is a directory rm: cannot remove '/etc/dietpi/..': Is a directory

DietPi-Set_Hardware ───────────────────────────────────────────────────── Mode: soundcard (none) Please wait...

[Info] Installing alsa-utils. Please wait... Reading package lists... Done Building dependency tree Reading state information... Done E: Unable to locate package alsa-utils [Failed] Unable to install alsa-utils.

- 🈯️  After reboot: grub rescue, no such partition/invalid filesystem => **Fixed wth: https://github.com/Fourdee/DietPi/pull/1318**

[MichaIng]

Some fixes: https://github.com/Fourdee/DietPi/pull/1318

Reboot issue stays on EFI system:

• Keep old / + /boot/efi fstab entries => no solution
• Run script until end of step 5 (APT removal and installation) => works!
• Run script until end of step 6 (DietPi source instalaltion) => works!
• Run script until step 7 before core environment script => works!
• Manual dietpi-set_core_environment => works!
• Run script until including fstab generation => works!
• Run script until end of step 7 (ENV preparation) => works!
• Run script until fs_resize => works!
• 🈯️ Error found: resize script: TARGET_PARTITION=\$(findmnt / -o source -n | sed 's/.*p//') leads to /dev/sda2 instead of 2 as partition number. After some tries, default 2 will be deleted, but not created afterwards. => Fixed with: https://github.com/Fourdee/DietPi/pull/1318

@Fourdee Why do we enable i386 architecture for APT on amd64 system?

[Fourdee]

@MichaIng

Legend, great work 👍

Why do we enable i386 architecture for APT on amd64 system?

To ensure software which requires i386 pre-reqs (eg: steam I believe), is functional. Theres no harm in enabling it, i386 software should only be installed via apt with apt-get install xxxx:i386.

[Fourdee]

Testing VM with Debian 9 minimal, to check for bugs due to recent global/dietpi-set_core_environment changes:

• 🈯️ Fixed with: https://github.com/Fourdee/DietPi/commit/a11322d177a431c55d5d9c72f61eea0e0493d33e

  ./PREP_SYSTEM_FOR_DIETPI.sh: line 564: [: missing `]'
  grep: ]: No such file or directory
  ./PREP_SYSTEM_FOR_DIETPI.sh: line 568: [: missing `]'
  grep: ]: No such file or directory

---

• 🈯️ Fixed with https://github.com/Fourdee/DietPi/commit/af5a1f203506f85d2a0e6613239f0b7a6b322af1 (I forgot to re-enable global script support for non-dietpi systems after the changes.
• /boot/dietpi/func/dietpi-set_core_environment is doing nothing, returns to prompt. I'll check it over. -- . $FP_SCRIPTS/dietpi/func/dietpi-globals # NB: HW_* info not loaded is cause

 [Info] (0): Passed

./PREP_SYSTEM_FOR_DIETPI.sh: line 851: /mnt/samba/readme.txt: No such file or directory
./PREP_SYSTEM_FOR_DIETPI.sh: line 852: /mnt/ftp_client/readme.txt: No such file or directory
./PREP_SYSTEM_FOR_DIETPI.sh: line 853: /mnt/nfs_client/readme.txt: No such file or directory

 [Info] Starting DietPi-RAMdisk service

Failed to start dietpi-ramdisk.service: Unit dietpi-ramdisk.service not found.

 [Failed] (5): Script aborted

---

Minor:

 [Info] Creating DietPi core environment

cat: /DietPi/dietpi/.install_stage: No such file or directory

 [Info] Generating DietPi /etc/fstab

/boot/dietpi/dietpi-drive_manager: line 121: dietpi-notify: command not found
/boot/dietpi/dietpi-drive_manager: line 190: dietpi-notify: command not found
/boot/dietpi/dietpi-drive_manager: line 198: dietpi-notify: command not found
/boot/dietpi/dietpi-drive_manager: line 249: dietpi-notify: command not found
/boot/dietpi/dietpi-drive_manager: line 370: /DietPi/dietpi/func/dietpi-set_userdata: No such file or directory

---

Net:

• Need to add @MichaIng renaming fix back in for current session
• Need to automate the kernel ifnames to grub and update for subsequent boots.

  
  [Info] Configuring Network:

eth0: error fetching interface information: Device not found

---

Locale:
- Still need to either automate ```enGB.utf-8```, or, prompt user to enable it.

--- 

Minor:
- 🈺 Not much we can do, relies on kernel/disk support.

[Info] (0): Passed

fstrim: /: the discard operation is not supported

[MichaIng]

@Fourdee Thanks for adding+fixing. Interesting, on my Jessie VM [ ... ] works great. Seems to be an inconsistency in using brackets within bash, and I am still a bid confused when to use what. Reading: https://stackoverflow.com/questions/2188199/how-to-use-double-or-single-brackets-parentheses-curly-braces 😆.

[Fourdee]

@MichaIng

Thanks for adding+fixing. Interesting, on my Jessie VM [ ... ] works great. Seems to be an inconsistency in using brackets within bash, and I am still a bid confused when to use what. Reading:

Unsure, i'll need to review it. Converted it to previous system i use for quick fix now, with int (( ))) and grep -ci which returns 0 on non-match, 1 on match.

I've added fixes for a few of the issue, made notes of outstanding. Will try and pick this back up tomorrow 😄

[MichaIng]

@Fourdee

Unsure, i'll need to review it

Not needed, this was mostly for myself. Your fix works!

[Fourdee]

Note to self:

• dietpi-global is now loadable in PREP_SYSTEM. I'll verify tomorrow and send commit. Means we dont have to define dietpi-notify and AGI twice.

  #apt-get update
  #apt-get install wget || or make it a pre-req?
  #wget
  #load globals

[Fourdee]

@MichaIng

Sending a large commit with vast changes to globals + this script. I'll be doing testing and resolving issues afterwards, could be very unstable, may want to avoid any commits/PR's at this stage.

I'll let you know when its completed.

[Fourdee]

Re-Tests:

• 🈯️ Odroid C2, running DietPi Stretch > Stretch
• 🈺 VM Debian 9 | eth0 outstanding. Once we update grub defaults, ~we dont need to rename during install, not required,~ subsequent reboot will load the kernel options with standard eth naming.

---

• VM Debian 9 > Buster (after all eth naming commit) | Gah:

  
  Preparing to unpack .../passwd_1%3a4.5-1_amd64.deb ...
  Unpacking passwd (1:4.5-1) over (1:4.4-4.1) ...

Hang,

FX6300 server died, hard lock. Old horse showing its age lol

Run 2:
- 🈴 Due to eth renaming on fly.

[Info] Applying G_HW_ARCH specific tweaks: [Info] APT update Ign:1 https://deb.debian.org/debian buster InRelease Ign:2 https://deb.debian.org/debian buster-updates InRelease Ign:3 https://deb.debian.org/debian-security buster/updates InRelease Err:4 https://deb.debian.org/debian buster Release Cannot initiate the connection to deb.debian.org:443 (128.31.0.62). - connect (101: Network is unreachable) [IP: 128.31.0.62 443] Err:5 https://deb.debian.org/debian buster-updates Release Cannot initiate the connection to deb.debian.org:443 (128.31.0.62). - connect (101: Network is unreachable) [IP: 128.31.0.62 443] Err:6 https://deb.debian.org/debian-security buster/updates Release Cannot initiate the connection to deb.debian.org:443 (128.31.0.62). - connect (101: Network is unreachable) [IP: 128.31.0.62 443] Reading package lists... E: The repository 'https://deb.debian.org/debian buster Release' no longer has a Release file. E: The repository 'https://deb.debian.org/debian buster-updates Release' no longer has a Release file. E: The repository 'https://deb.debian.org/debian-security buster/updates Release' no longer has a Release file. [Failed] G_AGUP: An issue has occured

root@debian:~# route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0 root@debian:~# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.0.37 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::20c:29ff:fe5d:6cb1 prefixlen 64 scopeid 0x20 ether 00:0c:29:5d:6c:b1 txqueuelen 1000 (Ethernet) RX packets 103606 bytes 152757286 (145.6 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 41871 bytes 3285556 (3.1 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback) RX packets 160 bytes 12960 (12.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 160 bytes 12960 (12.6 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[Fourdee]

@MichaIng

Ok, global naming/use are now finalized (took me a while to find the "right" naming): https://github.com/Fourdee/DietPi/issues/1311#issuecomment-353039600

PREP_SYSTEM still needs some work (https://github.com/Fourdee/DietPi/issues/1285#issuecomment-353015233), however, the major script wide changes are now completed. Please feel free to continue with PR's as needed.

[MichaIng]

@Fourdee Just used the script on DietPi VM-Stretch, upgrading to Buster:

• Jep, connection get lost in between because of interface reset.
• On first run dietpi-software connection check, it took significantly longer than 20s. Reason after investigation is, that wget by default tries to connect via IPv6, even that IPv6 is disabled on DietPi. For apt update, depending on test domain/ip, it is the same. We really need to consider, setting forced IPv4 for apt and wget by default. On first run we could however test IPv6 connectivity and remove forcing settings for apt and wget again. On enabling/disabling IPv6 within dietpi-config, this settings should also be set. I am not sure, if 'forcing' IPv6 is a good idea and will work on every access server, but forcing IPv4 is needed, as apt and wget do not check for enabled kernel module or other things.
• On applying network settings I get: Failed to disable unit: Unit file wifi_disable_powersave.service does not exist
• If ownCloud/Nextcloud was installed on DietPi system, their software directories stay in place on script run, but mysql/mariadb will be purged together with database files. On reinstallation of ownCloud/Nextcloud, installer will find a fully configured instance with installed flag in it's config file, thus will skip most steps, including mysql/mariadb database+user creation => Nextcloud/ownCloud will stay without database access. I was thinking of some smart installer checks, detecting if database settings within nc/oc config files match actual mysql/mariadb content, and in case remove installed flag from config file to allow occ terminal installation. On the other hand, other dietpi software might be affected as well, if just packages get removed, but no manually downloaded binaries and config files. We could also, if DietPi system is detected, uninstall all dietpi software first. This actually fit's better to the idea of creating a totally fresh instance.

[Fourdee]

@MichaIng

Jep, connection get lost in between because of interface reset.

Ok, i'll take a look, see if we can find a cure.

On applying network settings I get: Failed to disable unit: Unit file wifi_disable_powersave.service does not exist

~During PREP_SYSTEM run? Any idea what stage (eg: notify text before it)~ Found it, fixed, will be in next commit.

We really need to consider, setting forced IPv4 for apt and wget by default.

Agree, i'll see if we can apply them to the globals and script.

On the other hand, other dietpi software might be affected as well, if just packages get removed, but no manually downloaded binaries and config files. We could also, if DietPi system is detected, uninstall all dietpi software first. This actually fit's better to the idea of creating a totally fresh instance.

The initial idea (in my mind) was for PREP_SYSTEM to achieve the following:

• Allow users with non-DietPi systems, who are running a fresh minimal Debian installation, to convert it to DietPi, ending up with a clean fresh installation, ready for 1st time use.
• Allow me (and/or others) to quickly update our existing DietPi .img files, again, clean fresh installation for first time use.

I personally think if we go down the path of "system upgrades" (eg: converting existing system in-use, and its installed software, across to DietPi), we will open up a huge can of worms. I believe we should keep PREP_SYSTEM designed solely to install a fresh DietPi installation on the device, for 1st time use.

If ownCloud/Nextcloud was installed on DietPi system, their software directories stay in place on script run, but mysql/mariadb will be purged together with database files.

Located in /var/www? we could probably rm it during install.

[Fourdee]

Due to FX6300 on its way out, having to run VM tests on a AMD Athlon 5150, not bad, not great. Gives me lots of time to make coffee 😄 , starting to like it a little more lol. https://www.cpubenchmark.net/cpu.php?cpu=AMD+Athlon+5150+APU&id=2208

[Fourdee]

Notes:

• 🈯️ Fixed with grub commits below

  
  firmware-linux-nonfree can not be marked as it is not installed.
  grub2 can not be marked as it is not installed.

The following packages will be REMOVED: grub-common grub-pc grub-pc-bin grub2-common

[Fourdee]

Tests:

• 🈯️ VM Debian Stretch > Buster
• VM DietPi Debian Buster > Jessie

🈯️ fixed 1:

W: --force-yes is deprecated, use one of the options starting with --allow instead.
E: Unable to correct problems, you have held broken packages.
 [Failed] G_AGI: An issue has occured

 [Failed] (100): Script aborted

🈴 2:

Installing core DietPi pre-req APT packages
....
The following packages have unmet dependencies:
 bzip2 : Depends: libbz2-1.0 (= 1.0.6-7+b3) but 1.0.6-8.1 is to be installed
 initramfs-tools : Depends: initramfs-tools-core (= 0.130) but it is not going to be installed
 ntfs-3g : PreDepends: fuse but it is not going to be installed
 resolvconf : PreDepends: ifupdown (>= 0.7.5)
              PreDepends: initscripts (>= 2.88dsf-34) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

[MichaIng]

@Fourdee 🈴 2: Uff downgrading of cause brings several other issues and thanks god this is a scenario, no one will really need.

• Creating official images, we should always start with the same debian/raspbian image, as there will be always possible issues, obsolete files, incompatible configs etc. on distro version changes. Btw. I found the mini images to be the smallest for Debian, but of course no local repository is contained: http://ftp.nl.debian.org/debian/dists/stretch/main/installer-amd64/current/images/netboot/
• No one would want to downgrade his running production system. Most, I guess, want to stay, but indeed people are always looking for a way to upgrade, without need to start from scratch, e.g. http://dietpi.com/phpbb/viewtopic.php?f=9&t=2594 But this will always be a bid risky and outcome extremely depends on individual system, thus I agree with you, that we should not support this. But enabling a more reliable way, by first purging down a system as much as possible, would be no harm, from my point of view. We just need to make clear, that this is not the intention of the script and we will not provide any support for issues, that can easily occur.

  ---

  About the APT issues:

• This is expected, as we do not purge all packages, and the ones that stay, will not be downgraded according to priority rules. This we leave a mixture of packages from both distro versions where their versions may not fulfil each others dependencies. I guess if this should work reliably, we would need to force downgrading of all existing packages by temporary raising target repo priority to >1000: https://linux.die.net/man/5/apt_preferences
• Generally I think it would also make a sense, if we temporary overwrite existing package configs by current maintainers versions during APT step. After that we do our adjustments to fresh, repo compatible config files and then force keeping adjusted configs again.

  ---

  So generally about how to going on:

• Do we want to keep the option to choose distro version? => Then I would consider the mentioned steps to at least get this working on our current tests, but as mentioned would give a clear notification that there will be no support from our side (as the issues might become uncountable and hard to investigate).
• Do we want to keep the option (and handling) to run the script on already DietPi enabled system? => Then I would still consider to dietpi-software uninstall everything, before further cleaning up APT packages. I anyway want to implement some nice backup handling and restoring to ownCloud/Nextcloud, that data and database can survive uninstall/install procedures, if wanted by user.

  ---

Just reviewed your last commits, looks very nice 👍. Will run test runs again 😃.

[MichaIng]

Optional WiFi and conditional microcode package installation: https://github.com/Fourdee/DietPi/pull/1331

The first needs on demand installation (and uninstallation?) on WiFi activation/deactivation within dietpi-config and some other scripts need to check for installed packages before throwing errors.

---

Open tasks:

• 🈺 Make WiFi related packages optional, like Bluetooth? https://github.com/Fourdee/DietPi/pull/1331
• 🈺 Choose right microcode package based on architecture? https://github.com/Fourdee/DietPi/pull/1331
• Recognize and preserve user SSH server choice? That way we also preserve potential key authentication, port etc. settings and other stuff, user configured on his SSH client. Otherwise SSH connection potentially fails after reboot.
• Check if we need libpam-systemd, which looks necessary for logind functionality. Otherwise logind could be just disabled: https://packages.debian.org/de/stretch/libpam-systemd
• Make wrapper and file system packages optional, if not explicitly needed by certain DietPi script/software?
• Remove (all?) firmware packages for VM images. No real processor seems to be emulated (/proc/cpuinfo shows host CPUs, do microcodes have influence there?) and no other usual hardware, where firmware is needed for. Works well, tested on all distro versions on VirtualBox.

[Fourdee]

@MichaIng

We just need to make clear, that this is not the intention of the script and we will not provide any support for issues, that can easily occur.

Agree, i'll make a note of no downgrade support 👍

• 🈯️ I'll see if we can hide distro selection, if lower than current.
• 🈯️ firmware-realtek also contains ethernet firmware, I'll review these and add them back in if needed (https://packages.debian.org/stretch/firmware-realtek): https://github.com/Fourdee/DietPi/pull/1331/files#diff-e9e0c6c64b4937739e13ffac58f4a888L473

Odroid C2 + Buster = 258MB rootFS (nice) :

root@DietPi:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p2  113G  258M  108G   1% /

---

• 🈯️ Install WiFi packages on demand in DietPi scripts (current expects to be installed)

[k-plan]

I want to play around as well: 😄

• 🈴 Odroid HC1 (XU4/3), running DietPi Stretch > Stretch

Required Information:

• DietPi Version | v # # (note: it is v160 testing Branch updated) # #
• SBC Device | oDroid XU3/4 (armv7l) (index=11)
• Distro | stretch (index=4)
• Command | G_AGA:
• Error Handler | G_ERROR_HANDLER

Additional Information (if applicable):

• Software title | DietPi-PREP_SYSTEM_FOR_DIETPI

Expected behaviour:

• Build a new DietPi image for my Odroid HC1 from (NR_4.x)_DietPi_v159_OdroidXU4-armv7-(Stretch).img

Actual behaviour:

• [Error] Unable to continue, the program will now terminate. 😢

Steps to reproduce:

• set: GIT_BRANCH=testing
• run: :~# ./PREP_SYSTEM_FOR_DIETPI.sh

Additional logs:

Log file contents:
Removing libapt-inst1.5:armhf (1.0.9.8.4) ...
Removing libapt-pkg4.12:armhf (1.0.9.8.4) ...
Removing libcap2-bin (1:2.25-1) ...
Removing libirs-export91 (1:9.9.5.dfsg-9+deb8u14) ...
Removing libdns-export100 (1:9.9.5.dfsg-9+deb8u14) ...
Removing libestr0 (0.1.10-2) ...
Removing perl (5.24.1-3+deb9u2) ...
Removing libperl5.24:armhf (5.24.1-3+deb9u2) ...
Removing libgdbm3:armhf (1.8.3-14) ...
Removing libgnutls-deb0-28:armhf (3.3.8-6+deb8u7) ...
Removing libgpm2:armhf (1.20.4-6.2+b1) ...
Removing libhogweed2:armhf (2.7.1-5+deb8u2) ...
Removing libpsl0:armhf (0.5.1-1) ...
Removing libicu52:armhf (52.1-8+deb8u6) ...
Removing libisccfg-export90 (1:9.9.5.dfsg-9+deb8u14) ...
Removing libisc-export95 (1:9.9.5.dfsg-9+deb8u14) ...
Removing liblogging-stdlog0:armhf (1.0.5-2+b2) ...
Removing nfacct (1.0.1-1.1+b1) ...
Removing libnetfilter-acct1:armhf (1.0.2-1.1) ...
Removing libnettle4:armhf (2.7.1-5+deb8u2) ...
Removing libnfnetlink0:armhf (1.0.1-3) ...
Removing libpipeline1:armhf (1.4.1-2) ...
Removing libprocps3:armhf (2:3.3.9-9) ...
Removing libreadline6:armhf (6.3-8+b3) ...
Removing libssl1.0.0:armhf (1.0.1t-1+deb8u7) ...
Removing libusb-0.1-4:armhf (2:0.1.12-30) ...
Removing linux-image-4.9-armhf-odroid-xu3 (4.9.61-1) ...
Removing linux-image-4.9.61+ (4.9.61-20171118-XU3) ...
dpkg: error processing package linux-image-4.9.61+ (--remove):
 subprocess installed pre-removal script returned error exit status 1
Removing perl-modules-5.24 (5.24.1-3+deb9u2) ...
Removing xz-utils (5.2.2-1.2+b1) ...
dpkg: u-boot-tools: dependency problems, but removing anyway as you requested:
 linux-image-4.9.61+ depends on uboot-mkimage | u-boot-tools; however:
  Package uboot-mkimage is not installed.
  Package u-boot-tools is to be removed.

Removing u-boot-tools (2016.11+dfsg1-4) ...
dpkg: uboot: dependency problems, but removing anyway as you requested:
 linux-image-4.9.61+ depends on uboot; however:
  Package uboot is to be removed.
  Package uboot which provides uboot is to be removed.
 linux-image-4.9.61+ depends on uboot; however:
  Package uboot is to be removed.
  Package uboot which provides uboot is to be removed.

Removing uboot (20170824-9+deb8u1) ...
Errors were encountered while processing:
 linux-image-4.9.61+
E: Sub-process /usr/bin/dpkg returned an error code (1)

[k-plan]

Next run testing fun: 😃

• 🈴 Odroid HC1 (XU4/3), running Meveric Debian Jessie > Stretch

Required Information:

• DietPi Version | v
• SBC Device | NULL (index=11)
• Distro | stretch (index=4)
• Command | G_AGI: apt-transport-https apt-utils bash-completion bc bzip2 ca-certificates console-setup cron curl dbus debconf dosfstools dphys-swapfile dropbear ethtool fake-hwclock fbset firmware-realtek gnupg hdparm hfsplus htop initramfs-tools iputils-ping isc-dhcp-client locales nano net-tools ntfs-3g ntp p7zip-full parted psmisc resolvconf sudo tzdata unzip usbutils wget whiptail wput zip linux-image-armhf-odroid-xu3 crda firmware-atheros firmware-brcm80211 firmware-ralink iw rfkill wireless-tools wpasupplicant
• Error Handler | G_ERROR_HANDLER

Additional Information (if applicable):

• Software title | DietPi-PREP_SYSTEM_FOR_DIETPI

Expected behaviour:

[Info] Non-DietPi system

• Build a new DietPi image for my Odroid HC1 from https://oph.mdrjr.net/meveric/images/Jessie/Debian-Jessie-1.1.4-20171121-XU3+XU4.img.xz

Actual behaviour:

• after 1h of running and updating files .... 🕛 🕐 😢

[Error] Unable to continue, the program will now terminate.

• after this, system is f... up, apt-get update won't run any more.

Steps to reproduce:

• download the image, write it to sd card and after booting up run: :~# ./PREP_SYSTEM_FOR_DIETPI.sh

Additional logs:

Log file contents:
E: Method https has died unexpectedly!
E: Sub-process https received a segmentation fault.
E: Method /usr/lib/apt/methods/https did not start correctly

[k-plan]

---

General Notes:

• run apt-get update as first step because a_MIN_APT_PREREQS will fail elsewhere on a fresh image

• first run and set dpkg-reconfigure locales to en_GB.UTF-8 to avoid hundert message like this:

  warning: Setting locale failed.
  warning: Please check that your locale settings:
      LANGUAGE = (unset),
      LC_ALL = (unset),
      LANG = "en_GB.UTF-8"
  are supported and installed on your system.
  warning: Falling back to the standard locale ("C").
  locale: Cannot set LC_CTYPE to default locale: No such file or directory
  locale: Cannot set LC_MESSAGES to default locale: No such file or directory
  locale: Cannot set LC_ALL to default locale: No such file or directory

• set: passwd to a known password like "dietpi" for user "root"

---

NB:

• how will we handle different disk layouts? Not two partitions, which first is "boot" (FAT) / second is "/" (ext4), eg one big BTRFS ?

• yes, finalize was a PITA and very time-consuming. But PREP_SYSTEM_FOR_DIETPI.sh? For every image update? Puuuh ... I don't know.

---

Just like that, the fun's over for today. 😃

[k-plan]

MichaIng commented:

---

@k-plan €: This is about your first try ;). How did you solve the linux-image apt error for 2nd try?

Ah just found this one: http://www.hardkernel.com/main/products/prdt_info.php?g_code=G150229074080

Any OS for XU4 is fully compatible with the HC1.

And as I see the packages, the image seems to be based on Meverics?

• For this device we choose the default kernel meta package linux-image-armhf-odroid-xu3: https://github.com/Fourdee/DietPi/blob/testing/PREP_SYSTEM_FOR_DIETPI.sh#L553
• But it depends on ancient linux-image-3.10.106+: https://oph.mdrjr.net/meveric/dists/all/xu3/binary-armhf/Packages Thus the existing kernel package will removed first.
• On official Debian images, linux-image* packages are blocked from autoremove by /etc/apt/apt.conf.d/01autoremove(-kernels), which seems not the case for Meverics image.
• Removing the only installed kernel package seems to need failing, which causes the dpkg error?
• On HC1 product page they promote the use of newest 4.14 kernel on their official linux (Ubuntu based) image, so I assume linux-image-4.14-armhf-odroid-xu4 to be stable enough also for us? Otherwise at least we should use linux-image-4.9-armhf-odroid-xu3.
• xu3/xu4 naming mixed together there, both are meta packages depending on linux-image-4.XX.XX packages where no device naming appears any more, thus I think they are compatible vice versa and for HC1 as well.
• To prevent any kernel removal issues during image preparation: Install the chosen kernel package before G_AGA, or identify and choose already installed kernel package. I would vote for first solution, as we then keep control over which kernel is used with DietPi.

[k-plan]

@MichaIng

And as I see the packages, the image seems to be based on Meverics?

Yes!

Any OS for XU4 is fully compatible with the HC1

I have write hardware comparison here: http://dietpi.com/phpbb/viewtopic.php?f=11&t=2422&p=9583&hilit=HC1#p9583

To prevent any kernel removal issues during image preparation: Install the chosen kernel package before G_AGA, or identify and choose already installed kernel package. I would vote for first solution, as we then keep control over which kernel is used with DietPi.

How to do this? Image was on 4.9.61+. Have tested, you can upgrade to 4.14.x, but after this, you can't go back to 4.9.61+. https://forum.odroid.com/viewtopic.php?f=96&t=17542&p=209574#p208191

When asked if you want to STOP the removal of the active Kernel please answer with "NO".

If you will answer "YES" kernel update stops and fail. (tested)

---

sorry @Fourdee , no english, to time-consuming

Ja, die Odroid Welt ist etwas schwieriger ....

• Die wollen kompatibel bleiben für: XU4/XU3/HC1/MC1 und nur ein Image (+ Quellen) pflegen

We guarantee the production of ODROID XU4/XU3/HC1/MC1 to the middle of 2020, but expect to continue production long after.

Hardkernels Images (Ubuntu) sind schon zur Zeit auf 4.14.x , da gibt es auch die Kernel source für: https://forum.odroid.com/viewtopic.php?f=146&t=28895

Meveric bastelt aus der Kernel Source ein Debian draus und pflegt das schon seit Jahren. Sein Schwerpunkt ist die Gaming Welt ( https://forum.odroid.com/viewtopic.php?f=11&t=2684 ) daher auch die gute Unterstützung für Kodi, XBMC, Grafikausgabe über HDMI. Der ist aber auch eine "One-Man-Show" wie Dan bisher. Daher hängt er immer etwas hinterher. Hatte aber die gesamte Mali-T628 MP6 Hardware Grafik Unterstützung als erster laufen und lange Zeit die Nase vorn damit.

Mit dem 4.x Kernel ist es zum "Bruch" gekommen, so wie ich das bis jetzt verstanden habe. Bis 3.x nutzen alle ODROID XU4/XU3/HC1/MC1 nur exynos5422-odroidxu3.dtb Seit 4.x wir es spezifischer und man muss das richtige beim booten laden je nach verwendeten Board:

exynos5422-odroidxu3-lite.dtb
exynos5422-odroidxu3.dtb
exynos5422-odroidxu4-kvm.dtb
exynos5422-odroidxu4.dtb

Da gibt es zwar ein paar nette Tricks für, wie man das über die boot.ini steuern könnte, nur haben wir das im Moment (noch) nicht (Code Schnippsel entliehen):

if test "${board_name}" = "xu4"; then setenv fdtfile "exynos5422-odroidxu4.dtb"; fi
if test "${board_name}" = "xu3"; then setenv fdtfile "exynos5422-odroidxu3.dtb"; fi
if test "${board_name}" = "xu3l"; then setenv fdtfile "exynos5422-odroidxu3-lite.dtb"; fi

if ext4load mmc 0:1 0x00000000 "/boot/.next" || fatload mmc 0:1 0x00000000 ".next"  || ext4load mmc 0:1 0x00000000 ".next"; then echo "Found mainline kernel configuration"; else setenv fdtfile "exynos5422-odroidxu3.dtb"; fi
ext4load mmc 0:1 0x44000000 /boot/dtb/${fdtfile} || fatload mmc 0:1 0x44000000 dtb/${fdtfile} || ext4load mmc 0:1 0x44000000 dtb/${fdtfile}

Weitere Schwierigkeit, Fourdee will es "schlank" haben und will eine FAT boot Partition, in der man einfach Konfigs editieren kann. Beides finde ich sehr gut. Aber herzlichen Glückwunsch, wenn das Gerät von SD-Karte, eMMC, USB und HDD/SSD booten kann. Das wird dann mit der /etc/fstab sehr schnell unschön und die Tricksereien fangen an.

:~# apt-get install linux-image-
linux-image-3.10.105+                 linux-image-4.14.9+                   linux-image-4.9.35+                   linux-image-4.9.58+
linux-image-3.10.106+                 linux-image-4.14-armhf-odroid-xu4     linux-image-4.9.37+                   linux-image-4.9.60+
linux-image-3.16.0-4-armmp            linux-image-4.9.0-0.bpo.2-armmp       linux-image-4.9.38+                   linux-image-4.9.61+
linux-image-3.16.0-4-armmp-lpae       linux-image-4.9.0-0.bpo.2-armmp-lpae  linux-image-4.9.43+                   linux-image-4.9-armhf-odroid-xu3
linux-image-4.14.0+                   linux-image-4.9.0-0.bpo.4-armmp       linux-image-4.9.44+                   linux-image-armhf-odroid-xu3
linux-image-4.14.3+                   linux-image-4.9.0-0.bpo.4-armmp-lpae  linux-image-4.9.47+                   linux-image-armmp
linux-image-4.14.4+                   linux-image-4.9.30+                   linux-image-4.9.50+                   linux-image-armmp-lpae
linux-image-4.14.5+                   linux-image-4.9.33+                   linux-image-4.9.51+
linux-image-4.14.7+                   linux-image-4.9.34+                   linux-image-4.9.57+

Meine persönlichen "Erfahrungen" mit den verschiedenen Kernel:

• 3.x hat eine schlechtere RAM Performance, warum auch immer.

• 4.9.x ist mit dem RAM der schnellste

• 4.14.x ist etwas langsamer bei meinen Tests

Schreib/Lesegeschwindigkeit (SD-Karte vs eMMC), RAM Größe und Schreib/Lesegeschwindigkeit, machen bei den Devices, neben Prozessortakt und Kerne Anzahl, den größten Unterschied aus.

Klar haben neuere Kernel meist bessere Unterstützung für "neue Features", doch persönlich brauche ich die wenigsten davon. Würde also vorerst zu 4.9.x bei XU4 tendieren. Da wird aber von Seiten Hardkernel erst einmal nichts mehr kommen.

Ordoid C1/C1+/C2 schenke ich mir an dieser Stelle erst einmal. Das sind noch ganze andere Nummern mit anderen Schwierigkeiten.

[k-plan]

@Fourdee @MichaIng

Only for clarification: I like your work so much. You are on a good trip and I think and hope you have finally success. Only some difficulties arising from a new way. Keep one's tail up boys and girls. 😃

Or is it too early for testing?

PS: ... and the output github printing is awesome! Make putting things on here so easy! Love it! 👍 💯 Hope user will use it too!

[k-plan]

MichaIng commented

---

@k-plan Okay, about the kernel and boot.ini modification: For me that sounds, as would it be best to keep the kernel (linux-image*) package that already exists on the image and do not mess around to much with the boot.ini, at least for now? Or stays boot.ini unaffected by kernel upgrade? We do already the same with fstab, at least the script should just copy the drives mounted on the existing machine. Of course auto file system expansion is another topic 🙈, if the machine has that much different boot options with possibly different device name schemes.

Indeed, it might be more successful/happy ending for now, to test the preparation script on already existing devices. But on the other hand as HC1 is/should work more less the same than XU3/4, the error should have also occurred there. I guess we have to expect some more issues here and there with the script, creating images/testing for all devices. But as before it was also hand craft, the only downside is, that you can't make the script starting at a certain point, after you manually fixed the issue there. But copy/paste the commands after manually fixing is still possible 😆. Maybe we can create some entry points, e.g. for the 8 steps of the script. Means if you run ./PREP_SYSTEM_FOR_DIETPI.sh 4 the script starts at step 4.

But hmm, we are sort of in the wrong issue here @Fourdee is it possible for you to copy the last comments over to https://github.com/Fourdee/DietPi/issues/1285?

---

Another topic I found by chance: https://github.com/Fourdee/DietPi/issues/1357