search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.81k stars 494 forks source link

Emby: Error in getting File 41.deb #3929

Closed LeChatNoir666 closed 3 years ago

LeChatNoir666 commented 3 years ago

Hi! :) First: Great Development and it really makes fun to work with DietPi! :)

Now to my Problem: Emby will not install from the Software Menu.

[ INFO ] DietPi-Software | APT install for: ./41.deb, please wait... E: Unable to correct problems, you have held broken packages. [FAILED] DietPi-Software | APT install for: ./41.deb

Details:

Expected behaviour:

Extra details:

The following packages have unmet dependencies: emby-server:arm64 : Depends: zlib1g:arm64 (>= 1:1.1.4) but it is not installable E: Unable to correct problems, you have held broken packages.

Additional logs:

E: Unable to correct problems, you have held broken packages.

Have it something to do with 64bit needed? I have an other Problem too with DietPi and Docker but this is not primary on my list and I will try Docker again in some days. But I stumbled about 64bit and Docker too...

Joulinar commented 3 years ago

Hi,

could it be you are running a Raspberry OS 32 bit with forced kernel mode 64 bit?

That's basically causing the issue as you are trying to install 64bit executable on a 32bit OS and it's simply not able to install missing dependency.

Just setting arm_64bit=1 and switching the kernel, will not make a 64bit OS 😃

Just run dpkg --print-architecture and you will see, your system is still armhf and not arm64. I would recommend to remove arm_64bit=1 flag and continue with a normal 32bit system.

LeChatNoir666 commented 3 years ago

Many thanks for your reply! 👍 :)

Well... I installed this Version on BerryBoot and didn't touched something like kernel or so: dietpi_buster_all_rpi_2020.06.14_berryboot.img.gz from: https://sourceforge.net/projects/berryboot-updated-images/files/Raspberry-Pi/ because I wanted a BootLoader where I can do backups and some maintenance stuff. Not a good idea?

And yes, its armhf. :-/ Where can I remove the flag?

Joulinar commented 3 years ago

Just do nano /boot/config.txt and remove arm_64bit=1

BTW: on DietPi you can do backups using dietpi-backup 😉

LeChatNoir666 commented 3 years ago

In my config.txt is no arm_64bit=1. Should I add arm_64bit=0 instead?

I've tried the backup from dietpi-backup but wasnt able to make a backup on my FAT32 USB stick...

EDIT: Now I have a ext4 HDD on one USB and doing now a backup :) What kind of backup is it? Full or only Userdata?

Also I wanted BerryBoot for having different OS's on my MMC, like for testing. :)

Btw; The "regular" DietPi is 64 Bit?

Joulinar commented 3 years ago

Strange that you don't have arm_64bit=1 set as this is the usual way to force a 64bit kernel mode. 🤔

FAT32 did not support unix file system permission and this is a prerequisite for dietpi-backup. EDIT: It's full backup of the DietPi system

The question is not if DietPi is 64 bit. It's more if the used base image is 32 or 64 bit. 😉 Because DietPi is not an own OS or distribution. It's a cool set of scripts on top of a base image. In case of Raspberry Pi, DietPi is using Raspberry OS as base line. At the moment Raspberry OS is available as 32 bit version only. The Raspberry Foundation is working on a 64bit Raspberry OS, but this is still in Beta with limited or missing functionality. Something the Raspberry Foundation si still working on. However it's already supported by DietPi.

32bit default https://dietpi.com/downloads/images/DietPi_RPi-ARMv6-Buster.7z 64bit BETA https://dietpi.com/downloads/images/testing/DietPi_RPi-ARMv8-Buster.7z

MichaIng commented 3 years ago

Lol, I'm typing too slow 😄. Berryboot even has a testing DietPi Buster 64-bit image: https://sourceforge.net/projects/berryboot-updated-images/files/Raspberry-Pi/Testing/Raspberry-Pi-4/ So I'm wondering why and how they run the 32-bit OS image in 64-bit kernel mode 🤔.

I'm having a look into the Berryboot image.

LeChatNoir666 commented 3 years ago

Okay... I have now Plex installed and the installing worked like a charm! :) If I (we ;) ) get Emby working, can I run it together with Plex? Because some of my Family member use Pley, some Emby... And I dont want to discuss again with them, whats better... mihihi ;)

If my DietPi is running I could try to test the 64Bit, but im fresh to ux-systems :D I need to do the same procedure like setting up samba and webmin aso again then?

EDIT: Would be nice, MichaIng :)

Joulinar commented 3 years ago

using 64bit image is like flashing a new SD card. The versions are completely independent from each other, as they are using complete different sources and complete different software packages

However don't expect miracles from 64bit version. Usually there is not that much benefit and you could stay with the 32bit image without issues.

LeChatNoir666 commented 3 years ago

Okay... Well... I plan to buy a Pi 4 in the near future and try a 64Bit Version then on it. It was/is not on my list for my Pi 3b+ :) Again: Biiiig tnx so far!!

EDIT: Its offtopic, but can if have something to do with the 6Bit thingy, that Docker dont runs too as expected? I got it to run for only some minutes... The strange thing: I checked the status with: sudo systemctl status docker and its shows green. Then I checked it again and it stopped... In this time i did nothing to the system... Strange Docker behavior

Joulinar commented 3 years ago

64 bit is running on the RPi3B+ as well 😉

MichaIng commented 3 years ago

I had a look into the Berryboot image and still don't understand everything, but what I can say:

The big problem with all this is that all usual and documented OS/kernel (and in case drive) features and configuration do not work, especially also building kernel modules, which breaks our WireGuard implementation and I think Docker as well as it wants to load kernel modules, but none are present on the regular path.

Just to verify:

uname -a
ls -al /lib/modules
lsmod

The idea of a multi-boot system and extended bootloader capabilities is great, but when an OS is invoked with a complete different kernel implementation and configuration that causes limitations. In our case it is a larger issue since our scripts expect config.txt to work to configure the system for the chosen/installed software titles, but as well the APT repository packages do so (in case of RPi repo packages at least) but at least expect kernel modules at /lib/modules.

Joulinar commented 3 years ago

It's part of Berryboot config menu

PXL_20201123_122054779

LeChatNoir666 commented 3 years ago

vcgencmd get_mem gpu shows gpu=160M

uname -a ls -al /lib/modules

shows: root@BerryBootDietPi:/mnt/SERIEN# uname -a Linux BerryBootDietPi 5.4.73v64 #2 SMP PREEMPT Tue Nov 3 16:11:05 CET 2020 aarch64 GNU/Linux root@BerryBootDietPi:/mnt/SERIEN# ls -al /lib/modules total 18 drwxrwxr-x 1 root root 4096 Nov 3 16:11 . drwxr-xr-x 1 root root 4096 Oct 30 16:40 .. drwxr-xr-x 3 root root 348 Nov 21 16:31 5.4.72+ drwxr-xr-x 3 root root 348 Nov 21 16:59 5.4.72-v7+ drwxr-xr-x 3 root root 348 Nov 21 16:31 5.4.72-v7l+ drwxr-xr-x 3 root root 348 Nov 21 16:32 5.4.72-v8+ drwxr-xr-x 3 root root 4096 Nov 3 16:12 5.4.73v64

and lsmod shows: Module Size Used by binfmt_misc 24576 1 aes_neon_blk 32768 1 crypto_simd 16384 1 aes_neon_blk cryptd 24576 1 crypto_simd bnep 28672 2 hci_uart 49152 1 btbcm 24576 1 hci_uart bluetooth 405504 24 hci_uart,btbcm,bnep ecdh_generic 16384 2 bluetooth ecc 36864 1 ecdh_generic sg 36864 0 vc4 258048 0 cec 49152 1 vc4 drm_kms_helper 196608 3 vc4 drm 487424 3 drm_kms_helper,vc4 ip_tables 32768 0 x_tables 40960 1 ip_tables ipv6 495616 28 brcmfmac 315392 0 brcmutil 20480 1 brcmfmac sha256_generic 16384 0 libsha256 20480 1 sha256_generic drm_panel_orientation_quirks 20480 1 drm snd_soc_core 208896 1 vc4 snd_compress 20480 1 snd_soc_core snd_pcm_dmaengine 20480 1 snd_soc_core syscopyarea 16384 1 drm_kms_helper bcm2835_codec 49152 0 bcm2835_v4l2 45056 0 sysfillrect 16384 1 drm_kms_helper bcm2835_isp 32768 0 sysimgblt 16384 1 drm_kms_helper fb_sys_fops 16384 1 drm_kms_helper cfg80211 745472 1 brcmfmac bcm2835_mmal_vchiq 32768 3 bcm2835_codec,bcm2835_v4l2,bcm2835_isp v4l2_mem2mem 32768 1 bcm2835_codec videobuf2_vmalloc 20480 1 bcm2835_v4l2 videobuf2_dma_contig 20480 2 bcm2835_codec,bcm2835_isp videobuf2_memops 16384 2 videobuf2_vmalloc,videobuf2_dma_contig snd_bcm2835 28672 0 videobuf2_v4l2 28672 4 bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp joydev 28672 0 raspberrypi_hwmon 16384 0 videobuf2_common 57344 5 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,v4l2_mem2mem,bcm2835_isp snd_pcm 122880 4 vc4,snd_bcm2835,snd_soc_core,snd_pcm_dmaengine rfkill 32768 6 bluetooth,cfg80211 snd_timer 40960 1 snd_pcm snd 98304 5 snd_bcm2835,snd_timer,snd_compress,snd_soc_core,snd_pcm videodev 270336 6 bcm2835_codec,videobuf2_v4l2,bcm2835_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp vc_sm_cma 40960 2 bcm2835_mmal_vchiq,bcm2835_isp mc 53248 6 videodev,bcm2835_codec,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem,bcm2835_isp uio_pdrv_genirq 16384 0 uio 24576 1 uio_pdrv_genirq

Okay, then I will edit the config.txt in BerryBoot now and report back.

LeChatNoir666 commented 3 years ago

[pi3]

64-bit stuff for Pi 3

kernel=kernel_rpi64.img arm_64bit=0

results in not booting anymore... :-/ The "Rainbow" screen shows for maybe a half second and then black screen...

MichaIng commented 3 years ago

These defaults for not make sense, there is not point, just a lot of broken installers, when booting any 32-bit OS with a 64-bit kernel. The question is, if this "kernel_rpi64.img" is even able to run in 32-bit mode 🤔 or if the "kernel_rpi0123_aufs.img" is able to boot an RPi 4 (not an issue here, but in general). EDIT: It is not. Try to use kernel_rpi0123_aufs.img instead. ... the problem with this is most likely that no kernel modules are present for it.

Okay, so the obsolete kernel package files have been reinstalled already. You "should" be actually able to purge it: apt purge raspberrypi-kernel (good to have a backup, just in case) 5.4.73v64 seems to be coming from Berryboot, probably it is installed on first boot?

MichaIng commented 3 years ago

Different approach, as I now understood that you install a Berryboot base image, only the bootloader + kernel, and import/install the actual OS from there. To boot a 32-bit OS (correctly), you need to use the 32-bit Berryboot: https://sourceforge.net/projects/berryboot/files/berryboot-20190612-pi0-pi1-pi2-pi3.zip/download Which one did you use?

LeChatNoir666 commented 3 years ago

"EDIT: It is not. Try to use kernel_rpi0123_aufs.img instead." I tried but ended up with 7 times blinking Pi. I use this one: berryboot-20201103-pi4.zip

I made a Backup but I will make a new complete MMC image one. Shall I try then your purge raspberrypi-kernel?

MichaIng commented 3 years ago

I use this one: berryboot-20201103-pi4.zip

Jep, that will be the issue, as it ships the 64-bit kernel only. They should have named it berryboot-20201103-x64.zip as this is what it is, supporting all RPi models that have a 64-bit capable ARM, including RPi 2 v1.2. berryboot

The image I linked above is the correct one to boot the 32-bit DietPi image in intended 32-bit mode: berryboot Sadly it does not contain the RPi 4 kernel, while RPi 4 of course runs pretty well a Raspberry Pi OS 32-bit (+ DietPi 32-bit) image and the 64-bit one is beta with limitations in both cases.

LeChatNoir666 commented 3 years ago

After the Backup on my Notebook I can try to make a Backup of DietPi with BerryBoot himself, "install" then berryboot-20190612-pi0-pi1-pi2-pi3.zip and restore the copy from the "old" BerryBoot. Good idea? :) Or is there any chance to chance somehow the kernel to 32bit? Maybe I can copy the kernel_rpi0123_aufs.img to the running BerryBoot and change it again in the config.txt to:

[pi3]
# 64-bit stuff for Pi 3
kernel=kernel_rpi0123_aufs.img
arm_64bit=0
MichaIng commented 3 years ago

Maybe I can copy the kernel_rpi0123_aufs.img to the running BerryBoot

Then you would need to copy the kernel modules as well. Those are part of the shared.tgz shipped with the Berryboot image. But I guess the device tree files (dtb) do not work with the old kernel, as that is often tied to each other. Simply use the old image so you're sure that bootloader + kernel + device tree (and it's overlays) work well together.

LeChatNoir666 commented 3 years ago

I "ported" my working DietPi (btw: The BEST experience I have so far with my Pi's! Great work! :) ) and uname -a shows me now: Linux BerryBootDietPi 4.19.49v6v7-aufs #1 SMP Tue Jun 11 15:13:27 CEST 2019 armv7l GNU/Linux

I hope it sounds good! mihihi

I will try now to install Emby again and report back then.

Joulinar commented 3 years ago

armv7l that looks good, at least showing 32bit now. Kernel seems to be old, but should be updated during software installation.

LeChatNoir666 commented 3 years ago

Oops... the installation is running now on a backup Ive made with BB! Me dumb! :D Meanwhile: By accident I was in a Menu in DietPi where I entered something with "dietpi-cloudshell". I think, it was in autostart. Now I get everytime a message after booting, that I need authentication to start "dietpi-cloudshell.service". Where can I disable it?

(The installation runs flawless in the background so far happy) Also the problematic 41.deb seems to be installed now:

[  OK  ] DietPi-Software | APT install for: ./41.deb
[  OK  ] DietPi-Software | rm 41.deb

But as it is the wrong OS (i forgot to set the "main" OS as favorite), I need to do it twice, so it will take some time...

Im looking forward that Docker will work then too... :)

UPDATE: Emby is working now on my Backup OS! Big thanks again!

MichaIng commented 3 years ago

Wherever DietPi-CloudShell autostart comes from, run dietpi-autostart 0 to revert to default console login or systemctl disable dietpi-cloudshell to disable the system service manully.

Yes, the kernel is from older 4.19 branch and I just heard from Berryboot dev that the 32-bit images are not supported/maintained anymore. From what I found, it is quite easy to manually enable and build a 32-bit RPi image based on the current stable 5.4 kernel branch, but not sure about Berryboot internals. There is no kernel upgrade feature within the Berryboot GUI, is it?

LeChatNoir666 commented 3 years ago

Puh... I dont see something like "Update" or so in BerryBoot like in PINN Boot Manager. Is the Dev from BerryBoot not one step ahead, because Ive read a little about the 64Bit systems on the Pi's and that its not running flawlessly because of beta status...?

Hmmm...

dietpi@BerryBootDietPi:~$ sudo systemctl disable dietpi-cloudshell
Failed to disable unit: Unit file dietpi-cloudshell.service does not exist.

Btw: Donation is made! :)

Joulinar commented 3 years ago

I just heard from Berryboot dev that the 32-bit images are not supported/maintained anymore

That's bad as 64bit is still Beta and not all thinks are working, still a way to go for the Foundation

LeChatNoir666 commented 3 years ago

There is something I mention now: I uninstalled Docker and its not in the installed software list... But DietPi wants to start the service because i can see: [FAILED] DietPi-Services | start : Docker

Well... as I want to install Docker later anyway its maybe not a big deal but I wanted to mention it...

MichaIng commented 3 years ago

I uninstalled Docker and its not in the installed software list... But DietPi wants to start the service because i can see: [FAILED] DietPi-Services | start : Docker

That's maybe related to your backup restore? By default Docker is not installed, but in your case the docker.service seems to be: systemctl cat docker.service Similar with cloudshell, probably the symlink that enables it is placed while the service file does not exist as this is done only when installing DietPi-CloudShell: ls -al /etc/systemd/system/*wants/dietpi-cloudshell.service

That's bad as 64bit is still Beta and not all thinks are working, still a way to go for the Foundation

Yes, a step that has been made too early IMO, but I can understand it when developer time is limited and maintaining code and support for two architectures and older RPi models could simply cost too much time, so the decision was to go forward and support what will be used in the future and at least does work already, although with some limitations. Important to note is, if I didn't overlook something, that for the Berryboot kernel, headers are missing anyway, so WireGuard and other module builds will fail, similar to how it is with the RPi 64-bit image.

Joulinar commented 3 years ago

Hi guys,

do you like to keep the issue open or can it be closed?

LeChatNoir666 commented 3 years ago

Oops... i forgot to send this comment yesterday...

Luckily I dont needed to restore my backup to the "new" old 32bit BerryBoot. The OS was found and booted normally :)

Over night i left my Pi untouched because of syncing and for about some minutes I wanted to reinstall Docker and run in this problem:

Reinstalling Docker Error

Sorry to bother ya again... :-/

Btw: the "Cloudshell-Problem" is fixed with the "dietpi-autostart 0" command :)

MichaIng commented 3 years ago

Those files should not exist anymore, likely as well a leftover of the backup restore?

Not sure why those should be no "regular files", but please remove them:

rm -Rf 98-dietpi-no_translations 99-dietpi-forceconf 99-dietpi-norecommends

There is a single file now /etc/apt/apt.conf.d/97dietpi which contains all settings from the previous three ones.

But the error should be below these "notifications". You can use the page down or arrow down buttons to scroll the output. It's sadly a known whiptail bug that the scollbar is missing that would indicate this.

Another thing I find strange is the ugly whiptail border, which looks like no UTF-8 locale is selected. Could you paste:

locale
locale -a
cat /etc/default/locale
LeChatNoir666 commented 3 years ago 
LANG=
LANGUAGE=
LC_CTYPE="C.UTF-8"
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=C.UTF-8
dietpi@BerryBootDietPi:~$ locale -a
C
C.UTF-8
C.utf8
POSIX

and

#  File generated by update-locale
LC_ALL=C.UTF-8

Maybe this ugly is because of my Kitty setting... As I most time use Midnight Commander I can see the ugly borders sometimes too, if Kitty dont use my settings in "Translation" -> ISO-8859-1:1998 (Latin-1, West Europe)

I did: rm -Rf 98-dietpi-no_translations 99-dietpi-forceconf 99-dietpi-norecommends and tried the installation of Docker again... Same Output but this time my Pi crashed... Needed to power off and on to "revive" it...

Oh and this is still present at booting: [FAILED] DietPi-Services | start : Docker

Meanwhile Ive tried: sudo apt-get purge docker* and sudo apt-get autoremove Same screen when I try to install Docker...

And here what was cuttet in the screenshot: â + sh -c echo "deb [arch=armhf] https://download.docker.com/linux/raspbian buster stable" > â /etc/apt/sources.list.d/docker.list â sh: 1: cannot create /etc/apt/sources.list.d/docker.list: No such device or address

Next step I did: Deleting the Service in Webmins "Bootup and Shutdown" but docker is tenacious ;)

LeChatNoir666 commented 3 years ago

Update: After restoring a backup from and with dietpi-backup I got now a different error:

Docker Install New Error

Is this now one step forward or backward? :D

MichaIng commented 3 years ago

cannot create /etc/apt/sources.list.d/docker.list: No such device or address

Such btw means that the directory /etc/apt/sources.list.d does not exist. But if that is true, then something went wrong before already since it is or should be never removed.

Okay so now you're further, but Docker package install itself failed. Please try to run it manually and paste the output:

apt install docker-ce
LeChatNoir666 commented 3 years ago

Here it comes:

`dietpi@BerryBootDietPi:~$ sudo apt install docker-ce -y Reading package lists... Done Building dependency tree Reading state information... Done Package docker-ce is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source

N: Ignoring '98-dietpi-no_translations' in directory '/etc/apt/apt.conf.d/' as it is not a regular file N: Ignoring '99-dietpi-forceconf' in directory '/etc/apt/apt.conf.d/' as it is not a regular file N: Ignoring '99-dietpi-norecommends' in directory '/etc/apt/apt.conf.d/' as it is not a regular file N: Ignoring 'docker.list' in directory '/etc/apt/sources.list.d/' as it is not a regular file E: Package 'docker-ce' has no installation candidate`

The strange thing... After restoring I did your: sudo rm -Rf 98-dietpi-no_translations 99-dietpi-forceconf 99-dietpi-norecommends

In Midnight Commander I can see the 3 files have a ~ in front so ~98-dietpi-no_translations if this important for you...

If I decide to make a fresh install of DietPi, can I somehow keep and import my already configured stuff like for samba, webmin, plex aso? I seldom give up, but in worst case.... ;)

Joulinar commented 3 years ago

I guess docker.list source file was not created and therefore apt could not find the package

MichaIng commented 3 years ago

~98-dietpi-no_translations

Not sure where this might come from, maybe when those are opened by an editor and then editor is killed or drive unplugged? Remove them:

sudo rm -Rf '~98-dietpi-no_translations' '~99-dietpi-forceconf' '~99-dietpi-norecommends' '~docker.list'

Quoting is important here, else ~ would be extended to the users home directory.

Let's try to do that manually:

sudo dash -c 'echo "deb [arch=armhf] https://download.docker.com/linux/raspbian buster stable" > /etc/apt/sources.list.d/docker.list'
# Nasty to redirect to a file via sudo^^
sudo apt update
sudo apt install docker-ce
LeChatNoir666 commented 3 years ago

It drive me nuts.. :D Well... deleting the files was only possible as root. With sudo no chance to delete it.

And at first the dash command results in: dash: 1: cannot create /etc/apt/sources.list.d/docker.list: No such device or address

So I checked the directory with MC and found a ~docker.list Deleted it and tried the command again and its loading now... But with:

Unpacking libip6tc0:armhf (1.8.2-4) ...
dpkg: error processing archive /tmp/apt-dpkg-install-5ulU1l/1-libip6tc0_1.8.2-4_armhf.deb (--unpack):
 unable to open triggers ci file '/var/lib/dpkg/info/libip6tc0:armhf.triggers': No such device or address
Selecting previously unselected package libiptc0:armhf.
Preparing to unpack .../2-libiptc0_1.8.2-4_armhf.deb ...
Unpacking libiptc0:armhf (1.8.2-4) ...
dpkg: error processing archive /tmp/apt-dpkg-install-5ulU1l/2-libiptc0_1.8.2-4_armhf.deb (--unpack):
 unable to open triggers ci file '/var/lib/dpkg/info/libiptc0:armhf.triggers': No such device or address
Selecting previously unselected package libnetfilter-conntrack3:armhf.
Preparing to unpack .../3-libnetfilter-conntrack3_1.0.7-1_armhf.deb ...
Unpacking libnetfilter-conntrack3:armhf (1.0.7-1) ...
dpkg: error processing archive /tmp/apt-dpkg-install-5ulU1l/3-libnetfilter-conntrack3_1.0.7-1_armhf.deb (--unpack):
 unable to open triggers ci file '/var/lib/dpkg/info/libnetfilter-conntrack3:armhf.triggers': No such device or address
Selecting previously unselected package libnftnl11:armhf.
Preparing to unpack .../4-libnftnl11_1.1.2-2_armhf.deb ...
Unpacking libnftnl11:armhf (1.1.2-2) ...
dpkg: error processing archive /tmp/apt-dpkg-install-5ulU1l/4-libnftnl11_1.1.2-2_armhf.deb (--unpack):
 unable to open triggers ci file '/var/lib/dpkg/info/libnftnl11:armhf.triggers': No such device or address
Selecting previously unselected package iptables.
Preparing to unpack .../5-iptables_1.8.2-4_armhf.deb ...
Unpacking iptables (1.8.2-4) ...
Selecting previously unselected package containerd.io.
Preparing to unpack .../6-containerd.io_1.3.7-1_armhf.deb ...
Unpacking containerd.io (1.3.7-1) ...
Selecting previously unselected package docker-ce-cli.
Preparing to unpack .../7-docker-ce-cli_5%3a19.03.13~3-0~raspbian-buster_armhf.deb ...
Unpacking docker-ce-cli (5:19.03.13~3-0~raspbian-buster) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../8-docker-ce_5%3a19.03.13~3-0~raspbian-buster_armhf.deb ...
Unpacking docker-ce (5:19.03.13~3-0~raspbian-buster) ...
dpkg: warning: docker-ce: conffile '/etc/default/docker' is not a plain file or symlink (= '/etc/default/docker')
dpkg: warning: docker-ce: conffile '/etc/init.d/docker' is not a plain file or symlink (= '/etc/init.d/docker')
Errors were encountered while processing:
 /tmp/apt-dpkg-install-5ulU1l/1-libip6tc0_1.8.2-4_armhf.deb
 /tmp/apt-dpkg-install-5ulU1l/2-libiptc0_1.8.2-4_armhf.deb
 /tmp/apt-dpkg-install-5ulU1l/3-libnetfilter-conntrack3_1.0.7-1_armhf.deb
 /tmp/apt-dpkg-install-5ulU1l/4-libnftnl11_1.1.2-2_armhf.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Shall i better make a screenshot? Maybe its better for reading then :)

Strange... The 3 ~98-dietpi-blah files are back again...

After this operation, 288 MB of additional disk space will be used.
N: Ignoring '98-dietpi-no_translations' in directory '/etc/apt/apt.conf.d/' as it is not a regular file
N: Ignoring '99-dietpi-forceconf' in directory '/etc/apt/apt.conf.d/' as it is not a regular file
N: Ignoring '99-dietpi-norecommends' in directory '/etc/apt/apt.conf.d/' as it is not a regular file
Do you want to continue? [Y/n] y
Joulinar commented 3 years ago

fixed format for better reading. Pls if possible don't do screenshots

MichaIng commented 3 years ago

I guess /etc/init.d/~docker and /etc/init.d/~docker and /var/lib/dpkg/info/~libip6tc0:armhf.triggers etc exist as well?

We have to do some research where such files might come from, some file system magic, but obviously known since all programs fail to write the file without tilde ~ consistently, or the tilde is more a flag than part of the actual file name.

Meanwhile, could you check, to verify if the root file system is still ext4 and for kernel error:

df -T
dmesg -l emerg,alert,crit,err

And indeed, for pasting code or console output here, best is to use the Markdown fenced code syntax:

line1 line2 etc ...

becomes unformatted, space-preserving monotype "code" box 😃:

line1
line2
etc ...
LeChatNoir666 commented 3 years ago

I was experimenting until deep in the night and felt asleep before I was able to post this...hihihi

Thank you, Joulinar. I'm not experienced with Git, sorry...

Well... after observing and checking whats going wrong with the installation of docker I found out, that my system have many ~ files matching the names, docker fails to write or configure something. After manually (rm -Rf '~*' or something like this didnt worked) deleting all these ~ files, docker installed without errors. But now I cant start the "Docker Application Container Engine":

root@BerryBootDietPi:/etc/systemd/system# systemctl status docker.service
â docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2020-11-25 12:24:15 CET; 1min 59s ago
     Docs: https://docs.docker.com
  Process: 4680 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
 Main PID: 4680 (code=exited, status=1/FAILURE)

Nov 25 12:24:15 BerryBootDietPi systemd[1]: docker.service: Service RestartSec=100ms expired, scheduling restart.
Nov 25 12:24:15 BerryBootDietPi systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Nov 25 12:24:15 BerryBootDietPi systemd[1]: Stopped Docker Application Container Engine.
Nov 25 12:24:15 BerryBootDietPi systemd[1]: docker.service: Start request repeated too quickly.
Nov 25 12:24:15 BerryBootDietPi systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 25 12:24:15 BerryBootDietPi systemd[1]: Failed to start Docker Application Container Engine.

Here the FS:

root@BerryBootDietPi:/etc/systemd/system# df -T
Filesystem     Type     1K-blocks    Used Available Use% Mounted on
dev            devtmpfs    425760       0    425760   0% /dev
none           aufs      14924124 5407264   8739032  39% /
tmpfs          tmpfs       430452       8    430444   1% /dev/shm
tmpfs          tmpfs       430452   13520    416932   4% /run
tmpfs          tmpfs         5120       0      5120   0% /run/lock
tmpfs          tmpfs       430452       0    430452   0% /sys/fs/cgroup
tmpfs          tmpfs       430452   27636    402816   7% /tmp
tmpfs          tmpfs        51200      40     51160   1% /var/log
tmpfs          tmpfs        86088       0     86088   0% /run/user/1000

And:

root@BerryBootDietPi:/etc/systemd/system# dmesg -l emerg,alert,crit,err
[    7.022374] sd 0:0:0:0: [sda] No Caching mode page found
[    7.022399] sd 0:0:0:0: [sda] Assuming drive cache: write through
[    7.200520] sd 1:0:0:0: [sdb] No Caching mode page found
[    7.200543] sd 1:0:0:0: [sdb] Assuming drive cache: write through
[   22.759825] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[   22.973707] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[   22.986227] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Feb 27 2018 03:15:32 version 7.45.154 (r684107 CY) FWID 01-4fbe0b04
[  111.439014] Under-voltage detected! (0x00050005)

Yes I know, I need a new cable... hihihi

And indeed, for pasting code or console output here, best is to use the Markdown fenced code syntax: Pardon? :) Im not experienced with this but I will try to learn it :)

I read, that it can have something to do with docker 19 here: https://github.com/docker/for-linux/issues/476

So atm Im trying to downgrade it to docker-ce_18.06.1~ce~3-0~debian_armhf.deb

OH and I found out, that I dont have this directory: /etc/systemd/system/docker.service.d/

Joulinar commented 3 years ago

for me docker is working fine on RPi3B+ using 32bit image

root@DietPi3:~# docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46
 Built:             Wed Sep 16 17:07:02 2020
 OS/Arch:           linux/arm
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46
  Built:            Wed Sep 16 17:00:52 2020
  OS/Arch:          linux/arm
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
root@DietPi3:~#

This is not needed /etc/systemd/system/docker.service.d/.

Can you post following journalctl -u docker.service

BTW: RootFS looks quite interesting. Doesn't looks like ext4 file system none aufs 14924124 5407264 8739032 39% /

LeChatNoir666 commented 3 years ago

I didnt changed something on the Filesystems...

I tried to install it again but I run again in this:

Setting up docker-ce (18.06.1~ce~3-0~debian)
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service â /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket â /lib/systemd/system/docker.socket.
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "start" failed.
â docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Wed 2020-11-25 14:21:55 CET; 27ms ago
     Docs: https://docs.docker.com
  Process: 10064 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=1/FAILURE)
 Main PID: 10064 (code=exited, status=1/FAILURE)
dpkg: error processing package docker-ce (--install):
 installed docker-ce package post-installation script subprocess returned error exit status 1
Processing triggers for systemd (241-7~deb10u4+rpi1) ...
Errors were encountered while processing:
 docker-ce

journalctl -u docker.service gives this info:

Nov 25 14:21:56 BerryBootDietPi systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Nov 25 14:21:56 BerryBootDietPi systemd[1]: docker.service: Failed with result 'exit-code'.
Nov 25 14:21:56 BerryBootDietPi systemd[1]: Failed to start Docker Application Container Engine.
Nov 25 14:21:56 BerryBootDietPi systemd[1]: docker.service: Service RestartSec=100ms expired, scheduling restart.
Nov 25 14:21:56 BerryBootDietPi systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Nov 25 14:21:56 BerryBootDietPi systemd[1]: Stopped Docker Application Container Engine.
Nov 25 14:21:56 BerryBootDietPi systemd[1]: Starting Docker Application Container Engine...
Nov 25 14:21:56 BerryBootDietPi dockerd[10110]: unable to configure the Docker daemon with file /etc/docker/daemon.json: invalid character '"' after object key:value pair

Ive checked the /etc/docker/daemon.json but I didnt found something like "value pair"... This is the daemon.json:

{
    "data-root": "/mnt/dietpi_userdata/docker-data",
    "log-driver": "journald",
    "log-level": "warn",
    "debug": false
    "storage-driver": "vfs"
}

UPDATE: I deleted now the daemon.json and now I have this:

root@BerryBootDietPi:/etc/systemd/system# systemctl status docker.service
â docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: activating (start) since Wed 2020-11-25 15:01:41 CET; 6s ago
     Docs: https://docs.docker.com
 Main PID: 11503 (dockerd)
    Tasks: 27
   Memory: 29.5M
   CGroup: /system.slice/docker.service
           ââ11503 /usr/bin/dockerd -H fd://
           ââ11511 docker-containerd --config /var/run/docker/containerd/containerd.toml

Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.285036714+01:00" level=warning msg="Your kernel does not support cgroup cfs quotas"
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.285093745+01:00" level=warning msg="Your kernel does not support cgroup rt period"
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.285149162+01:00" level=warning msg="Your kernel does not support cgroup rt runtime"
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.287060412+01:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.287143172+01:00" level=info msg="scheme \"unix\" not registered, fallback to defaul
t scheme" module=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.287323693+01:00" level=info msg="ccResolverWrapper: sending new addresses to cc: [{
unix:///var/run/docker/containerd/docker-containerd.sock 0  <nil>}]" module=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.287387547+01:00" level=info msg="ClientConn switching balancer to \"pick_first\"" m
odule=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.287510047+01:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0x139
fe940, CONNECTING" module=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.288337859+01:00" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0x139
fe940, READY" module=grpc
Nov 25 15:01:42 BerryBootDietPi dockerd[11503]: time="2020-11-25T15:01:42.288536349+01:00" level=info msg="Loading containers: start."

So I have a Kernel problem?

Well... as v19 works well on yours, I will try it again with installing it from your dietpi-software... :)

Joulinar commented 3 years ago

there is a missing comma , on your /etc/docker/daemon.json

should looks like this "debug": false,

I guess it's because you have an additional value "storage-driver": "vfs". This is not present on DietPi-Software installations. Therefore DietPi parameter injection is not expecting it, leading to the missing comma ,

LeChatNoir666 commented 3 years ago

Well, as Ive deleted it, I was able to get one step ahead as I posted above...

Installing via dietpi-software results again in:
Warning: apt-key output should not be parsed (stdout is not a terminal)
+ sh -c echo "deb [arch=armhf] https://download.docker.com/linux/raspbian buster stable" > /etc/apt/sources.list.d/docker.list
+ sh -c apt-get update -qq >/dev/null
+ [ -n  ]
+ sh -c apt-get install -y -qq --no-install-recommends docker-ce >/dev/null
E: Sub-process /usr/bin/dpkg returned an error code (1)
[FAILED] DietPi-Software | Running Docker installer
 - Command: ./DockerInstall.sh

I think will learn about "apt clone" for some settings and try it again then with a fresh system.

EDIT: Here some more info coming from journalctl -xe:

`Nov 25 17:04:17 BerryBootDietPi dhclient[430]: No DHCPOFFERS received.
Nov 25 17:04:17 BerryBootDietPi dhclient[430]: No working leases in persistent database - sleeping.
Nov 25 17:04:23 BerryBootDietPi dhclient[430]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
Nov 25 17:04:27 BerryBootDietPi dhclient[430]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 6
Nov 25 17:04:30 BerryBootDietPi systemd[1]: Reloading.

Nov 25 17:04:44 BerryBootDietPi dockerd[3857]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: Iptables not found

`

And for some last additional info, trying to start dockerd gives:

root@BerryBootDietPi:/etc# dockerd
INFO[2020-11-25T17:27:31.700772715+01:00] Starting up
INFO[2020-11-25T17:27:31.706098132+01:00] parsed scheme: "unix"                         module=grpc
INFO[2020-11-25T17:27:31.706304694+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-11-25T17:27:31.706471517+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2020-11-25T17:27:31.706591725+01:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2020-11-25T17:27:31.711480371+01:00] parsed scheme: "unix"                         module=grpc
INFO[2020-11-25T17:27:31.711618757+01:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2020-11-25T17:27:31.711714642+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}  module=grpc
INFO[2020-11-25T17:27:31.711774694+01:00] ClientConn switching balancer to "pick_first"  module=grpc
WARN[2020-11-25T17:27:37.097945109+01:00] Usage of loopback devices is strongly discouraged for production use. Please use `--storage-opt dm.thinpooldev` or use `man dockerd` to refer to dm.thinpooldev section.  storage-driver=devicemapper
WARN[2020-11-25T17:27:37.213237192+01:00] Base device already exists and has filesystem ext4 on it. User specified filesystem  will be ignored.  storage-driver=devicemapper
INFO[2020-11-25T17:27:37.360359327+01:00] [graphdriver] using prior storage driver: devicemapper
WARN[2020-11-25T17:27:37.360472036+01:00] [graphdriver] WARNING: the devicemapper storage-driver is deprecated, and will be removed in a future release
WARN[2020-11-25T17:27:39.195999900+01:00] Your kernel does not support swap memory limit
WARN[2020-11-25T17:27:39.196151670+01:00] Your kernel does not support cgroup cfs period
WARN[2020-11-25T17:27:39.196241931+01:00] Your kernel does not support cgroup cfs quotas
WARN[2020-11-25T17:27:39.196510941+01:00] Your kernel does not support cgroup rt period
WARN[2020-11-25T17:27:39.196681097+01:00] Your kernel does not support cgroup rt runtime
INFO[2020-11-25T17:27:39.198400264+01:00] Loading containers: start.
WARN[2020-11-25T17:27:39.200447504+01:00] Failed to find iptables: exec: "iptables": executable file not found in $PATH
WARN[2020-11-25T17:27:41.052499430+01:00] unmountAndDeactivate: open /var/lib/docker/devicemapper/mnt: no such file or directory  storage-driver=devicemapper
INFO[2020-11-25T17:27:42.601940732+01:00] stopping event stream following graceful shutdown  error="<nil>" module=libcontainerd namespace=moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: Iptables not found

Anyway... Big Big Big thanks again for helping me!! :) Docker will have now a lower priority for me so I will try it again in near future :)

Joulinar commented 3 years ago

You are missing iptables WARN[2020-11-25T17:27:39.200447504+01:00] Failed to find iptables: exec: "iptables": executable file not found in $PATH and ...failed to create NAT chain DOCKER: Iptables not found

MichaIng commented 3 years ago

Hmm, strange that the installer did not install it, if it's required? Probably depends on the container, e.g. for network forwarding. Try apt install iptables.

Joulinar commented 3 years ago

actually iptables is a dependency 

root@DietPiVM1:~# apt show docker-ce
Package: docker-ce
Version: 5:19.03.13~3-0~debian-buster
Priority: optional
Section: admin
Maintainer: Docker <support@docker.com>
Installed-Size: 107 MB
Depends: docker-ce-cli, containerd.io (>= 1.2.2-3), iptables, libseccomp2 (>= 2.3.0), libc6 (>= 2.3.2), libdevmapper1.02.1 (>= 2:1.02.97), libsystemd0

But I guess Docker installation failed as show above

+ sh -c apt-get install -y -qq --no-install-recommends docker-ce >/dev/null
E: Sub-process /usr/bin/dpkg returned an error code (1)
[FAILED] DietPi-Software | Running Docker installer
MichaIng commented 3 years ago

The dependency should have been installed first, before reaching the configuration step where the service was started. Very strange.

Also the broken config file is strange. We only inject the settings, if the config file exists already, and then all with trailing comma. Else, only if the file not exists, we create it and then correctly skip the last comma, else JSON syntax is wrong: https://github.com/MichaIng/DietPi/blob/master/dietpi/dietpi-software#L12147-L12162 So in this case, something else must have added this setting, not sure what it is for, and then the wrong way. But the install script AFAIK does not touch this config file AND it runs before we touch it. I'm hence confused where it comes from and can not believe that Docker itself is not able to correctly extend its self-chosen JSON config file format (which I hate for being that strict and complicated to script, btw 😄). However easy to fix, adding the missing comma or probably better removing the unexpected and unknown additional setting?

none aufs 14924124 5407264 8739032 39% /

Okay, I think we have the culprit, which is likely also responsible for the added Docker storage driver setting (some buggy Docker init script adding it when seeing aufs root fs). Interesting is the history of this file system type: https://wikipedia.org/wiki/Aufs

That does not sound like a robust root file system to me, at least not one that is well supported. Another issue is that there are some parts of DietPi which expect ext4 as root file system. I'm aiming to liberate this, but first of all with F2FS and Btrfs as it requires to detect and select an additional set of tools for automated first boot file system expansion and at least one other thing I currently don't remember. However that is more minor, a question which file system we ship our own images with where everything must work of course.

It seems that Berryboot requires the OS image to be on/mounted as aufs file system. And that seems to be generally incompatible with Docker: https://raspberrypi.stackexchange.com/a/112582 Probably there is some info or discussion around this topic on Berryboot repo/forum already then, considering how widely used Docker is 🤔.

MichaIng commented 3 years ago

I'm gonna mark this as closed. We know now that Berryboot ships images with the Aufs file system, which breaks a few limitations and additionally the config.txt, kernel, firmware etc on the image are not used, hence cannot be directly configured from within the OS, which is what our scripts rely on.

So while I generally like the idea of Berryboot, the way it works currently cannot be easily supported by us, the amount of rework required is by far to much for our small team. But as long as someone can live with the limitations, the core OS boots and is functional.