Image | Raspberry Pi OS (64-bit)

FusionPlmH commented 4 years ago

ADMIN EDIT

Beta Image ready for testing: https://dietpi.com/downloads/images/

Known missing/broken software installs, until the RPi repository ships arm64 packages:

Kodi: http://archive.raspberrypi.org/debian/pool/main/k/kodi/
Chromium: http://archive.raspberrypi.org/debian/pool/main/c/chromium-browser/

Due to the newest new, Raspberry Pi has been started the offical 64bits image on beta .

Raspberry Pi OS (64 bit) beta test forum thread: https://www.raspberrypi.org/forums/viewtopic.php?t=275370

Downloads: https://downloads.raspberrypi.org/raspios_lite_arm64/images/

I wanna know Does dietpi want to follow offical , to make up a 64bits OS ? @MichaIng

Joulinar commented 4 years ago

Hi,

many thanks for your request. Basically DietPi on Raspberry Pi devices is a Raspbian Lite and it will be support. As well it's planned to create 64bit images for RPi2 PCB1.2 and up.

See https://dietpi.com/phpbb/viewtopic.php?p=25160#p25160

FusionPlmH commented 4 years ago

So we may start to have some work on it

MichaIng commented 4 years ago

Lol, copy&paste from forum:

I'm was just checking the 64bit beta OS. It is not called Raspbian, lets see which repo they use: https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2020-05-28/ ... Lol, it's default Debian:

deb http://deb.debian.org/debian buster main contrib non-free
deb http://deb.debian.org/debian-security/ buster/updates main contrib non-free
deb http://deb.debian.org/debian buster-updates main contrib non-free

Actually makes sense. Raspbian was actually only created since the first RPi1 (and RPi Zero now) are armv6hf which is not supported by Debian. For ARMv7 + ARMv8 boards there is no point in using a different distro. All RPi specific firmware/kernel packages, special board-feature/GPU builds of things like Kodi, ffmpeg etc, are all provided by the archive.raspberrypi.org repo and have nothing to do with Raspbian.

I think this means that we need to change our code in many places so that we do not assume RPi <=> Raspbian anymore but instead check /etc/os-release for whether it's Raspbian or Debian.

GitHub repo: https://github.com/raspberrypi/Raspberry-Pi-OS-64bit

MichaIng commented 4 years ago

I've started working on this: #3617

RPi is now detected based on the existence of a device tree file, which is required to boot any RPi with official kernel. I wanted to allow correct detection as well when chrooting into such image or using systemd-nspawn, hence it must not depend on /proc or /sys content which one would use usually. Checking for a device tree file looked like the most compatible, still quite accurate solution. Basically it must be an ondisk file (or content) which is only present on RPi images (as much assured as we can be) while being present on most of them, even when e.g. customising it, e.g. removing the APT kernel package and installing via rpi-update only, or compiling oneself. If someone finds a more accurate/failsafe/better method, I'm open to change it. Note that this check is only reached of no /etc/.dietpi_hw_model_identifier is present, which is created during DietPi-PREP for all non-RPi models.
On RPi, there is a new global variable G_RASPBIAN introduced which equals 1 by default and 0 if /etc/os-release contains ID=debian. Also here in rare circumstances theoretically this file can be wrong, so the choice is whether to assume Raspbian by default (and check for Debian explicitly) or the other way round. For now, where Raspbian is still the most used image, we'll assume it by default.
ToDo: Implement into DietPi-Survey, so we can see how the share between both versions develops over time and could and differentiate benchmark results between both, which might be quite interesting.

Now the new variable needs to be rolled out into our scripts, most importantly the one for APT sources.list creation/change. Means we'll need to go through every G_HW_MODEL call, see where we do some Raspbian-only step if we find RPi and in case check for G_RASPBIAN instead or additionally. Puhh... will take some hours, lets see 😅.

Joulinar commented 4 years ago

sounds like quite a lot of search > find > replace > next 😂

MichaIng commented 4 years ago

Good that we have grep and https://github.com/MichaIng/DietPi/search?q=G_HW_MODEL&unscoped_q=G_HW_MODEL 😄.

MichaIng commented 4 years ago

Support ready for review/testing. I'll now create an image and test DietPi-PREP compatibility by this.

Joulinar commented 4 years ago

Let me know if you need something tested. Otherwise I will download Raspberry OS 64bit and run PREP script

MichaIng commented 4 years ago

Otherwise I will download Raspberry OS 64bit and run PREP script

Already in progress 😃, would be great if you could test the resulting image instead.

The image can run on all RPi models from RPi1 v1.2 on, those have an ARMv8-capable CPU.

Joulinar commented 4 years ago

Currently I have a RPi3B+ and RPi4B 4GB only. Don't have that much SBC's :smile:

MichaIng commented 4 years ago

Here we go: https://dietpi.com/downloads/images/

A note to all testers:

This is bloody alpha on several layers!
Kernel is a snapshot from the current in-dev RPi Linux 5.4 branch, possibly this commit: https://github.com/raspberrypi/firmware/tree/62fc8c01165a80021054a430182b504f7b877c2d
Many think that you expect from Raspbian (and DietPi RPi) images have not yet been finished/ported to 64-bit. See the bug tracker for Raspberry Pi OS (64-bit) for known issues: https://github.com/raspberrypi/Raspberry-Pi-OS-64bit/issues
DietPi code is based on current dev branch, support for Debian-based RPi images (which Raspberry Pi OS (64-bit) is) has been added in one full day of searching, coding, reviewing and testing, so bear with me if I might have overseen something 😉.

All RPi models from RPi2 v1.2 and up (excluding RPi Zero) are supported, hence all which have an ARMv8-capable CPU: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications

Joulinar commented 4 years ago

ok first observation. On boot, the system tries to login as user pi automatically.

20200624_201847

MichaIng commented 4 years ago

Oh, there must be some getty@tty1.service drop-in left:

systemctl cat getty@tty1

Found it:

cat /etc/systemd/system/getty@tty1.service.d/autologin.conf
[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin pi --noclear %I $TERM

=> e.g. on SSH session

rm /etc/systemd/system/getty@tty1.service.d/autologin.conf
systemctl daemon-reload
systemctl restart getty@tty1

Done by PREP script: https://github.com/MichaIng/DietPi/commit/3f4cc7b0e14d275fe82b7974fce78e10a22e0eb2

Image repacked with removed "pi" autologin.

Joulinar commented 4 years ago

sorry for delay. I used the old image still. attached the install output from CLI.

RPi64bit.txt

I tried to change NTP to my Router, but this is failing. Same behaviour once installation finished using dietpi-config

root@DietPi:~# dietpi-config
[  OK  ] DietPi-Config | Setting in /boot/dietpi.txt adjusted: CONFIG_NTP_MIRROR=Gateway
[ SUB1 ] DietPi-Set_software > ntpd-mode (2)
[FAILED] DietPi-Set_software | No local gateway detected. Reverting NTP mirror back to system defaults.
[  OK  ] DietPi-Set_software | Setting in /boot/dietpi.txt adjusted: CONFIG_NTP_MODE=2
[FAILED] ntpd-mode 2 | An issue has occurred
[  OK  ] DietPi-Run_NTPD | systemctl restart systemd-timesyncd
[  OK  ] DietPi-Run_NTPD | systemd-timesyncd synced
[  OK  ] Network time sync | Completed

MichaIng commented 4 years ago

I used the old image still.

Jep that is perfectly fine, no difference besides the removed autologin.

I tried to change NTP to my Router, but this is failing.

Good find 👍, a syntax issue (local does not allow to assign and alter a variable in one call, ; required): https://github.com/MichaIng/DietPi/commit/66f937ab3e4e2edba74918bd8107162c36078de0

Joulinar commented 4 years ago

how did you like to continue from here? Not sure what to test currently. Do we need to run thru all software title to ensure they are running on Raspberry OS 64bit? I already found WireGuard not working. I guess due to kernel dependency not fully available yet. Maybe I'm wrong.

MichaIng commented 4 years ago

Yes WireGuard currently fails as the is no matching headers package available yet for the arm64 kernel package: http://archive.raspberrypi.org/debian/pool/main/r/raspberrypi-firmware/
Amiberry requires as well new pi64 target builds.
OpenTyrian can/needs to be pulled from Debian repo (is not available on Raspbian): https://packages.debian.org/search?searchon=names&keywords=opentyrian
DXX-Rebirth need to be rebuild for arm64 or in general as an update, otherwise disabled for arm64 for now.

And 3rd party installers might not yet be prepared, e.g. Pi-hole, PiVPN, Docker.

Joulinar commented 4 years ago

ok let's wait on the further development of Raspberry OS 64bit.

camdenorrb commented 4 years ago

This is awesome! :)

FusionPlmH commented 4 years ago

Just managed to install on running on raspberrypi 3b+ , with pihole and unbound , it working like a charm performance over better than Ubuntu 20.04 64bit and dietpi 32bit system

MichaIng commented 4 years ago

Many thanks for testing. Awesome. Please let us know if you face any issues at a later time or if any expected feature is missing.

irousom commented 4 years ago

How I missed this earlier I don't know, but will share some other results after many weeks' time on and off getting a 64-bit OS to run correctly that would execute the DietPi conversion script successfully. Summary:

Starting with the new Raspberry Pi OS 64-bit on Raspberry Pi 3 B+, I could run the conversion script - but the amount of bloat that was removed was quite high.
Starting with the new Raspberry Pi OS 64-bit on Raspberry Pi 4, I could run the conversion script, but the subsequent reboot would fail.
Starting with the vanilla Debian image in UEFI mode vis-a-vis https://pete.akeo.ie/2019/07/installing-debian-arm64-on-raspberry-pi.html was very fast and ran the conversion script just fine. The resultant image was small and easy to reproduce.
I plan on testing the alpha image offered here on the Raspberry Pi 4 and compare the results to the process in step #3 above but with the RPi 4 firmware https://github.com/pftf/RPi4/releases instead.

@MichaIng - will this ultimately help meet your objectives, or is this testing redundant with previous efforts?

MichaIng commented 4 years ago

Starting with the new Raspberry Pi OS 64-bit on Raspberry Pi 3 B+, I could run the conversion script - but the amount of bloat that was removed was quite high.

Indeed, the current 64-bit image is a huge fully fledged desktop image, so not the best start. However there will be a Lite version for sure at a later time, when a final release comes closer.

Starting with the new Raspberry Pi OS 64-bit on Raspberry Pi 4, I could run the conversion script, but the subsequent reboot would fail.

Strange, that worked quite fine for our testing image. Did you face any error messages or some display (or serial console) output on failing boot? Since that image contains future version bootloader/kernel/firmware packages, the essential parts should not be touched.

Starting with the vanilla Debian image in UEFI mode vis-a-vis https://pete.akeo.ie/2019/07/installing-debian-arm64-on-raspberry-pi.html was very fast and ran the conversion script just fine. The resultant image was small and easy to reproduce.

Not sure if UEFI has any benefit here, however good to know it works. Yes result for sure is cleaner. Cleanest and what I aim for mid/long-term is creating our images from scratch via debootstrap which allows to still install bootloader+kernel+firmware from official repositories, archive.raspberrypi.org in this case.

I plan on testing the alpha image offered here on the Raspberry Pi 4 and compare the results to the process in step #3 above but with the RPi 4 firmware https://github.com/pftf/RPi4/releases instead.

While this is generally interesting and might bring some future benefits, we don't aim to go with experimental firmware and IMO the use for UEFI here is currently minimal. But at least we can take care that DietPi-PREP is compatible with an RPi UEFI images in general and probably the RPi foundation is gonna pick it up by times as well.

From my side, most important is that our scripts and software installers work as expected with the new official Raspberry Pi OS 64-bit, as this is a fundamental difference compared to Raspbian and we needed to change a lot of code throughout DietPi 🙂.

magloui commented 4 years ago

Start test with x64 version I mount old NTFS drive without issue, and share with samba without problem add drive to fstab But after restart the drive is not mounted again, need to mount manually And in samba I have access to the share but not see the content anymore I check multiple time the config and all seems correct, and work perfectly on x86 version waiting for my new hard drive for testing with Ext4 partition

Joulinar commented 4 years ago

pls can you have a look to /etc/fstab how the entry looks like for your drive. Usually it should be auto mounted as soon as someone is trying to access

magloui commented 4 years ago

pls can you have a look to /etc/fstab how the entry looks like for your drive. Usually it should be auto mounted as soon as someone is trying to access

I check in the x86 config fstab UUID=E69EB5EF9EB5B87F /mnt/plex ntfs noatime,lazytime,rw,permissions,nofail,noauto,x-systemd.automount

In X64 is not the same UUID=E69EB5EF9EB5B87F /mnt/plex ntfs mft_zone_multiplier=1,fmask=0177,dmask=077,nls=utf8,errors=continue,noatime,noauto 0 0

But after copy x86 in the fstab of x64 and reboot, issue exactly the same, no automount and samba return an empty link

But may be it's mistake work on the system only since 2 week ;-)

MichaIng commented 4 years ago

UUID=E69EB5EF9EB5B87F /mnt/plex ntfs mft_zone_multiplier=1,fmask=0177,dmask=077,nls=utf8,errors=continue,noatime,noauto 0 0

But this is not the result of dietpi-drive_manager. The x86 is what dietpi-drive_manager adds, containing x-systemd.automount which is responsible for automount. Please run dietpi-drive_manager 3 on the x64 system which should enable automount.

magloui commented 4 years ago

Ok I manually mount drive, clean up and i mount drive with dietpi-drive_manager, I saw system install ntfs-3g and after my drive is mounted all the time, I used and samba correctly show disk content Thank's

Pillendreher commented 4 years ago

Since I was reinstalling everything anyway, I decided to give this one a try.

1.Initial flash and install to SD works. 2.Moving everything to a USB SSD with the latest bootloader work as well. 3.SSH is functional as well. 4.Resize of the RootFS to use the full SSD capacity works as well. 5.Ethernet works, Internet connectivity test as well. 6.The following apps were installed sucessfuly and are working without any issues: -vsFTPD -RealVNC Server -SABNzbd -nzbhydra2 -Nextcloud -Sonarr -Radarr

I'll update this as I move along with configuring the Pi4.

MichaIng commented 4 years ago

Many thanks for testing and feedback!

Pillendreher commented 4 years ago

One issue I've been able to reproduce: Whenever I upload files to the SSD via FTP, my SSH client suddenly turns very sluggish (there's a noticeable pause between key strokes turning into letters). Connecting via VNC, the system itself is sluggish as well. The cursor is moving slowly, the file manager opens five seconds after clicking on the icon.

htop has the load average of the last minute at 1.71.

MichaIng commented 4 years ago

Which file system is on the SSD? VNC is sluggish only during file upload or in general? Which VNC server do you use? TigerVNC or RealVNC? And which desktop?

Sorry for question bombardment, but those could all be relevant 😉.

Pillendreher commented 4 years ago

Which file system is on the SSD?

ext4

VNC is sluggish only during file upload or in general?

only during the file upload. Otherwise is very quick. The same goes for SSH btw.

Which VNC server do you use? TigerVNC or RealVNC?

RealVNC

And which desktop?

I'm afraid I don't know what you mean - the Raspberry Pi is not set up with a monitor and I didn't install another UI. Under dietpi-config => Display Options => Display Resolution I'm using the standard "vc4-kms-v3d-pi4 : OpenGL | 1920 x 1080"

Sorry for question bombardment, but those could all be relevant 😉.

No worries at all, I'm just glad I can be of help.

MichaIng commented 4 years ago

Okay, I'm not sure how CPU intense FTP usually is (I use SFTP for everything), also of course it depends on transfer rate, as the SSD write speed very likely is not the limiting factor. Have you tried other file transfer methods, just to compare?

ext4 and RealVNC should be best options, although, are those very large files you upload (GiB sizes) or many smaller/usual ones?

With RealVNC, a desktop environment is installed, if you didn't choose one explicitly, its LXDE, so the lightest option. However desktop and VNC do not play a role here since the FTP transfer and/or writes through USB to SSD ext4 fs is the culprit.

Ah and did you try another USB port, respectively compared USB3.0 with USB2.0?

FusionPlmH commented 4 years ago

Screenshot_20200716-215007_Chrome Now just update the pihole to v5.1 result like this , and I have been try to uninstall and install it

Joulinar commented 4 years ago

I had the same yesterday after doing the update on a normal 32bit image. Restart your system, close browser and clear cache.

FusionPlmH commented 4 years ago

Screenshot_20200717-020522_Chrome Screenshot_20200717-020528_Chrome

Now i have been restall all the dietpi + pihole + unbound

i use unbound as my own primary dns server

now i face another issues like

after use pihole v5.1

i was not able to input my local ipv6 dns server ::1#5335 address at the pihole web admin page

here is my unbound config: https://github.com/FusionPlmH/unbound/blob/master/pi-hole.conf

Joulinar commented 4 years ago

I don't think this is related to DietPi. Might be better to check on PiHole board.

FusionPlmH commented 4 years ago

I don't think this is related to DietPi. Might be better to check on PiHole board.

just checking on pihole board , the answer look like "No need for ipv6 loopback when ipv4 is available",seems this answer is acceptable for me .

Pillendreher commented 4 years ago

Okay, I'm not sure how CPU intense FTP usually is (I use SFTP for everything), also of course it depends on transfer rate, as the SSD write speed very likely is not the limiting factor. Have you tried other file transfer methods, just to compare?

ext4 and RealVNC should be best options, although, are those very large files you upload (GiB sizes) or many smaller/usual ones?

With RealVNC, a desktop environment is installed, if you didn't choose one explicitly, its LXDE, so the lightest option. However desktop and VNC do not play a role here since the FTP transfer and/or writes through USB to SSD ext4 fs is the culprit.

Ah and did you try another USB port, respectively compared USB3.0 with USB2.0?

Oh my, this is embarassing....I think I figured out what was happening....

I was outside yesterday and pushed a movie file (30 gb) to the Pi over Wifi. Naturally, the connection wasn't the best. I think the whole sluggishness was caused due to the simple fact that I was maxing out the connection to the Pi with the upload and therefore ssh/vnc appeared sluggish because there was not enough bandwith left.

I just tried both downloading and uploading big files to and from the Pi via both Samba and FTP over Gigabit Ethernet and didn't encounter any issues.

Consider this issue irrelevant for now. I'll give it another try with my laptop over wifi in the house just to make sure my theory is correct...

MichaIng commented 4 years ago

No worries, great that you found the root 😃.

CactiChameleon9 commented 4 years ago

Hello... I am having issues with wifi. I have found that a headless install does not seem to work by just entering in the details before first boot as I could not ssh or find the pi 4 on the network.

So I reinstall and change nothing: On first boot, I set up wifi and it then "updates" dietpi, but I think it downloads and installs 32 bit stuff (downloads and replaces a lot of stuff in /boot). I setup installing only proftpd and it reboots me. I then cannot connect to wifi, with it being "not found" in the dietpi-config network menu. I think there is an issue with the detecting an update system, does it need to be pointing to a 64bit download place instead?

Edit I just tried with the 32bit image and I have the same issue... Ill open a new issue

Joulinar commented 4 years ago

I guess related to #3685 and seems to be fixed

MichaIng commented 4 years ago

Image has been updated: https://dietpi.com/downloads/images/

Based now on DietPi master/stable branch
Based on now released stable RPi Linux kernel 5.4

I'd call it beta phase now since there are still feature missing on "Raspberry Pi OS (64-bit)" in general and its the first official Linux 5.4 release.

CactiChameleon9 commented 4 years ago

That's great news. I assume I'll have to start fresh to install. Ill do that as soon as libreoffice online compiles for my nextcloud.

Pillendreher commented 4 years ago

Image has been updated: https://dietpi.com/downloads/images/
* Based now on DietPi master/stable branch

* Based on now released stable RPi Linux kernel 5.4
I'd call it beta phase now since there are still feature missing on "Raspberry Pi OS (64-bit)" in general and its the first official Linux 5.4 release.

Thank you! Any chance to update without having to do a complete re-install?

MichaIng commented 4 years ago

No need to start fresh, simply do:

G_CONFIG_INJECT 'DEV_GITBRANCH=' 'DEV_GITBRANCH=master' /boot/dietpi.txt

to switch to master branch. This is effective with next DietPi release.

apt update
apt full-upgrade

to upgrade kernel packages.

csolisr commented 4 years ago

No need to start fresh, simply do:
G_CONFIG_INJECT 'DEV_GITBRANCH=' 'DEV_GITBRANCH=master' /boot/dietpi.txt
to switch to master branch. This is effective with next DietPi release.
apt update
apt full-upgrade
to upgrade kernel packages.

To clarify: does this allow users of the 32-bit version to upgrade to the 64-bit version in-place, or does this only apply for users of the previous 64-bit image?

Joulinar commented 4 years ago

This applies for 64bit image only.

MichaIng commented 4 years ago

You in fact cannot upgrade/switch from 32-bit to 64-bit. Raspbian (Raspberry Pi OS 32-bit) and the new Raspberry Pi OS 64-bit are fundamentally different, the whole system is build up on two different major repositories (Raspbian armhf vs Debian arm64).

The underlying OS is btw independent from DietPi version/branch vise-versa. DietPi simply detects detects if Raspbian or Debian and acts accordingly (since DietPi v6.31).

MichaIng / DietPi