RPi | Support for + provide F2FS images

trajano commented 7 years ago

Is it possible to set it up so the boot disk is formatted as F2FS rather than ext4?

https://en.wikipedia.org/wiki/F2FS

I am checking this out to see if I can just convert my existing one. http://whitehorseplanet.org/gate/topics/documentation/public/howto_ext4_to_f2fs_root_partition_raspi.html

Fourdee commented 7 years ago

@trajano

Not tried it personally, but tutorial looks good. You may run into issues if F2FS doesn't support permissions and symlinks. DietPi (and the majority of software in dietpi-software) needs those to function correctly.

trajano commented 7 years ago

Just asked stackexchange before I go on now ^_^

http://unix.stackexchange.com/questions/323155/does-f2fs-support-permissions-and-symlinks/323156#323156

trajano commented 7 years ago

Using f2fs on my box right now. No noticeable change except for a periodic f2fs GC.

trajano commented 7 years ago

This is probably best done on dietpi.txt perhaps we should have an option to specify what file system we want there.

trajano commented 7 years ago

I would hold off on using F2FS due to https://github.com/Fourdee/DietPi/issues/765 power fluctuations can kill a raspberry pi installation especially for a writable file system. The forcefsck recovery does not fix the file system errors as I would expect.

Fourdee commented 7 years ago

Marking as closed. Please reopen if required.

trajano commented 7 years ago

Yup we have to hold off on this until we get get a proper Fsck to work #765

Cjkeenan commented 3 years ago

What is the current state of F2FS and dietpi? I am upgrading to a Pi 4 with a USB SSD and was hoping to use a more modern file system. It also seems that the issue with fsck was resolved. Is there an official way to go about using F2FS or is the guide in the OP still good?

MichaIng commented 3 years ago

Yes the guide will still work.

In case it's okay to still boot from SD card but only have the rootfs on USB SSD you can also use dietpi-drive_manager to move it to an external drive which supports formatting it as F2FS and Btrfs already.

Btw as I was never really experimenting with it: What is the practical benefit of F2FS and Btrfs over ext4? Probably we can create a testing image with the one or the other to allow more wide-spread testing an identifying possible issues with our or 3rd party tools expecting a different one.

Cjkeenan commented 3 years ago

I was under the impression that it was simply a filesystem designed around flash/nand memory, and as such has optimizations for them such as reducing unnecessary writes, associated with journaling filesystems I think, given that flash has a limited amount of those. I do not know the specifics but I read that for any flash memory in some cases it improves performance, but in most cases it improves longevity of the drive. It also has improved power-loss characteristics I believe through its checkpoint system.

I am debating using dietpi or raspios, but since there is a test image for raspios/I found a image builder that supports f2fs, I think I'll be heading down that route, however if you make an image I will be more than happy to test it for you.

For reference, I use my pi headless for Node-RED, a Network UPS Tools server, PiVPN, PiHole, and maybe Home Assistant soon. It's really a just supplement currently to my smart home powered by Hubitat. Any thoughts on if DietPi has any real advantages for that use case?

I just found this reddit post, interesting comments in the about f2fs. It also links to a paper that has an overview.

MichaIng commented 3 years ago

Some benchmark on Linux 5.0: https://www.phoronix.com/scan.php?page=article&item=linux-50-filesystems&num=1 It's on SSD but we would expect similar comparisons on SD cards. Yes I think it makes sense to provide an F2FS testing image. Probably we can add an option to change the rootfs type to our image creation script: https://github.com/MichaIng/DietPi/blob/master/.meta/dietpi-imager But this can only work with direct cmdline access to change rootfstype of course and support by kernel/firmware (RPi).

Cjkeenan commented 3 years ago

OK, sounds good, is there anything you need from me besides a tester? Also do you have any idea how long it would take to modify the image creation script? My current Rpi install has a corrupted gcc compiler so it is in a static state right now without the ability for me to make any changes, so I was hoping to fix it, but I am willing to wait if you think it is relatively easy to get a test image.

I think the only prerequisite software for f2fs support is f2fs-tools.

MichaIng commented 3 years ago

Probably I can do that today (not rewrite DietPi-Imager but creating an RPi F2FS image manually), but have two other tasks in the pipeline.

Cjkeenan commented 3 years ago

Yeah no biggie and not going to hold you to that, just wanted to get a sense was all. Love the work you and I appreciate you.

MichaIng commented 3 years ago

Image for testing up (I didn't find time to test it myself): EDIT: REMOVED INVALID LINK

There is some more work to do to add full support, as well to our build tools:

We use resize2fs -M to shrink the images root file system to the minimal possible size. There is resize.f2fs but this currently only supports increasing the size, not shrinking it. So the only way to create an image is either creating an ext4 image first, shrink it and do conversion, or create a new partitioning with a moreless guessed min fs size so that all files fit. "Apparent" size + file count times block size should be failsafe? Testing required...
Then f2fs-tools needs to be installed for resize.f2fs, fsck.f2fs etc. This cannot be done nicely during DietPi-Imager step but is better done during DietPi-PREP already.
First boot expansion script requires this support as well: https://github.com/MichaIng/DietPi/blob/dev/rootfs/var/lib/dietpi/services/fs_partition_resize.sh#L53 (I hacked resize.f2fs into the image above)

I'll step by step add rootfs type detection (inkl. Btrfs) and using the correct tools to DietPi-FS_partition_resize, DietPi-PREP and DietPi-Imager first and then we need to think about a good concept where and how to add type selection and create or transform the partitions without leaving plenty of free space.

Cjkeenan commented 3 years ago

Sounds amazing, anything special I have to do or just flash via Raspberry Pi Imager? Also any specific tests do you want run or just give day to day feedback?

Also is there a reason the image says "ARMv6" versus the other images for the RPi being "ARMv7" and the RPi 3B/3B+/4 being "ARMv8" as far as I know and I am genuinely curious, are the architectures backwards compatible or does it not really matter?

MichaIng commented 3 years ago

Just flash it with any method you usually do, dd or Rufus or Etcher or RPi Imager (which I never used so far, but plan to suggest/commit 7z support to it 🙂). For now I simply hope it boots and automated file system expansion on first boot works fine.

If you have some benchmark result with ext4, would be interesting to see possible differences with F2FS. Else, as you anyway wanted to setup a new production system, simply use it as normal and report if you face any issue. Since ext4 is so widely used as rootfs default, some 3rd party software might expect it or behave unexpected, however from Linux-side what I read it should be pretty stable and supported and for our scripts I should be able to quickly provide solutions, if required (if there is more than I listed above already).

I hope I find some time next week to do own tests and as well try it on my NanoPi NEO3 which should moreless be a proof of concept to add support for all (Linux 5.X) Armbian-based images (FriendlyARM + PINE64).

Joulinar commented 3 years ago

@MichaIng doesn't looks like fs get extended on first boot. Initial setup failed with

[FAILED] DietPi-Software | Free space check: path=/ | available=86 MiB | required=500 MiB
[FAILED] DietPi-Software | Install aborted due to insufficient free space

MichaIng commented 3 years ago

Does running resize.f2fs /dev/mmcblk0p2 manually work? What does the log say?

cat /var/tmp/dietpi/logs/fs_partition_resize.log

Joulinar commented 3 years ago

root@DietPi:~# cat /var/tmp/dietpi/logs/fs_partition_resize.log
Removed /etc/systemd/system/local-fs.target.wants/dietpi-fs_partition_resize.service.
Disk /dev/mmcblk0: 29.7 GiB, 31914983424 bytes, 62333952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x907af7d0

Old situation:

Device         Boot  Start     End Sectors   Size Id Type
/dev/mmcblk0p1        8192  532479  524288   256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      532480 2274095 1741616 850.4M 83 Linux

/dev/mmcblk0p2:
New situation:
Disklabel type: dos
Disk identifier: 0x907af7d0

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        8192   532479   524288  256M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      532480 62333951 61801472 29.5G 83 Linux

The partition table has been altered.
Info: Mounted device!
        Error: Not available on mounted device!
root@DietPi:~#

EDIT

root@DietPi:~# resize.f2fs /dev/mmcblk0p2
Info: Mounted device!
        Error: Not available on mounted device!
root@DietPi:~#

MichaIng commented 3 years ago

Ahh, dammit, it cannot be used on a mounted device? Hmm, how to achieve that... Probably on a read-only mounted rootfs at least?

mount -o remount,ro /
resize.f2fs /dev/mmcblk0p2
mount -o remount,rw /

To do that reliably on boot, AFAIK our service needs to run earlier, not After= but Before=systemd-remount-fs.service: https://github.com/MichaIng/DietPi/blob/dev/rootfs/etc/systemd/system/dietpi-fs_partition_resize.service#L6 But then the service cannot disable itself: https://github.com/MichaIng/DietPi/blob/dev/rootfs/var/lib/dietpi/services/fs_partition_resize.sh#L4 Puhh, needs some more throughts...

Joulinar commented 3 years ago

not working that easy.

root@DietPi:/boot# mount -o remount,ro /
mount: /: mount point is busy.
root@DietPi:/boot#

MichaIng commented 3 years ago

Yes that is the reason why it needs to be done very early. I'm not sure how to reliably determine the process that "blocks" the fs, sometimes lsof shows not a single write-opened file and it still does not work or only after playing a bid around suddenly without being able to identify any specific step...

Idea:

F2FS images are shipped with rootfs being ro-mounted via fstab.
On first boot, our service expands partition and file system, then manually does mount -o remount,ro /, alters fstab to make rootfs rw-mounted from now on and disables itself.

Joulinar commented 3 years ago

ok I relay did the manual way while mounting the SD card on a different system and expanding the fs. First Boot now completed successful. Need to run some test on it. Boot times seems to be higher. Mounting the device takes quite some time. To be verified.

root@DietPi3:~# systemd-analyze blame
         12.870s dev-mmcblk0p2.device
         11.945s systemd-fsck-root.service

root@DietPi3:~# lsblk -o name,fstype,label,size,ro,type,mountpoint
NAME        FSTYPE LABEL  SIZE RO TYPE MOUNTPOINT
mmcblk0                  29.7G  0 disk
├─mmcblk0p1 vfat   boot   256M  0 part /boot
└─mmcblk0p2 f2fs         29.5G  0 part /
root@DietPi3:~#

Cjkeenan commented 3 years ago

I noticed the same thing when I was experimenting F2FS on Raspios, I think it has something to do with fsck running every boot and that taking extra time with F2FS.

Cjkeenan commented 3 years ago

ok I relay did the manual way while mounting the SD card on a different system and expanding the fs. First Boot now completed successful. Need to run some test on it. Boot times seems to be higher. Mounting the device takes quite some time. To be verified.
root@DietPi3:~# systemd-analyze blame
         12.870s dev-mmcblk0p2.device
         11.945s systemd-fsck-root.service
root@DietPi3:~# lsblk -o name,fstype,label,size,ro,type,mountpoint
NAME        FSTYPE LABEL  SIZE RO TYPE MOUNTPOINT
mmcblk0                  29.7G  0 disk
├─mmcblk0p1 vfat   boot   256M  0 part /boot
└─mmcblk0p2 f2fs         29.5G  0 part /
root@DietPi3:~#

So how exactly did you accomplish this? Because outside of this resizing issue the build needs to be tested so if I can help with that I want to.

Update: I managed to resize it successfully with GParted partition check on another computer after unmounting it.

Cjkeenan commented 3 years ago

So I got basically everything up an running except for a couple quirks, and I don't know if it is with DietPi in general or with F2FS

In Node-RED, I use node-red-contrib-alexa-remote2 to interface with my alexa devices, but it needs to authorize with Amazon's server and save that cookie, but for whatever reason no matter the path I give it, it does not want to actually write that cookie to a file, I am theorizing due to lack of permissions but I do not know for sure. As a result, it requires re-authentication every time a single change is made, which is quite annoying. I think the issue comes down to that the node-red instance is started by the nodered user and I am guessing that user has stripped down permissions. Is there any way to give that user extra write permissions?
DietPi-Backup does not seem to support F2FS and as a result does not generate a backup image unless another path is provided onto another drive. Does anyone know a way of uploading the image backups from DietPi-Backup to a cloud storage such as Google Drive or OneDrive?
Node-RED commands that come with the install such as start, stop, restart, and status do not seem to work, stating that dietpi-services: command not found and if I copy and paste the exact line of code from that command, it works fine. Those commands are located in /usr/bin/ but even when I cd there, and run the code it works fine so idk what is going on.

node-red-start command:

dietpi-services start node-red
journalctl -f -n 0 -u node-red -o cat

Finally in terms of performance, I got a 16% gain over ext4 in the benchmark I ran with the same SD card.

MichaIng commented 3 years ago

Many thanks for testing.

Within the Node-RED environment you need to call DietPi scripts with full paths, e.g. /boot/dietpi/dietpi-services. The aliases are only created for interactive bash shells. And actually it does not make much sense to use dietpi-services which is just an error handled wrapper with nice output, at least for single service handling not more. For Node-RED I'd simply use:

systemctl start node-red
systemctl stop node-red

Also take care that those commands require root permissions, so you need to run those with sudo from within Node-RED.

While sudo can be used to grant the user write permissions, for file I/O I'd grant it native UNIX permissions to those files/directories, or at best write to the services/users home directory where possible: /mnt/dietpi_userdata/node-red Where does this cookie needs or is aimed to be created and which other service/process/user has/needs write access there?

DietPi-Backup does not seem to support F2FS

Good find 👍. No reason to not support it, easy to fix: https://github.com/MichaIng/DietPi/commit/b6d7fb6447fcec4d23fca465c31bdf3f6f22ce2f Changelog: https://github.com/MichaIng/DietPi/commit/6fb8aa99f85a9bb1794c0c585bf7ae498bd27925

Cjkeenan commented 3 years ago

Within the Node-RED environment you need to call DietPi scripts with full paths, e.g. /boot/dietpi/dietpi-services. The aliases are only created for interactive bash shells. And actually it does not make much sense to use dietpi-services which is just an error handled wrapper with nice output, at least for single service handling not more. For Node-RED I'd simply use:
systemctl start node-red
systemctl stop node-red
Also take care that those commands require root permissions, so you need to run those with sudo from within Node-RED.

Thank you so much, got these commands working now. I ended up using the full services path since I did not want to deal with sudo permissions.

While sudo can be used to grant the user write permissions, for file I/O I'd grant it native UNIX permissions to those files/directories, or at best write to the services/users home directory where possible: /mnt/dietpi_userdata/node-red Where does this cookie needs or is aimed to be created and which other service/process/user has/needs write access there?

This is the File Path I am trying to write: /mnt/dietpi_userdata/node-red/alexaAuth/BH_authFile. Am I misunderstanding something or for is that last entry being considered a directory that does not exist so it is freaking out? I tried manually creating a file of that name in that directory with touch but no luck either.

This is the error I get in NR: "EACCES: permission denied, open '/mnt/dietpi_userdata/node-red/alexaAuth/BH_authFile'"

Glad to be of help, is that backup fix already pushed and available via DietPi-Update?

Joulinar commented 3 years ago

Glad to be of help, is that backup fix already pushed an available via DietPi-Update?

nope, not yet available on any update. Will be released on upcoming version 6.33. However you could switch to development branch if you really need it. Or you do the update manual on dietpi-backup script.

Cjkeenan commented 3 years ago

No, no direct need right now, just curious was all. Thank you guys so much for your help.

MichaIng commented 3 years ago

I hope it is easy enough to add it manually to /boot/dietpi/dietpi-backup?

About the permissions error, can you try:

chown -R nodered:nodered /mnt/dietpi_userdata/node-red

Cjkeenan commented 3 years ago

About the permissions error, can you try:
chown -R nodered:nodered /mnt/dietpi_userdata/node-red

Perfect, it worked, is there a way to do this by default? OMG I cannot tell you how happy I am that this fixed it, I have tearing my hair out trying new directories, trying new files, even went to the node's github to see if it was an issue there.

MichaIng commented 3 years ago

I'm glad it's working 😄.

Actually this is done automatically on install: https://github.com/MichaIng/DietPi/blob/master/dietpi/dietpi-software#L8327 Did you create files manually there or used the node-red CLI with a different user than "nodered"?

Cjkeenan commented 3 years ago

No I installed from DietPi-Software and I primarily use this through the Web Interface, so I am not sure if another user is being used there.

MichaIng commented 3 years ago

Hmm strange, I cannot imagine a reason then. However, keep an eye on it if there are permission issues again, probably we did overlook something.

Cjkeenan commented 3 years ago

I will say, and I do not know why exactly, but if there are logs, I can retrieve them for you. But a couple of programs, including pihole, pivpn, and node-red, failed on their initial install and then after I retried via the prompt, it installed fine, but that was the only oddity I remember. All installed via DietPi-Software. The only thing I have installed manually is NUT and the benchmarking tools I used.

Also another oddity and I do not know the cause, but when setting up the NR security measures, I put my password in intially and it seemed to accept it and it worked fine. But then this morning it suddenly said login failed. So I went through the process again, rehashed the same password and then it accepted it again. I will monitor to see if it does it again, but previously I have never seen that issue.

MichaIng commented 3 years ago

Partly done by adding support for F2FS and Btrfs partitions to the code via: #4164 Now we need to create images to test it thoroughly.

Micha-Btz commented 2 years ago

Are there images? I would like to test images with btrfs for /. I really like the snapshot possibility.

For now my raspi 3B+ runs on ssd with ext4 / and btrfs for home. Sadly I don't get a working btrfs for boot. This tutorial seems not to work anymore. https://hackmd.io/@seaQueue/HJOFQphWW?type=view#Btrfs-Preparation

MichaIng commented 2 years ago

First imaging task is to migrate everything to Bullseye. Then I'll see if I find time again to create images with alternative root filesystem types.

I'd skip that subvolume part first, to keep it simply. This can be done later as well. Also elevator=deadline should not be added to cmdline anymore, as it is deprecated, both, the deadline scheduler (replaced with deadline-mq) as well as the elevator option itself.

Micha-Btz commented 2 years ago

@MichaIng since the buster image for the raspis is not more available, can you point me to the latest used cmdline.txt and config.txt? Thanks

Joulinar commented 2 years ago

The image is still available on our server https://dietpi.com/downloads/images/

Micha-Btz commented 2 years ago

yeah, but I need ARMv7 Buster and there is only bullseye

MichaIng commented 2 years ago

There never was an ARMv7 Buster image. Note that the ARMv6 images is compatible with all RPi models.

Current cmdline.txt: https://github.com/MichaIng/DietPi/blob/ad19d27/PREP_SYSTEM_FOR_DIETPI.sh#L560 Current config.txt: https://github.com/MichaIng/DietPi/blob/e662281/config.txt

Joulinar commented 2 years ago

The link above I posted our Buster image for Raspberry Pi

Micha-Btz commented 2 years ago

ok, thanks. Have successfully migrate to btrfs, with converting the original ext4 partition.

rekcodocker commented 1 year ago

This issue warped into a discussion on BTRFS. Fine that this works, but I am really interested in F2FS as it is designed to extend the life of SD cards.

So, still the request: Can DietPi be provided out of the box with an F2FS image? I am definitely willing to test this.

MichaIng commented 1 year ago

We had a working RPi F2FS images available for testing a while ago. Looks like I missed to link it here. Has become easier than ever to generate one.

There are a few e2fsck calls in our build script, but for NanoPi R5S/R6S and Allo GUI images only, otherwise it should work: https://github.com/MichaIng/DietPi/blob/dev/.build/images/dietpi-build Let me trigger a build and we see how it goes.

MichaIng commented 1 year ago

Image ready: https://dietpi.com/downloads/images/testing/ Let me know if you need an ARMv6 (Raspberry Pi 1/Zero) or ARMv7 (Raspberry Pi 2 v1.1) image.

rekcodocker commented 1 year ago

That is absolutely fantastic, thank you so much!

Found an issue with the size. I install it on a 16GB sd card. I tried to install docker-compose and it failed because there was not enough disk space.

Tried 'resize filesystem' using dietpi-drivemanager. It reboots.

Issue: size has not changed.

Tried to alter the filesystem on my laptop. I found that the partition is sized at 14.7GB. But the f2fs filesystem reports 890M (!)

I thought that maybe the partition is resized, but the filesystem still needs to 'grow' into the new partition size. Tried 'resize.f2fs' on my laptop. Did not work, this was the output:

sudo resize.f2fs /dev/sdb2 
Info: [/dev/sdb2] Disk Model: STORAGE DEVICE  
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 30845952 (15061 MB)
Info: MKFS version
  "Linux version 5.15.0-1031-azure (buildd@lcy02-amd64-010) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #38-Ubuntu SMP Mon Jan 9 12:49:59 UTC 2023"
Info: FSCK version
  from "Linux version 5.15.0-1031-azure (buildd@lcy02-amd64-010) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #38-Ubuntu SMP Mon Jan 9 12:49:59 UTC 2023"
    to "Linux version 5.15.0-58-generic (buildd@lcy02-amd64-101) (gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023"
Info: superblock features = 0 : 
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 1826816 (892 MB)
Info: CKPT version = 223718b2
Info: Duplicate valid checkpoint to mirror position 1024 -> 512
Info: Write valid nat_bits in checkpoint
        Error: Device size is not sufficient for F2FS volume, more segment needed =12534

MichaIng / DietPi

RPi | Support for + provide F2FS images #606