search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.8k stars 494 forks source link

DietPi-Drive_Manager | Re-allow moving rootfs on single-partition images #5295

Open yandritos opened 2 years ago

yandritos commented 2 years ago

Additional Information (if applicable)

Can this issue be replicated on a fresh installation of DietPi? Yes

Steps to reproduce After a fresh install of dietPI in Odroid C2 Execute the dietpi-drive_manager Select an external HDD connected via USB

-(My comment) Missing the option transfer rootFS, only the user data can be transferred.

Expected behaviour -Complete the transfer process to the external hard drive

Actual behaviour -(My comment) I can't transfer rootFS to the external hard drive, the option is missing. If I use the old ISO from the last year on my micro SD the option is there and the whole os filesystem goes to the external HDD, with this last release this option is not available, I can only move the user data which is not enough for my purposes. ¿Is the option in a different place or was it simply removed?

Joulinar commented 2 years ago

What file system format your HDD has?

yandritos commented 2 years ago

Ext4 the same one I used for the previous dietpi version, is the sale hdd which I had working with the previous version.

El sáb., 19 feb. 2022 11:47, Joulinar @.***> escribió:

What file system format your HDD has?

— Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/5295#issuecomment-1045989186, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVTQ2NS6KV27E4UIVJISVD3U35YKBANCNFSM5O2KYCAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

Joulinar commented 2 years ago

Ah another possibility, that your system has no single rootFS partition anymore. Therefore you are not able to transfer.

MichaIng commented 2 years ago

This is expected. The new Odroid C2 image doesn't need and have a dedicated boot partition anymore. In this case it doesn't make sense and would be complicated to setup to copy the rootfs without /boot directory elsewhere. Simply clone the whole SD card onto the USB drive, at best flash the DietPi image on the USB drive in the first place.

In theory this can be done from the running system as well: https://dietpi.com/blog/?p=1236 I however strongly recommend to do the cloning offline from an external system.

Then you do not need the SD card at all.

Though the question remains if the Odroid C2 supports USB boot hardware wise. I couldn't find a clear answer for now, simply testing it would be best.

yandritos commented 2 years ago

Hi @Joulinar you are fully right, in the same way as Michalng.

Hi @MichaIng thanks for the fast answer. Unluckily the Odroid C2 is unable to boot the firmware from any other device different than the microSD card or the eMMC memory (Priority is eMMC if not present microSD, explained here https://forum.odroid.com/viewtopic.php?t=23451 ).

So, for the Odroid C2 is not an option to have the global filesystem in the USB of the Odroid C2 with this new DietPI release in an easy way (like it was in the version 7 with the option "transfer rootFS"Space). For anyone on my situation I assume it is required to have a dedicated partition boot.

@MichaIng, @Joulinar is there an easy way to have back a release (or installation option) with a dedicated boot partition allowing then to have back the "rootFS" transfer? If not I'll be back to the release core 7, it's a pitty to sacrify the new Linux11 advantages but not critical in my case.

Thanks for all this great work!

MichaIng commented 2 years ago

Ah, that is a bumper 😢. Have you at least tried it once? The post is six years old and while it is talked much about U-Boot configs and defaults I see no clear statement that the hardware itself is unable to read the bootloader from elsewhere, and it may have changed with revisions.

If it is really not possible, we may at least try it with a dedicated boot partition, but I fear that the Armbian U-Boot does not work with that, neither a FAT boot partition, nor a dedicated boot partition at all. On Odroid N2 at least it was impossible. And going back to legacy kernel causes much more serious trouble, considering that eMMC is probably the best option anyway 😉.

yandritos commented 2 years ago

I am going to try it anyway it is not hard to complete :) , maybe I can have a great surprise. I'll be back with my feedback as soon as finish the test. I'll simply flash a usb pendrive with dietpi (using rufus).

Thanks again!

yandritos commented 2 years ago

Well, I tried but no way at any USB port. As I don't have an eMMC unit I think I am going back to the old ISO I have with version core 7, it is not a big drama :)

Thanks for all!

MichaIng commented 2 years ago

Okay, that is really a downside then. I can build an image for C2 with a dedicated FAT boot partition and one with a dedicated ext4 boot partition, so we can test how well this works.

yandritos commented 2 years ago

If you can do that I'll be more than happy to test it :)

El sáb, 19 feb 2022 a las 21:14, MichaIng @.***>) escribió:

Okay, that is really a downside then. I can build an image for C2 with a dedicated FAT boot partition and one with a dedicated ext4 boot partition, so we can test how well this works.

— Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/5295#issuecomment-1046095513, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVTQ2NRK5YVIL4RD5UZU473U372YVANCNFSM5O2KYCAQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- David Romero Portal Consultor Tecnológico SAP XI/PI SOA, EAI & ESB

Ruud68 commented 1 year ago

I would like to add my thoughts / solution (?) here, thanks @MichaIng for pinging me over on the DietPi forum. So I also have a 'must have' requirement for having rootfs on a USB drive (performance, durability, stability). I was able to achieve this with the current DietPi version without breaking things. So basically I 'embraced' the decision of DietPi to go for one-have-it-all partition. The limitiation here is not in the single partition making it impossible to have two mount points (one for /boot and one for /) but in our trying to stick with / go back to the separated partitions.

I used the BIND mount method: with bind mount you do not mount a partition (the bootpartition), but you mount a directory (/boot).

So I described how to do it here (note that this is basically the same as armbian does it with the exception of the bind mount method). Upside to this is that the armbian nand-sata-kernel script that does the the whole move to USB drive thing, can be used for DietPi with a (very) few alterations.

https://dietpi.com/forum/t/tutorial-odroid-c4-rootfs-on-usb-drive/15662

@yandritos I know that this is an old issue, but if you are still able to test / comment that would be great.

If needed I could also play a role in developing this as a feature for DietPi, if not needed also okay as just following my own instructions does what it needs to be doing for me :)

MichaIng commented 1 year ago

The bind mount is clever to avoid the need to basically erase the SD card apart of its boot dir and move that boot dir to its root. Is that what Armbian's tools is doing? Cleaner, but if for some reason mounting the USB drive as root does not work, you cannot revert easily.

However, instead of a bind mount, which needs to be done and succeed on every boot, I'd do a symlink:

rmdir /mnt/usb/boot
ln -s /mnt/sdcard/boot /mnt/usb/boot

Cleaning up everything (moving SD card's boot into its root) could be done as an optional step after the system has booted successfully with root on USB.

yandritos commented 1 year ago

Hi guys, @Ruud68 and @MichaIng

I know this is an old issue and I am happy to get news :) I still running Dietpi 7.61 on my Odroid C2 (with the security fixes I could add) as my main home NAS + Domotic Centric + VPN server just because of this USB boot feature removal.

So, if you provide me an easy way for me to have this with a low time task in the newer releases I´ll be more than happy to test. This year The family is bigger and I have less time, but I´ll do my best to provide a fast feedback.

Have a nice weekend!

Ruud68 commented 1 year ago

difference between symlink and (bind) mount is that the symlink is done in a later stage. Mounting is always one of the first things done while booting. I do not have enough 'experience' with dietpi to evaluate if this is good or bad. I think I saw something like storing a boot step / install step into a file upon boot, so not sure things like that will be impacted when or not.

I have no bad experiences with bind mount, but also ot with symlinking. So either way is good by me as long as it works.

As for Armbian, I think they have the two partitions so no bind mount / symlink needed: to be honost I have tried many distributions and also versions of armbian so I do not recall how they did it explicitly.

Benefits of not being in production yet with my setup is that I can experiment :) I have only one board / sdcard, so once in production it will be 'impossible' to test / develop this

MichaIng commented 1 year ago

difference between symlink and (bind) mount is that the symlink is done in a later stage.

The symlink does not need to be done, it is simply there, as an integral part of the filesystem itself. This is why I prefer it 😉.

As for Armbian, I think they have the two partitions

Armbian ships their images with a single ext4 partition, just like we do. So their tool needs to do something similar you do, with symlink, bind mount of reordering the SD cards content 😉.

Benefits of not being in production yet with my setup is that I can experiment :)

If you have ext4 access from another system to revert in case, would be great if you could test it with a symlink, removing the bind mount instead.

Ruud68 commented 1 year ago

ok, just totally 'borked' my setup by removing pi-hole via ssh (which for some reason crashed while doing iproutes, resulting in my odroid network card not doing anything anymore...) anyway... every disadvantage has an advantage :) I will redo my setup with the symlink and see how that goes

Joulinar commented 1 year ago

ok, just totally 'borked' my setup by removing pi-hole via ssh

I guess you confirmed the question to uninstall not needed apt packages. Unfortunately, a downside of the official PiHole script. We tried many times asking PiHole guys to change the behaviour. Basically, they confirmed the challenge but don't see an urgent need to change 😢

MichaIng commented 1 year ago

😄 so true, and so long it is somewhere on my ToDo (contributing changes to the Pi-hole uninstall script).

Ruud68 commented 1 year ago

so just found nand-sata-install.sh in an 'old' forked repo from armbian/build (as i cannot find it there anymore)

So armbian uses the same 'principle', but uses the bind method (like I did). Basically the nand-sata-install.sh script is a wrapper around the instruction I setup, with the possibility of not only sdcard but also emmc and different target etc. and a lot of eye candy (progressbars, etc.)

I have just reinstalled my setup with the symlink and that also works, also did some other small changes: https://dietpi.com/forum/t/tutorial-odroid-c4-rootfs-on-usb-drive/15662/2

MichaIng commented 1 year ago

The tool is now called armbian-install 😉. Thanks for testing with symlink. So we have some longer term test with a simple solution to in case add to an own tool.

yandritos commented 1 year ago

@MichaIng Is this meaning that this feature is going to be added to a new release? At least as a script should be great.

MichaIng commented 1 year ago

No guarantee. There are other topics on the list. Actually the milestone is a bid misleading. I was thinking to generate some images with dedicated boot partition, but I think this would go into wrong direction, especially there is already a tool and now a guide which covers this properly.

yandritos commented 1 year ago

Well, whatever happens, I can live for sure with the old v7. Anyway if I have some time in the future to rework all again, I´ll follow for sure the guide or try any script if available.

Thanks you both for paying attention to this!

MichaIng commented 1 year ago

I can live for sure with the old v7

The DietPi version is not responsible, but the partitioning of the new images. This never worked with a single partition image the way it was implemented in dietpi-drive_manager but broke the system partly.

But @Ruud68 shared a great tutorial above to do it manually, which isn't hard to follow.

Ruud68 commented 1 year ago

and just to follow up: currently my setup isn't on production yet. Need to fix some 'want-to-haves'. This gives me the opportunity to get to better know how all of this works together. Had to reinstall several times, but never because of the moving of the rootfs to the USB drive. So I feel confident to say that the way described in the instructions works and also works together with the DietPi components (like Drive manager). As for prioritizing to implement this as part of the DietPi / Armbian toolbox, I'm not sure: thing is, this is probably a one time function (and not something you do multiple times) so developing a function to automate this is IMO opinion only profitable if the demand from the community for this function is high... I don't think it is, and even then I think that 90% of the target users of DietPi are able to follow the instruction.

I have been thinking of developing it myself, but I am facing a high learning curve as to how to do that. I am a developer, but if I want to make this happen I need to first get the knowledge how to do so. haven't found any basic developer documentation into this ecosystem so that would mean reverse engineer things and trial and error. And for this the same prioritization 'rules' apply when it comes to my time :)