search

raspberrypi / noobs

NOOBS (New Out Of Box Software) - An easy Operating System install manager for the Raspberry Pi
http://www.raspberrypi.org/downloads
2.21k stars 434 forks source link

Support for ExFAT filesystem? #597

Open michaelbaudino opened 3 years ago

michaelbaudino commented 3 years ago

Hello,

Recalbox core team member here :wave:

We are switching one of our partition filesystem to ExFAT in our next release (due in a few weeks), but from what I can see in this piece of code there's no way to ask NOOBS to create an ExFAT filesystem from our partitions.json file :sob:

Are there any plans to support this? Or at least any custom filesystem creation command? If not, which approach do you think we should consider?

Thanks a lot in advance for your help.

PS: I've tried a few things already to reformat and create the filesystem on first boot, but all my attempts failed:

michaelbaudino commented 3 years ago

Just to be clear, I'd be happy to provide a PR to support this (let me know if you want it!), but it doesn't sound like a good solution to me because it's not backward compatible: people who own (or receive from a manufacturer) a NOOBS SD-card created with an older version would not be able to install our OS.

maxnet commented 3 years ago

Or at least any custom filesystem creation command?

You can run your own arbitrary commands as post-installation (partition_setup) script. Windows does that as well.

Regarding partitions.json: Set "empty_fs" to true to tell NOOBS you don't want the partition formatted. "partition_type" to whatever MBR partition id exFAT uses (2 hex digits).

sfdisk doesn't seem to support GPT

Neither does NOOBS.

michaelbaudino commented 3 years ago

Awesome idea, I'll try that :raised_hands: Thanks a lot for the hint @maxnet :pray:

lurch commented 3 years ago

I hope you don't mind me asking, purely out of curiosity...

We are switching one of our partition filesystem to ExFAT in our next release

Why? :)

I've just googled and found https://en.wikipedia.org/wiki/ExFAT#Adoption and to answer your original query NOOBS doesn't (yet) use a 5.4 kernel :wink: (Although apparently it does support FUSE, which I didn't realise!)

lurch commented 3 years ago

it doesn't sound like a good solution to me because it's not backward compatible

We've had similar backwards-compatibility problems in the past, and that's why current versions of NOOBS download os_list_v3.json with older versions still being supported.

michaelbaudino commented 3 years ago

@lurch:

I hope you don't mind me asking, purely out of curiosity...

We are switching one of our partition filesystem to ExFAT in our next release

Why? :)

I've just googled and found https://en.wikipedia.org/wiki/ExFAT#Adoption and to answer your original query NOOBS doesn't (yet) use a 5.4 kernel :wink: (Although apparently it does support FUSE, which I didn't realise!)

Ahah sure, feel free to ask :hugs:

This partition is the one where users can store their media content (roms (games), music, movies, …). Some users have a lot of content.

Since we are targeting non-technical users, we need this partition to be readable/writable on Windows, thus we excluded EXT4. We also excluded FAT32 due to technical limitations (partition size, file size, access rights, no journaling, …). We've been supporting NTFS for some time too, but have a few issues with it (mounting is slow, partitions are randomly not detected, …). It's probably an issue on our side, since NTFS seems quite mature, but we never had time to properly track that kind of not reproductible bugs. Finally, we recently tried ExFAT, which is well supported on both Windows (natively) and Linux, and is quite solid/stable, from what we can see.

Anyway, we may move back to NTFS at some point (since NTFS optimizes space better than ExFAT, iirc), but for now, we're quite happy with ExFAT broad support and performance.

Regarding the driver, we are using this FUSE-based driver which is actually provided by Buildroot. We may switch to native Linux 5.4 support, but we didn't bump to 5.4 on all architectures yet (only on Pi4 at the moment, actually).

michaelbaudino commented 3 years ago

@maxnet:

Regarding partitions.json: Set "empty_fs" to true to tell NOOBS you don't want the partition formatted. "partition_type" to whatever MBR partition id exFAT uses (2 hex digits).

It worked :raised_hands:

Well, actually, I also had to add "filesystem_type": "unformatted" too, otherwise NOOBS tries to run mkfs, which is not what we want.

Actually, I was expecting "filesystem_type": "unformatted" to be enough (i.e. that it would imply "empty_fs": true), but no :man_shrugging: With only "filesystem_type": "unformatted", NOOBS does not create the filesystem (i.e. it does not run mkfs, as we would expect), but it still tries to download a tarball and populate the (not-existent) filesystem.

Let me try to make this more  clear  nerdy.

Current behaviour pseudo-code:

Shouldn't it rather be:

Anyway, thanks a lot for your help :pray: