search

ps2homebrew / Open-PS2-Loader

Game and app loader for Sony PlayStation 2
https://ps2homebrew.github.io/Open-PS2-Loader/
Academic Free License v3.0
2.03k stars 262 forks source link

[FR] ext2 support from E2OPL or exFAT #238

Closed eadmaster closed 11 months ago

eadmaster commented 4 years ago

Backport the changes made in this fork.

Key advantages over FAT32:

Probably the best choice over NTFS and exFAT.

sp193 commented 4 years ago

The latter is not an advantage of the EXT2 filesystem itself, but the author added logic to handle the filesystem blocks. I was skeptical about whether he really managed to achieve that with no loss of game compatibility, since I once tried to shrink the FAT32 part of the USBHDFSD driver to being as small as possible and some games stopped working as a consequence. When I did that, I found that even adding a 512-byte buffer for storing a sector and the logic for handling the filesystem, would cause some games to stop working due to a lack of memory...

A minority of users use an OS that supports EXT2 natively.

rickgaiser commented 4 years ago

When I did that, I found that even adding a 512-byte buffer for storing a sector and the logic for handling the filesystem, would cause some games to stop working due to a lack of memory.

Are you sure it was becouse of a lack of IOP memory? I remember we had a conversation about this once. SLPM-66438 "Melty Blood Act Cadenza" is the game that wasn't working, but that same game was working with ETH just fine. I can test this game again with a large (64KiB?) buffer added to iop-core?

The HDD (PFS) driver is passed an array of 65 partitions/fragments: https://github.com/ifcaro/Open-PS2-Loader/blob/ad182dcf95e3322206f5c2f5ced99666ac1621f8/modules/iopcore/cdvdman/atad.h#L84 https://github.com/ifcaro/Open-PS2-Loader/blob/3b37ef7f250ca85e03cd86eacb9e060d8f6a3413/modules/iopcore/cdvdman/device-hdd.c#L42

I've tried to do the same before for the usb mass (BDM) driver. My intention was to use it with PFS, FAT32 and EXT2 (also inspired by e2opl), but in theory any filesystem can be used. The GUI simply passes the fragments, and when more than 65 fragments are detected an error must be shown to the user, requesting it do defragment the drive. The in-game driver is only aware of fragments and doesn't need to know anything about the filesystem itself.

Velocet commented 4 years ago

exFAT would make more sense than anything else:

The mentioned implementation seems pretty nice as it doesn't consume much resources...

hardlevel commented 3 years ago

Hi, sorry for opening this issue again...

But just a crazy idea... Is possible to use the same fs used in internal hdd, the pfs, but on usb?

If this was possible, it could solve many problems

rickgaiser commented 3 years ago

Not so crazy at all. BDM was created to separate the block devices from the file systems. If this is also applied to PFS and HDD, then it would be possible to use PFS on USB. Or FAT32 on HDD. Or any other combination.

hardlevel commented 3 years ago

I know that pfs can fragment too, but much less than fat32... this may be a good solution, if OPL could support two formats for usb, fat32 and pfs would be amazing, considering that we already have tools to install games in this format...

AKuHAK commented 3 years ago

There is no benefits in using any other filesystem cause of possible fragmentation. Specially pfs, which is too specific and doesn't have many pc side utilities. It is possible to use device without filesystem at all, just directly write ISO files and then calculate LBA offsets. @sp193 oneday worked on this, but stopped, I dont remember why (maybe no benefits in comparing with non-fragmented fat32).

sp193 commented 3 years ago

We don't use PFS for games. It's actually a proprietary filesystem that once belonged exclusively to HDLoader. If a different filesystem is used, then there's also a question if anyone is actually interested in using yet a different fileysystem, only for the PS2.

I did try to add a micro FAT32 driver to OPL before. But stopped since I couldn't get it to work desirably, which I believed was due to a lack of memory. But as you know, @rickgaiser has a different view and I don't know why our results just differ. Neither do I remember the exact observations from what I did back then, which was 2014. It's not because FAT32 results in more or less fragmentation, but we just did not want to support a full filesystem driver - ostensibly due to memory and speed requirements.

@AKuHAK I remember once having an idea like that, but I don't remember telling many people that I wanted to try it. The concept sounds like the HDLoader filesystem. But if I did, then idea could have been thrown out, as not many people seemed to be interested in yet another fileystem; that would require totally different tools, not compatible with other game loaders (or just older versions) and the space cannot be used for other things (outside of the PS2). However, I conducted no study, and perhaps just got the idea that it'll be unpopular, from the few people I interacted with.

ahmedmoselhi commented 3 years ago

@sp193 one other thing to take care of that may make difference using other file systems.. i have got 128gb didcated for ps2 games but when i try to load any game i stuck at white screen after loading . the games are placed on usb using usbutil app and no fragmentation found.. when i try to place the whole iso file in opl dvd folder game load successfully! but i have limitation of 4gb file size of iso so using other file system can solve this issue

DraconicNEO commented 3 years ago

A minority of users use an OS that supports EXT2 natively.

It's still good to have versitile file system support. Also there's an Ext file system support plugin for Windows.

halotroop2288 commented 3 years ago

I agree with @DraconicNEO. Having versatile file system support is important. Personally, I would love for OPL to support any commonly used format that can be used on PC. It would make transfers much easier! I'm not well versed in the technology that's required. But ext2 seems like a bad first choice. NTFS, or something FAT-based would be my first choice, with ext4 in close second. But if there's a technical issue preventing ext3/4 from being used then I guess ext2 is a fine choice as well.

DraconicNEO commented 3 years ago

I agree with @DraconicNEO. Having versatile file system support is important. Personally, I would love for OPL to support any commonly used format that can be used on PC. It would make transfers much easier! I'm not well versed in the technology that's required. But ext2 seems like a bad first choice. NTFS, or something FAT-based would be my first choice, with ext4 in close second. But if there's a technical issue preventing ext3/4 from being used then I guess ext2 is a fine choice as well.

I suggested ext2/3/4 because EXT filesystems are open source and don't have legal restrictions, unlike NTFS or Newer Fat based filesystems. Hence why the Dev is very rigid about adding NTFS and exFAT support.

dotexe1337 commented 2 years ago

Would love to see this. I am still using the old Ext2 fork due to the benefits Ext2 offers over FAT32 for the PS2.

DraconicNEO commented 2 years ago

Would love to see this. I am still using the old Ext2 fork due to the benefits Ext2 offers over FAT32 for the PS2.

I agree, Hopefully they'll implement this soon.

Wolf3s commented 2 years ago

@dotexe1337 check this here https://github.com/Doom-modding-and-etc/libext2-for-ps2

TnA-Plastic commented 2 years ago

ExFAT is on its way.

Wolf3s commented 2 years ago

ExFAT is on its way.

👍

DraconicNEO commented 2 years ago

ExFAT is on its way.

Awesome, Any idea if EXT support will also be implemented?

Badxzz commented 2 years ago

finally let's drop fat32

DraconicNEO commented 2 years ago

finally let's drop fat32

I don't think we should completely drop it, especially for the many people who have existing setups based around FAT32.

It's good to have support for multiple different file systems they all have their benefits and drawbacks. Why only support one file system, after all other loaders such as USB Loader GX on Wii support many different File systems.

J013k commented 2 years ago

Let me put this differently... If something will be wrong with a game and you want to test older reviews\commits... Some games can be compatible only with OPL 0.9.3 or 1.0.0. Unfortunately only FAT32 file system is supported by them. Dropping its support IMHO is not a good idea.

Haker120 commented 2 years ago

Even in ideal situation, for example, recently updated OPL has various file system support and 100% compatibility, even MacOS etc, dropping any FS support is kinda dump, as do not we want ULTIMATE OPL PS2 LAUNCHER? And even better for end user because they do not to even know basics, they put game on USB with FS they have and it just works. Not to mention again (as someone stated) people already have pre-set-up configurations around FAT32.

And not to mention exFAT is rather innovation for ps2 so lots of older, still fully functional apps, do not support it yet and big part of them won't get it.

BatRastard commented 2 years ago

ExFat is the future. Windows Media Creation Tool will default to it on USB flash drives larger than 32GB. Ventoy is using it. Rufus is using it. The author of YUMI Multiboot is working on an ExFAT version as well. We'd be fools not to but we gotta be conservative about it. For example, if we implement ExFat in OPL 1.3, we should abandon FAT32 by OPL 1.8 or 2.0 (or sooner if everything behaves). Rest assured: FAT32 is on borrowed time. It'll take another 10 or 12 years for full phase out. Microsoft even announced that they'll be preventing Windows from being installed and booted from mechanical hard drives. I predict motherboard manufactures will eventually abandon soldered flash chips and CR2032 batteries for UEFI BIOS. Stick the UEFI BIOS ROM on a nano flash drive and plug it into a USB-C header on the motherboard.

rickgaiser commented 2 years ago

The new driver based on FatFs supports FAT12, FAT16, FAT32 and FAT64(exFat). No file systems will be removed, unless there's a good reason for it.

DraconicNEO commented 2 years ago

For example, if we implement ExFat in OPL 1.3, we should abandon FAT32 by OPL 1.8 or 2.0 (or sooner if everything behaves).

exFAT has its benefits and its drawbacks, you shouldn't be advocating the removal of a feature which would essentially ruin everyone's existing setups.

I predict motherboard manufactures will eventually abandon soldered flash chips and CR2032 batteries for UEFI BIOS. Stick the UEFI BIOS ROM on a nano flash drive and plug it into a USB-C header on the motherboard.

How is this in any way relevant to OPL on PS2?

albertowd commented 2 years ago

The new driver based on FatFs supports FAT12, FAT16, FAT32 and FAT64(exFat). No file systems will be removed, unless there's a good reason for it.

Already working here, thanks! The issue can be closed already! <o/

DraconicNEO commented 2 years ago

The new driver based on FatFs supports FAT12, FAT16, FAT32 and FAT64(exFat). No file systems will be removed, unless there's a good reason for it.

Already working here, thanks! The issue can be closed already! <o/

What about EXT/2/3/4 support?

eadmaster commented 2 years ago

My initial request here was to get a fs driver with fragmented and big isos support, does this FatFs provides both?

(I don't mind an uncommon fs like ext2 if i avoid the hassle of iso splitting and defragmenting)

DraconicNEO commented 2 years ago

My initial request here was to get a fs driver with fragmented and big isos support, does this FatFs provides both?

(I don't mind an uncommon fs like ext2 if i avoid the hassle of iso splitting and defragmenting)

I'm not sure, I don't think fatfs provides ext support. I personally think ext support would be good since it's natively supported by Linux and MacOS and is more stable than exFAT which has corruption issues.

Not saying exFAT shouldn't be an option but I would definitely prefer to use something like ext4, or even ext2.

Wolf3s commented 2 years ago

It doesn't provides the extfs support

sp193 commented 2 years ago

The reason why you have a need to defragment, comes from there not really being a filesystem driver within OPL. It has nearly nothing to do with the filesystem used, but the original intent was to save memory and to improve performance. Do note that the PS2 only has USB 1.1, so every extra sector you transfer to read data will be quite a bit of overhead.

eadmaster commented 2 years ago

Since it's exfat should support isos bigger than 4GB already, so the 1st point is covered.

As for the fragments part, my guess is that if an iso is lightly fragmented (e.g. splitted in 3 frags on the storage), the overhead would be minimal.

DraconicNEO commented 2 years ago

Since it's exfat should support isos bigger than 4GB already, so the 1st point is covered.

As for the fragments part, my guess is that if an iso is lightly fragmented (e.g. splitted in 3 frags on the storage), the overhead would be minimal.

There is one reason why someone would want to use ext over exFAT though, and that is because exFAT is much more prone to corruption.

AmyRoxwell commented 11 months ago

So, after a long year since this was open I was wondering, is there still plans to implement ExFat to the project or at least provide something like ext4 so Linux and Mac users can actually use opl? Grimdoomer (https://github.com/grimdoomer/Open-PS2-Loader/tree/hdd_large_capacity_support) made a fork that works pretty great and even allows you to just drag and drop the files directly into the HDD or USB without having to rely on an external program.

We shouldn't remove FAT32 compatibility, but currently, having to rely on that format is making things difficult on Linux, due to distros like EndeavourOS not supporting APA drives and also due to the lack of functional programs to load the games on said drives for Linux.

rickgaiser commented 11 months ago

Closing this issue becouse exFat support has been added to ps2sdk and opl by @JoseAaronLopezGarcia a year ago already. See #660.

This works on all BDM devices (usb/ilink/mx4sio).

Only missing support on hdd, but another feature request has been opened for this.

AKuHAK commented 11 months ago

Closing this issue becouse exFat support has been added to ps2sdk and opl by @JoseAaronLopezGarcia a year ago already. See #660.

this is related to EXT2 fs, not EXFAT :)

rickgaiser commented 11 months ago

Title says: ext2 OR exfat, so if you ask me that request is complete.

DraconicNEO commented 11 months ago

Title says: ext2 OR exfat, so if you ask me that request is complete.

So does that close the door for the possibility of ext2/3/4 in the future? I mean if someone made another issue requesting it I imagine it would be closed as a duplicate since this one already exists.

JoseAaronLopezGarcia commented 11 months ago

Title says: ext2 OR exfat, so if you ask me that request is complete.

So does that close the door for the possibility of ext2/3/4 in the future? I mean if someone made another issue requesting it I imagine it would be closed as a duplicate since this one already exists.

There's really no reason to support ext when we already have exFat. ext is only used by Linux, neither Mac nor Windows users would be able to use it easily (need to install drivers and apps), while exFAT is supported everywhere natively.