[Problem] MPF.Check with "-p /mounted/disc/folder" non-functionality in Linux, for PS2 DVDs

[JohnVeness]

Version What version are you using?

• [x] Stable release (version 3.2.0)

Build What runtime version are you using?

• [x] .NET 8.0 running on (Linux)

Describe the issue This is a follow-up from https://github.com/SabreTools/MPF/issues/706#issuecomment-2212521248

I am attempting to use MPF.Check to generate a !submissionInfo.txt file for a PS2 DVD disc previously dumped with Redumper. As noted in that issue, MPF.Check doesn't trust the disc region from the Redumper log, so needs to check the mounted physical disc, but the behaviour of this appears to be different between MPF.Check in Linux vs Windows.

To Reproduce Steps to reproduce the behavior:

In Linux:

1. Dump a PS2 DVD, e.g. SLES-51252, in Redumper
2. cd into the folder output from Redumper, e.g. /home/john/redump/SLES-51252
3. Mount the disc as e.g. /run/media/john/333d3c3734333d32
4. Run MPF.Check dvd ps2 -u redumper -p /run/media/john/333d3c3734333d32 SLES-51252.iso
5. Look at the created !submissionInfo.txt file

Expected behavior The !submissionInfo.txt file to contain "Region: Europe" (in this case), as it does correctly do in Windows with MPF.Check dvd ps2 -u redumper -p d: SLES-51252.iso.

Observed behavior The !submissionInfo.txt file contains "Region: SPACE! (CHANGE THIS)". It does, however, contain correct PS2 ISN, version and EXE date.

Additional context As a further test, I deleted the following section from the Redumper.log file:

PS2 [SLES-51252.iso]:
  EXE: SLES_512.52
  EXE date: 2002-09-20
  version: 1.00
  serial: SLES-51252
  region: Europe

and ran the MPF.Check test again. My theory being that this should make MPF.Check do its own check for the EXE name, date, version, serial and region (as opposed to doing its own check just for the region previously), rather than taking that info from the Redumper log. Running MPF.Check dvd ps2 -u redumper -p /run/media/john/333d3c3734333d32 SLES-51252.iso now shows that the version, ISN, exedate etc. are no longer written to the !submissionInfo.txt file at all.

In other words, as far as I can tell, MPF.Check's code to look for PS2 exe details from the mounted disc doesn't seem to work at all in Linux :(. Unless I'm doing something wrong, which is quite possible!

I should add that listing files in /run/media/john/333d3c3734333d32 does indeed show the system.cnf file, the SLES_512.52 file, etc., and the files are readable.

[JohnVeness]

Ah, I have run this through a debugger and worked out what is going on. Line https://github.com/SabreTools/MPF/blob/2983266e8aaf0252f87728f3ecf254180265f16e/MPF.Frontend/Drive.cs#L256 looks for a file called SYSTEM.CNF, but in Linux the file shows up as system.cnf. So the var systemCnf = new IniFile(systemCnfPath) bit doesn't find the file and doesn't fill in the fields.

I don't know if there's a way to make this case-insensitive...

[JohnVeness]

It looks like PS2 DVDs contain both an ISO9660 filesystem and a UDF filesystem. They contain the same files but the case can vary, depending on the game. Windows seems to mount the ISO9660 filesystem and Linux the UDF by default. Annoyingly, even if I force Linux to mount the ISO9660 fs, this still seems to show filenames in lowercase, even though Windows shows in uppercase. As far as I know, ISO9660 isn't case-sensitive, but different operating systems choose different conventions for how to show the filenames. There is probably a way to force Linux to mount the ISO9660 fs using uppercase filenames, but I haven't investigated. I also don't know what macOS does.

While researching I came across this academic paper https://dfrws.org/sites/default/files/session-files/2010_USA_pres-different_interpretations_of_iso9660_file_systems.pdf which explained the difficulties on interpreting just the ISO9660 format, let alone UDF too! There doesn't necessarily seem to be a single correct method.

Of more importance than how Windows or Linux or macOS interpret discs, is how the actual PS2 does, as it's that that we're trying to match. Presumably there's something at the BIOS level that looks for the system.cnf file, and I don't know if that uses ISO9660 or UDF to do so, or what.

I wonder if it would be better to not rely on how the operating system decides to display the files, and instead to write code to interpret the physical disc (or the already-dumped .iso file) to match the PS2. That way you can ensure a consistency between host OSes. Obviously basically reinventing the wheel to interpret a filesystem is a lot of work, but maybe there are already libraries out there that can do some of the work for you. I found https://www.nuget.org/packages/LTRData.DiscUtils.Iso9660 but haven't researched much. Obviously using a third-party library could introduce yet more variables, but maybe with the right set of parameters you can ensure consistency.

A lot of work just for reading a couple of files, I appreciate :)

[JohnVeness]

More interesting reading - https://www.obscuregamers.com/threads/how-to-create-a-working-playstation-2-master-cd-r-or-dvd-r-image.772/post-17636.html

[Deterous]

Easiest solution would probably be to try opening as "system.cnf" if "SYSTEM.CNF" fails.

[JohnVeness]

Yeah, except Sod's Law there'll be one game that uses System.cnf or SYSTEM.cnf or something!

I was wondering, at least in the case of Redumper and PS2 discs, if it was worth reinvestigating whether to trust Redumper's output regarding ISN/EXE/date/region. Redumper clearly has low-level disc-reading code (it certainly doesn't mount the disc so isn't relying on Windows' or Linux's interpretation of filenames), so is probably accurate at finding SYSTEM.CNF and the EXE.

And, as far as I can tell from a brief look at its code, it finds the region in the same sort of way (by looking at the ISN) that MPF.Check does. If it isn't 100% accurate, presumably they would accept PRs to make it so.

Then you could eliminate the entire code in MPF.Check (I realise it's really in MPF.Frontend) for doing your own thing finding system.cnf and the exe, and users wouldn't need to have the physical disc and mount it (in the case where they do the Redumpering and the MPF.Checking at different times).

[Deterous]

Yes, in fact you'll find I contributed to redumper the code that does PS3-5 info extraction, as well as improving the PS2 info extraction. Redumper parses the ISO9660 filesystem from the .iso file (it doesn't access the drive when doing redumper info). superg isn't keen on doing fine-grained region detection like MPF does.

MPF should be getting the data from the Redumper .log if it is present, see this function: https://github.com/SabreTools/MPF/blob/2983266e8aaf0252f87728f3ecf254180265f16e/MPF.Processors/Redumper.cs#L1103

[JohnVeness]

That's great, I'm glad there's collaboration.

So, if it's just the fine-grained region detection that you don't trust from Redumper, maybe you could just trust Redumper's ISN output (which in fact you already do, to copy into the !submission.txt) and use that to generate the region, rather than MPF.Check looking for the system.cnf file itself, and therefore requiring the physical disc present and mounted etc.?

[Deterous]

Should be able to have the best of both worlds by taking Redumper's ISN and putting it into MPF's region logic.

[JohnVeness]

Yep, exactly, that would be perfect. Then I could just run e.g. MPF.Check dvd ps2 -u redumper SLES-51252.iso without having to have the physical disc present at MPF.Checking time, and still have everything filled-in in !submissioninfo.txt.