OG Xbox FATX fixes

[rapperskull]

This commit avoids the collision between OG Xbox's Cache X and 360's Cache 0 by checking the type beforehand. Note that the magic is guaranteed to exist on the OG Xbox HDD. I don't know if it's the same for the "Josh" sector. I also added support for a partition table, as defined by XBP Table Writer. I fixed the FATX directory recursion (it was off by one and tried to access the ChainMap at position 0xfffffff8).

Unfortunately, even if the directory structure is correctly parsed, when I run the program I just get a message saying that the partitions have been loaded, but no new drive has been mounted under Windows.

BTW take this pull request as a draft. I think the code is far from perfect, but at least it's a first step.

[rapperskull]

Sorry for this second large commit. Visual Studio decided to convert all line endings to CRLF.

[emoose]

Mostly looks good so far, thanks for taking a look at it!

Only problem I've spotted is with the // Work out the retail data partition size section being moved inside if(IsXbox360Device()), seems that'd prevent the size being calculated for the OG xbox "User Defined F" & "User Defined G" partitions. Both are sized 0 to make this calculate the size for them (though that code probably won't work well with more than one 0-sized partition atm, maybe just needs to be changed so it can use next-partition-offset to determine the size if there's a partition that comes afterward) Maybe the code following // Check partition validity & remove any invalid ones should be after the IsXbox/IsXbox360 checks.

(Also just realized it'd probably be better to move the // Filter kXboxPartitions to the ones that could fit onto this drive/image code to after devkit/homebrew partitions are added, before the // Check partition validity & remove any invalid ones section, I can change that later though)

Unfortunately, even if the directory structure is correctly parsed, when I run the program I just get a message saying that the partitions have been loaded, but no new drive has been mounted under Windows.

Is that with a physical disk itself? Last I remember there were some issues with the wrong permissions being set, since we need to run xbox-winfsp as admin to access the physdisk it also mounts all the drives it creates with admin-only permissions, so explorer won't show them :/ If you run something like notepad as admin it should let you browse them through the open-file-dialog though, but it's really not that ideal, tried a bunch of things to see if I could fix that but couldn't really get anywhere, AFAIK it's something to do with the SID/SDDL that gets applied to FileSystemHost/accessed with GetSecurityByName, there's some examples about using SDDLs in the winfsp github but last I tried nothing really seemed to help, maybe something has changed since then though.

[rapperskull]

Implemented some of the suggested changes. Please take a look and tell me if you think something is off. I tried the notepad method and in fact found a bug that I fixed. I'll try to research if there's a way to make the partitions show up in Explorer.

[rapperskull]

Ok this includes a lot.

• Created a DeviceStream, so that the logic for FatxDevice can be simplified
• Implemented Virtual File System. It is used to mount an entire drive, with each partition as a folder in the root directory
• Now LoadPartitions does not mount the file systems, but just calls Init, returning them in a list
• Changed "Mount as Xbox STFS/GDF" to "Mount with XBOX-WINFSP" and fixed install/uninstall
• Implemented mount with -u
• Fix unmounting
• Now the program scans for physical drives to mount only if no other mode is selected
• Many small fixes

The main goal was to allow WinFSP to mount an entire disk to a drive letter. Unfortunately, even if -u is now correctly implemented (the format is \xbox-winfsp\PhysicalDriveN), net use still doesn't work. I suspect that's because of permissions, even if we should be running as SYSTEM in that case. It's possible to mount multiple devices at the same time, each one with its own root. It's also possible to load an image of the entire drive or of a single partition. My only concern is with STFS, that I haven't understood very well.

Maybe this could allows us to mount the FS as a normal user, but frankly I haven't understood much of it.

[emoose]

Nice changes! I like the VFS idea, a lot cleaner keeping a single drive under a single drive letter.

With "net use" I'm assuming it works fine with ISOs/packages but only has issues with PhysicalDrive, do you know if it's the same issue as before with the created drives being admin-only, or is it not being given enough permissions to access the PhysicalDrive at all?

Unfortunately it seems our issue with admin permissions hasn't really come up on WinFsp's issue tracker before, at least I couldn't find anything there with the same problem... https://github.com/billziss-gh/winfsp/issues/204 has some info about using FspTool & other utils to view security info for mounted drives, but pretty sure I'd looked into it with them before and couldn't find out much.

It could be the case that whatever needs to be changed for it isn't actually exposed to .NET atm, like the VolumeParams ReadOnlyVolume flag that currently needs a .NET reflection hack to add in.

[rapperskull]

I think the problem is with opening the PhysicalDrive. Unfortunately I haven't found a way to debug the program when called by WinFSP loader, so I can't investigate further.

[emoose]

Ah darn, no idea how we'd be able to fix that, it seems WinFsp's loader is ran as SYSTEM but I guess doesn't pass that onto the FS being mounted, maybe need to look into the loaders src and see if there's a regkey or something that can allow that.

BTW I'm guessing you've been testing with Xbox OG drives? I'll have to dig out my X360's HDD to try with it soon, also have an OG devkit drive I can check with.

Also I think I've fixed the CRLF issue with FatxDevice so we can see the changes in this PR more clearly, think you'll need to git pull on your end before committing anything else though. (maybe copy out any changed files since the last commit here though in case git overwrites them)

E: hmm, seems WinFsp is already running the xbox-winfsp launched by "net use" as SYSTEM, strange that it wouldn't be able to access the drive...

I did notice the VolumePrefix code for handling the path given by "net use" seemed broken for non-PhysicalDrive paths, maybe the PhysicalDrive section also needs fixing.

To fix it for STFS/non-PhysicalDrive I used

                if (null == ImagePath && null != VolumePrefix)
                {
                    I = VolumePrefix.IndexOf('\\');
                    if (-1 != I && VolumePrefix.Length > I && '\\' != VolumePrefix[I + 1])
                    {
                        I = VolumePrefix.IndexOf('\\', I + 1);
                        ImagePath = VolumePrefix.Substring(I);
                        if(ImagePath.StartsWith(@"\PhysicalDrive", StringComparison.InvariantCultureIgnoreCase))
                        {
                            // Format has to be \\.\\PHYSICALDRIVE{0-N}
                            ImagePath = String.Format(@"\\.\{0}", ImagePath);
                            VolumePrefix = null;
                        }
                        else
                        {
                            ImagePath = String.Format("{0}:{1}", VolumePrefix[I + 1], VolumePrefix.Substring(I + 3));
                        }
                    }
                }

Maybe VolumePrefix shouldn't be getting set as null there, not sure.. I'll take a look at it with my drives later.

[rapperskull]

I followed these instructions and managed to mount a physical disk as SYSTEM. It shows up in Explorer! So we only have to understand why WinFSP doesn't work. When I try to mount it, it asks for username and password. As I understood it's the default behaviour in case of errors, including timeouts.

As for non-physicaldrive paths, what is the format?

[rapperskull]

I added support for device letters (\xbox-winfsp\X:) and kept the VolumePrefix. If the prefix is null, the filesystem is treated as a network FS, otherwise as a disk FS. I don't know which is "better" in our case.

[rapperskull]

It works! We were setting the wrong registry key. WinFSP is a 32-bit application, so it loads the registry keys from HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\WinFsp\Services on 64-bit machines. The code should work on both 32 and 64-bit systems.

[emoose]

Hmm, on my machine it was already setting up using the Wow6432Node key when using -s to install it, maybe because I used "Any CPU" to compile or something. Of course if it wasn't doing it that way for you then there was an issue there, I'll give your new code a try later to make sure it's all still working for me.

So now you're able to use "net use" to mount a PhysicalDrive and it's accessible with explorer fine? Great news if so! I hadn't thought of using "net use" for it (as you probably could tell by the incomplete code :P). It's too bad the normal xbox-winfsp.exe has issues though, I wonder what difference the launcher could be making for it to work...

Maybe it's worth adding some option as a "frontend" to that command, so you could eg. run xbox-winfsp.exe as admin, which would then check each physdrive & run "net use" on any detected Xbox drives.

As for non-physicaldrive paths, what is the format?

When I was trying with STFS packages I used net use X: \\xbox-winfsp\C$\src\test to mount a package from C:\src\test, not sure if that will still work with your latest change though.

AFAIK the VolumePrefix given to that bit of code is the whole path passed to the service, \C$\src\test in this case (the name of the var is maybe a little misleading, that was taken from passthrough example IIRC)

E: Seems it's checking for : there instead of $, it also needs to change the $ into : for the ImagePath to be valid, the code I posted earlier was also from passthrough example and seemed to work fine for paths like this, but don't think that supported PhysicalDrives properly.

[rapperskull]

Maybe I'm missing something. Why using net use to mount a STFS package if you can simply right click it, or use the -i switch? Is it simply a convenience thing, or I'm missing some pieces? I got it. It's to avoid keeping the cmd window open, right?

[rapperskull]

I removed support for drive letters (it wasn't working and I can't think of any real world use), and added back file support. The format is the usual \\xbox-winfsp\X$\path\to\file

[rapperskull]

I added one new commit to account for Dancing Stage Unleashed (Prototype) that has one file that ends beyond the end of the ISO. Truncate the file in this edge case to be consistent with other tools.

BTW, is there any chance this PR will eventually get merged?