FileRedirectionShim and redirection base

TimMangan commented 6 years ago

I believe that the FileRedirectionShim should allow the json configuration tupple to be a triple that allows one to define a base for the relocation.

Redirecting everything to AppData/Local is not going to cut it. Some of the file data, especially user settings files, should be redirected to a place that follows the user (such as AppData/Roaming, home drive, or even OneDrive). One should be able to specify this in the json configuration.

We are going to use this to remediate the behaviors of apps that were doing the wrong thing. Please allow the flexibility for us to do so. IT Pros will not have the ability to modify the original application source. Without this flexibility in the FileRedirectionShim, we will be forced to continue to rely upon the AppCompatToolkit and skip the PSF.

TimMangan commented 6 years ago

Oh, and the same might apply to the registry redirection shim.

riverar commented 6 years ago

So, sounds like @TimMangan envisions a sort of 'redirect base' so file access could be redirected to a roaming location.

Non-working example:

"fixups": [
        {
          "dll": "FileRedirectionFixup32.dll",
          "config": {
            "redirectedPaths": {
              "packageRelative": [
                {
                  "base": "",
                  "patterns": [
                    ".*\\.settings"
                  ],
                  "theoreticalRedirectBase": "FOLDERID_RoamingAppData"
                }
              ]
            }
          }
        }
      ]

(Local App Data is currently hardcoded in the PathRedirectionFixup)

TimMangan commented 6 years ago

The idea is on hold while we work out the process of outsiders contributing. I had the tracemonitor ready to go but by the time we got me unblocked, the source on GitHub was replaced with a lot of renaming to classes and files so I'm manually resolving those.

Ultimately, I believe copy-on-write and redirection should be built into the MSIX Runtime so that it is automatic for most applications that need it (like it was for App-V) and the file redirection fixup should be reserved for special situations. But, alas, I have no magic wand.

vladimp commented 6 years ago

Thanks folks! it is a great suggestion. Do you know about specific apps that would require that? On the MSIX runtime - we chose a slightly different approach, instead of creating a complex sub systems that tied to the OS, the idea with the PSF is to create simple and app targeted fixes.

TimMangan commented 6 years ago

The issue is not app specific, but customer environment specific. Environments where the OS is non-persistent (pooled VDI) or semi-persistent (RDS/Citrix Farms) require user settings and/or data created/altered by the end-user to follow the user from machine to machine.

While some of these customers may deploy additional user environment management software, such as Ivanti (formerly Appsense or RES products), or even Microsoft's own UEV, the OS already has built in Roaming Profiles, or more likely Folder Redirection, that may be used to retain this data when the data ends up in the roaming profile are of the user without the need of these additional products.

When we can craft a solution within the app package that puts data that must be retained into the appdata/roaming area then all of the techniques customers implement will easily work without necessarily having to add app-specific rules to that layer.

An even more ideal solution is to not have this work be done at the app packaging layer but as part of the platform. At a minimum, the redirection fixup should be to just make roaming folder the default for all the redirection, or the default with possible override elsewhere.

This is, in essence, what the original SoftGrid through App-V 4.6 runtime did at the platform layer, which made app specific redirections at the app layer unnecessary. The cost of this was that both data that needed to be retained and that which didn't would also be captured, but we were generally willing to live with that.
In App-V 5, this changed with platform layer redirection choosing to redirect app data that would have natively roamed into appdata/roaming and that which would not have natively roamed into appdata/local. This made the packaging more work as we now needed to identify missing data and creatively handle it with workarounds.

In the end, I now believe the platform, not the app-specific shim, should strive to make the packaging effort easier, not harder, than existing packaging (i.e. App-V). I'll soon have a proposal written up for Andrew to clarify my thoughts on this.

vladimp commented 6 years ago

Thanks for sharing the details and I am looking forward to see your proposal as well. In respect to the AppData folder redirection, we have a similar logic to App-V 5 today, in the platform, we redirect AppData local to local and roaming to roaming, we can consider changing it. I am only concerned about the redundant files that we will roam, outlook stores the huge .OST file under local not roaming. Do you by chance know what was the rational of the change in App-V 4.6 --> 5? And I totally agree with you that we should strive to make packaging easier.

riverar commented 6 years ago

In my shoes, I'm less interested in App-V and more interested in lighting up roaming capabilities in apps that write configuration data to Local AppData. Specifying the target of the redirection enables this scenario.

TimMangan commented 6 years ago

The polite public answer is that Microsoft was unable to provide a reasonable rational for the 4.6 to 5 change on redirection location. Most App-V customers opted to implement additional UEM products to work around it.

But yeah, we want to make the packaging easier, and enable typical redirections to useful places is part of that. Microsoft (privately) has my suggestions in this area. Whether or not they decide to act on them at the MSIX runtime level would affect how any changes to the fileredrection fixup would be implemented.

As to outlook OST, Because of the vast size of the typical OST file, I think that if you want that to roam you should instead look to something like FSLogix solution which is considerably more performant than anything you'd do with PSF for those OST files. It would also cover the case of a user running outlook from two different devices simultaneously, which is becoming more common these days.

TimMangan commented 5 years ago

So now that the other PRs are in, I am ready to look more deeply at this. In addition to the originally written basis for a need for control of the location of redirection, there is also the need to control whether cow should be done for an item. So while I might generally want COW behavior, there may be specific files I don't want that on, and I don't want to specify only the ones subject to COW.

Here is what I am thinking of for the config.json:

"fixups": [ { "dll": "FileRedirectionFixup32.dll", "config": { "redirectedPaths": { "packageRelative": [ { "base": "", "patterns": [ ".\.settings" ], "readOnly" : "true" }, { "base": "", "patterns": [ "." ], "redirectBase": "FOLDERID_RoamingAppData" } ] } } } ]

I'm not sure about what the reference to the destination should look like. I'm just copying from Rafael here, but it probably needs to be consistent with any other folder references in use in the PSF..

The defaults for these values when not specified would be "readOnly": "false", "redirectBase": "FOLDERID_LocalAppData" which matches the current behavior.

The concepts around readOnly might not be as obviously needed for file based settings as it will be for registry based settings. But establishing this here will be consistent with needs for RegistryFixup (when we can have one of those). A file based example might be used for something like VLC. The app stores settings in the file system under appdata, so the file redirection fixup is needed for the app to see the settings file within the package (at least on 1809). But if the packager sets the setting in that file to prevent autoupdates, we don't want the user to be able to turn it back on (as really bad things WILL happen)! Thus the need for readOnly. Clearly a better solution would be to target just that one setting in the file and allow the user to change others, but this isn't possible in a general external solution (the vendor really needs to rewrite their code to allow for this).

vladimp commented 5 years ago

On the "readOnly" - I am not clear about the VLC scenario, Autoupdates will be controlled by the IT Pro via MSIX stack and not by the app in most cases, is it really needed? Do you know about any other scenarios?

TimMangan commented 5 years ago

First:

In my working copy I have renamed this item to "exclusion" for now. The IT Pro does not have the time or ability to determine every individual file in a package that might need to be changed in some part of the application. And in some cases, even a full test of the app today won't reveal the presence of something that comes into play over time (like updates being available). The vendor doesn't document what files might change are and the IT Pro sometimes lacks credentials to test the full application anyway.

The exclusion marking would allow for the fileredirection config to cotain two entries in the packageRelative array. The first entry is marked as an exclusion. The patterns for this exclusion would clearly include normal WinPE filetypes, but possibly other patterns as well. The second entry would not be marked as an exclusion and would cast a wider net, possibly just the .* pattern to request that the fixup apply to all files not excluded.

But here is the issue app to your question about the need for readonly. VLC. It uses a configuration file that is stored in the user profile of the package and has a setting that controls updates. An IT Pro can turn off the autoupdate feature in this file for the package they deliver, but without a MSIX FileRedirectionFixup added to the package, the application CANNOT read that config file.

For some unknown reason the normal VFS layering does not apply to AppData Local and Roaming -- meaning if the containerized app requests C:\Program Files\Venfor\Foo.cfg the MSIX runtime will check the VFS path in the package and find Foo.cfg. But if the same app asks for C:\User\username\AppData\Local\Vendor\Bar.cfg the MSIX runtime never looks at the VFS path for the file and the file is unfound and the app reverts to default settings (including checking for updates each time the app is run). So to keep VLC from prompting the user for updates after 30 days, we must have the FileRedirectionFixup target this file, causing it to redirect requests to a copy it makes in the LocalCache of the package (which ironically is in AppData Local). But this makes the file read/write so the end-user can just turn it back on. Now they get the popup as soon as a new version is available, and if the end-user has admin rights they'll end up with the newer version natively installed with the vendors MSI straight from the vendor website in addition to the MSIX package deployed by IT. And that is why we need a readonly setting (it might just look different than above).

UPDATE: As predicted, I saw a Microsoft employee on stage demoing the VLC MSIX package and right after launching the app, a vendor popup appeared looking for permission to "install" updates. If the demo jock had clicked yes, it would have proceeded to download and install a native VLC MSI outside of the container.

TimMangan commented 5 years ago

PR #37 is now submitted to cover this item.

microsoft / MSIX-PackageSupportFramework

FileRedirectionShim and redirection base #5