[tracking] Publish .NET assemblies in Webcil files

lambdageek commented 1 year ago

Summary

Webcil is a new container format for .NET assemblies that looks less like a normal Windows PE (Portable Executable) .dll file. A .NET assembly in the Webcil format is a normal binary WebAssembly module with a particular set of exports that will copy a "Webcil payload" to WebAssembly linear memory. It has ~~a new .webcil extension~~ a standard application/wasm MIME type and .wasm extension and the detailed format is specified in webcil.md.

There are two primary motivations for webcil:

In some restricted environments, users may be prevented from downloading .dll or PE files, and some AV tools may quarantine .dll files found in browser caches.
By using a new container, the runtime may be able to find additional flexibility in how we bundle and serve .NET assemblies. A goal is for the container to be invisible to higher layers of the runtime - unlike .netmodules in .NET Framework, as far as most of the runtime is concerned, we're still loading ordinary Ecma335 assemblies and any partioning is not visible at the metadata level. But at the loader level, the runtime may actually receive .webcil files that encapsulate one, part of one, multiple, or parts of multiple assemblies. (Version 0.0 of webcil only supports one assembly per .webcil container)

.NET 8 Preview 1

The goal is to make Webcil work in the wasm-experimental workload with the wasmbrowser template

The initial runtime work was done in #79416

Update MIME types for .webcil
- [x] in xharness https://github.com/dotnet/xharness/pull/989
- [x] and dotnet SDK webserver configs https://github.com/dotnet/sdk/pull/30077
- [x] aspnetcore UseBlazorFrameworkFiles https://github.com/dotnet/aspnetcore/pull/46244
[x] Implement WebcilReader.ReadPdbChecksumDebugDirectoryData and refactor DebugStore AssemblyInfo.FromWebcilReader and AssemblyInfo.FromPEReader to share PDB logic (maybe using a new IDebugInfoProvider adapter to cover the common parts of PE and Webcil) https://github.com/dotnet/runtime/pull/81099

Later in .NET8

[x] Implement webcil in blazor bundling #84977
- [x] Implement a standalone WebcilGenerator MSBuild task for Blazor lazy loading scenarios #85335
- [x] Add a lazy loading + satellite assemblies WBT test (BugRegression_60479_WithRazorClassLib)
- [x] Update ComputeBlazorPublishAssets and elsewhere in the BlazorWasmSDK
- [x] Update various places where file extension is checked (tasks, BlazorWebAssemblyLazyLoad, AssemblyLoader.LoadAssembliesAsync)
[x] #85932
[x] Make webcil-in-wasm the default packaging for standalone .NET WebAssembly apps (dotnet new wasmbrowser) and Blazor WebAssembly apps (dotnet new blazorwasm) https://github.com/dotnet/runtime/pull/86330
[x] Enable debugger tests with Webcil
[ ] Gather feedback and identify further mitigations for restricted environments
[x] https://github.com/dotnet/runtime/issues/86597

Post .NET 8

[ ] Take advantage of webcil custom loader to optimize wasm bundles. (Define a multi-assembly .webcil container. Define a partial-assembly webcil container. Etc.)

Other

[ ] https://github.com/dotnet/runtime/issues/86522

dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost commented 1 year ago

Tagging subscribers to 'arch-wasm': @lewing See info in area-owners.md if you want to be subscribed.

Issue Details

Follow-up to #79416 - [ ] Enable debugger tests with Webcil - [ ] Implement a standalone WebcilGenerator MSBuild task for Blazor lazy loading scenarios - [ ] Gather feedback and identify further mitigations for restricted environments - [ ] Take advantage of webcil custom loader to optimize wasm bundles. (Define a multi-assembly .webcil container. Define a partial-assembly webcil container. Etc.)

Author:	lambdageek
Assignees:	-
Labels:	`arch-wasm`
Milestone:	-

lambdageek commented 1 year ago

Code review nits:

[x] use strlen(LITERAL) instead of 4 or 7. For example here: https://github.com/dotnet/runtime/pull/79416#discussion_r1081874365
[x] In WasmAppBuilder, use Utils.CopyIfDifferent(string src, string dst, bool useHash) to place the webcil file in the final output path and add to fileWrites; FileCopyChecked doesn't do a checksum or anything like that. https://github.com/dotnet/runtime/pull/79416#discussion_r1081941996
Need to update the various web servers to serve .webcil as octet-stream https://github.com/dotnet/runtime/pull/79416#issuecomment-1397720168
- [x] here https://github.com/dotnet/sdk/blob/ea03dc7f05bfbaac843dc2ac09e26a484acfe8f8/src/BlazorWasmSdk/Targets/BlazorWasm.web.config#L13
- [x] and here https://github.com/dotnet/runtime/blob/12f65cd40c26a0eee613eaccb8c0e53b708b2a8f/src/mono/wasm/host/WebServerStartup.cs#L85

lambdageek commented 1 year ago

[x] Also need to add mime type to xharness https://github.com/dotnet/xharness/blob/main/src/Microsoft.DotNet.XHarness.CLI/Commands/WebServer.cs#L119

charlesroddie commented 1 year ago

As technically impressive as this work may be, it's an extensive workaround to the current blazor inadequacy of deploying dlls to user systems rather than compiling everything to WASM. Full compilation (via a fix to Mono WASM AOT or NativeAOT) will make this obsolete unless it finds some other application than browsers.

ishepherd commented 1 year ago

Re @charlesroddie's point. ☝️

Does the creation of .webcil - given the implied eternal support overhang - Is this an announcement that AOT won't and can't target WASM?

jez9999 commented 1 year ago

Maybe a bit of a dumb question, so sorry in advance, but if I'm using this new format, what will my dev tools Network tab look like if I do a hard reload? Will I see a ton of files being downloaded like I do now (the equivalent of the .dll's) or just one big webcil file? What will the MIME type be?

paulguz-datapa commented 1 year ago

Thanks, this is an exciting development.

There was some chat on the WebAssembly.MultipartBundle thread that the "application/octet-stream" MIME type is likely to be flagged by AV tools.

How can you be sure that AV and firewalls won't block this new and unfamiliar file type as being potentially malicious?

lambdageek commented 1 year ago

As technically impressive as this work may be, it's an extensive workaround to the current blazor inadequacy of deploying dlls to user systems rather than compiling everything to WASM. Full compilation (via a fix to Mono WASM AOT or NativeAOT) will make this obsolete unless it finds some other application than browsers.

We can work on more than one thing 😁

AOT approaches are a multi-release long-term effort. Webcil is a relatively low-effort potential mitigation for problems that customers are having today.

Re @charlesroddie's point. ☝️

Does the creation of .webcil - given the implied eternal support overhang - Is this an announcement that AOT won't and can't target WASM?

Absolutely not. We're still working on AOT.

I'm not sure what you mean by "eternal support overhang".

Webcil is: (1) experimental, (2) an implementation detail. I full expect the format may change as we get more customer data about whether it's solving their issues.

The intention of webcil is to be an artifact of the final app bundling step before deployment. It's not seeking to replace regular Ecma-335 .NET assemblies as output of existing compilers for C#, F# or other languages; not intended to be supported as an input to tools such as the IL trimmer, etc. It's just a transformation we apply to an app before deployment.

lambdageek commented 1 year ago

Maybe a bit of a dumb question, so sorry in advance, but if I'm using this new format, what will my dev tools Network tab look like if I do a hard reload? Will I see a ton of files being downloaded like I do now (the equivalent of the .dll's) or just one big webcil file? What will the MIME type be?

It will be exactly like today but with .webcil files instead of .dll. Mime type will be application/octet-stream.

lambdageek commented 1 year ago

Thanks, this is an exciting development.

There was some chat on the WebAssembly.MultipartBundle thread that the "application/octet-stream" MIME type is likely to be flagged by AV tools.

How can you be sure that AV and firewalls won't block this new and unfamiliar file type as being potentially malicious?

We're not sure. If that is happening, I would love to get some reproduction steps: a specific AV tool and configuration that I should try. From the discussion in the other issue, it was unclear to me if any application/octet-stream data will get flagged or only application/octet-stream containing PE data ("MZ" bytes etc...)

I have some further ideas if application/octet-stream is a problem. But it will require some more intrusive changes to the runtime, so I wanted to avoid going in that direction unless it's necessary.

jez9999 commented 1 year ago

Maybe a bit of a dumb question, so sorry in advance, but if I'm using this new format, what will my dev tools Network tab look like if I do a hard reload? Will I see a ton of files being downloaded like I do now (the equivalent of the .dll's) or just one big webcil file? What will the MIME type be?

It will be exactly like today but with .webcil files instead of .dll. Mime type will be application/octet-stream.

That doesn't deal with the problem of tons of files being downloaded, then. Why can't they all be bundled into 1 file, at least optionally?

lambdageek commented 1 year ago

That doesn't deal with the problem of tons of files being downloaded, then. Why can't they all be bundled into 1 file, at least optionally?

Yea, we have some flexibility here. Right now it's still 1 assembly -> 1 webcil file because I wanted to get feedback about whether this actually solves the AV & firewall issues for anyone. But with the infrastructure already in place we can be more flexible with how we bundle.

There could be issues with bundling everything into one file, however. for example if you deploy an update of an app, then it'll be a brand new bundle and users won't have it in their cache. the way things are today, if something (for example CoreLib, or some assemblies from some third party nuget) hasn't changed, that stuff will be in the user's cache already and only the updated assemblies would be downloaded.

jez9999 commented 1 year ago

I think a good way to do it would be to separate it into 3 packages, framework code, NuGet packages, and application code. The latter are more likely to get updated, but it'd be a good balance because it's still only 3 downloads.

campersau commented 1 year ago

Please make sure to also update the docs here https://github.com/dotnet/AspNetCore.Docs/blob/main/aspnetcore/blazor/host-and-deploy/webassembly/_samples/web.config for the new extension / mime type, so it works with IIS hosting as well for blazor.

paulguz-datapa commented 1 year ago

Thanks, this is an exciting development. There was some chat on the WebAssembly.MultipartBundle thread that the "application/octet-stream" MIME type is likely to be flagged by AV tools. How can you be sure that AV and firewalls won't block this new and unfamiliar file type as being potentially malicious?

We're not sure. If that is happening, I would love to get some reproduction steps: a specific AV tool and configuration that I should try. From the discussion in the other issue, it was unclear to me if any application/octet-stream data will get flagged or only application/octet-stream containing PE data ("MZ" bytes etc...)

I have some further ideas if application/octet-stream is a problem. But it will require some more intrusive changes to the runtime, so I wanted to avoid going in that direction unless it's necessary.

I hope you will be proactive in testing this against multiple AV and firewalls. There are numerous products listed in 31048 and on the BlazorWasmAntivirus package page. Barracuda Cloudgen has been mentioned too.

I'm sure all interested parties will do their best to try on known affected products, but there will be most success if you can try on the known products as you go.

Will you inform all the AV and firewall manufacturers about webcil, so as to fend off as many false positives as possible?

paulguz-datapa commented 1 year ago

Maybe a bit of a dumb question, so sorry in advance, but if I'm using this new format, what will my dev tools Network tab look like if I do a hard reload? Will I see a ton of files being downloaded like I do now (the equivalent of the .dll's) or just one big webcil file? What will the MIME type be?

It will be exactly like today but with .webcil files instead of .dll. Mime type will be application/octet-stream.

That doesn't deal with the problem of tons of files being downloaded, then. Why can't they all be bundled into 1 file, at least optionally?

Do we need bundling with HTTP/2 and later?

jez9999 commented 1 year ago

You can't assume that's always being used.

Schaeri commented 1 year ago

Is it possible to try out the new WebCIL file format in Preview 1 of dotnet 8? We would like to get feedback from our customers as quickly as possible to determine whether this new format solves the firewall problems in our environment. Is there any documentation or how-to guide on how to create a Hello World Blazor WebAssembly application using the new WebCLI format?

lambdageek commented 1 year ago

@Schaeri Webcil doesn't work with blazor yet - that's coming in a future .NET 8 Preview.

You can try it out by building a wasmbrowser app using the wasm-experimental workload (details in this blog post) and setting the WasmEnableWebcil property to true in the csproj:

  <PropertyGroup>
    <WasmEnableWebcil>true</WasmEnableWebcil>
  </PropertyGroup>

Something like this will work:

$ dotnet workload install wasm-tools
$ dotnet workload install wasm-experimental
$ dotnet new wasmbrowser
$ dotnet publish -c Release -p:WasmEnableWebcil=true

Now serve the bin/Release/net8.0/browser-wasm/AppBundle/ folder and have the affected customer load the page. They should see something like "Hello, World! Greetings from [whatever URL you served]".

Please let us know whether it is working or not. If it doesn't work, it woudl be great to know what AV tool is being used and how it is configured (please ping me - we can discuss in private, if necessary)

peterthorpe81 commented 1 year ago

That doesn't deal with the problem of tons of files being downloaded, then. Why can't they all be bundled into 1 file, at least optionally?

Yea, we have some flexibility here. Right now it's still 1 assembly -> 1 webcil file because I wanted to get feedback about whether this actually solves the AV & firewall issues for anyone. But with the infrastructure already in place we can be more flexible with how we bundle.

There could be issues with bundling everything into one file, however. for example if you deploy an update of an app, then it'll be a brand new bundle and users won't have it in their cache. the way things are today, if something (for example CoreLib, or some assemblies from some third party nuget) hasn't changed, that stuff will be in the user's cache already and only the updated assemblies would be downloaded.

I would think bundling would hit loading performance for Blazor? It couldn't begin loading until its got the full bundle rather than logical smaller files. In a https/2 or 3 world i'm not sure of the benefits of a bundle.

There could be some opportunities to reduce the perceived loading time of Blazor by partitioning files independently of dll structure. Blazor could download an initial webcil file containing the classes necessary to begin loading. Other webcil files download in the background and are hopefully obtained before they are needed as the site loads. Essentially a lazy loading process not bound by dll boundaries.

I think you will get more testers once its available for Blazor Wasm.

rffsgate commented 1 year ago

@Schaeri Webcil doesn't work with blazor yet - that's coming in a future .NET 8 Preview.

You can try it out by building a wasmbrowser app using the wasm-experimental workload (details in this blog post) and setting the WasmEnableWebcil property to true in the csproj:
  <PropertyGroup>
    <WasmEnableWebcil>true</WasmEnableWebcil>
  </PropertyGroup>
Something like this will work:
$ dotnet workload install wasm-tools
$ dotnet workload install wasm-experimental
$ dotnet new wasmbrowser
$ dotnet publish -c Release -p:WasmEnableWebcil=true
Now serve the bin/Release/net8.0/browser-wasm/AppBundle/ folder and have the affected customer load the page. They should see something like "Hello, World! Greetings from [whatever URL you served]".

Please let us know whether it is working or not. If it doesn't work, it woudl be great to know what AV tool is being used and how it is configured (please ping me - we can discuss in private, if necessary)

The small demo using webcil worked for us. Working with 2 companies that have ZScaler enabled, and both have an issue where blazor WASM projects don't load.

Need to have a solution for this before we can continue.

paulguz-datapa commented 1 year ago

Need to have a solution for this before we can continue.

BlazorWasmAntiVirus doesn't work for you?

rffsgate commented 1 year ago

Need to have a solution for this before we can continue.

BlazorWasmAntiVirus doesn't work for you?

No, unfortunately it put my project in a state where it was just error after error. Spent way too many hours trying to recover afterwards. Kept on coming up with errors that the file was locked, and couldn't be processed. Then it failed because it could not load the dll. Then it ended up that something was cached somewhere, because even after removing the nuget package, and deleting the compiled artifacts, it still didn't recover. It put my installation in a state where I can't even create a new blazor web assembly project. The dotnet 6 framework work, but the dotnet 7 one is broken properly.

paulguz-datapa commented 1 year ago

@rffsgate I have experienced the same on development machines. Sadly it is a known issue that the author is unable to fix. The workaround for us is to only use it in release builds, where it doesn't seem to be a problem.

rffsgate commented 1 year ago

Thanks @paulguz-datapa, at least I know I am not going nuts. Very frustrating. I guess, that's why I am really looking forward to a solution that is built-in and not a work-around.

paulguz-datapa commented 1 year ago

I've tried running this today using .Net 8 Preview 3.

The publish fails with the following, after a long pause:

C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\8.0.0-preview.3.23174.8\Sdk\WasmApp.Native.targets(
328,5): error MSB3073: The command "embuilder.bat build MINIMAL" exited with code 1. [C:\Users\xxx\wasmbrowser\wasmbro
wser.csproj]

There is no AppBundle folder. There is a 'publish' folder, but I get a JS error from dotnet.js if I run 'dotnet run' in that folder.

The embuilder error looks like a known issue that may well be fixed in Preview 4.

Schaeri commented 1 year ago

@lambdageek: We received the first feedback from our three largest customers, who blocked our application even with the currently available workarounds.

The feedback was positive, for two out of three the first .Net 8.0 preview webcil sample worked. The blocking had nothing to do with the WebCil. The Azure URL for our test seems to be on a blacklist.

The Webcil solution seems to be promising.

We hope for more feedback, as soon as we hear more we will report it here again.

lambdageek commented 1 year ago

Hey folks,

Just want to highlight something that we're currently working on with webcil. Our updated plan is to wrap the webcil payload in a WebAssembly module. The upshot is that a browser-based .NET WebAssembly app, we will consist of:

JavaScript files (.js)
WebAssembly modules (.wasm)

In addition to the usual dotnet.wasm that contains the runtime, we will have additional .wasm files containing the .NET assemblies (encoded as Webcil).

The rationale is that we want to be in a situation where we only serve standard web app MIME types and file extensions: text/javascript and application/wasm

As an implementation note, the webcil-in-wasm modules will export a pair of functions getWebcilSize and getWebcilPayload that will copy the size (respectively data) of a webcil-encoded .NET assembly to linear memory.

The updated Webcil spec and implementation are in #85932 the goal is to land this in an upcoming .NET 8 Preview after Preview 4 is out. (That is, Preview 4 will not have webcil-in-wasm - it will have webcil as .webcil same as earlier previews)

lambdageek commented 1 year ago

@jeromelaban @mikecodesdotnet @maxkatz6 FYI ^^^ https://github.com/dotnet/runtime/issues/80807#issuecomment-1540449454

In .NET 8, we're going to change how we package .NET assemblies for the browser. For Blazor this affected some of their publishing support around lazy loading https://github.com/dotnet/runtime/pull/85400, so you may also need to react in some way (or opt out)

.NET 8 Previews 1, 2, 3, 4: webcil is opt-in. If you opt in, assemblies are served as .webcil files
later .NET 8 Previews: webcil will be opt-out. If you opt in, assemblies are served as .wasm WebAssembly modules with webcil-in-wasm.

MikeCodesDotNET commented 1 year ago

@jeromelaban @MikeCodesDotNET @maxkatz6 FYI ^^^ #80807 (comment)

In .NET 8, we're going to change how we package .NET assemblies for the browser. For Blazor this affected some of their publishing support around lazy loading #85400, so you may also need to react in some way (or opt out)

.NET 8 Previews 1, 2, 3, 4: webcil is opt-in. If you opt in, assemblies are served as .webcil files

later .NET 8 Previews: webcil will be opt-out. If you opt in, assemblies are served as .wasm WebAssembly modules with webcil-in-wasm.

Thanks for the heads up! We really appreciate you letting us know of a potential issue for us. We'll investigate it this week and see what we need to do on our side.

lambdageek commented 1 year ago

Hello everyone,

Webcil has now been turned on by default for Blazor WebAssembly projects in .NET 8 Preview 5. Blazor apps should now consist entirely of .js .wasm, .html and .css files. No more .dlls. If you previously had problems running Blazor WASM apps due to firewalls or AV software, please try building with P5 and let us know how it goes. I am interested both in successes and failures. If it's working, let me know. If it's not working, definitely let me know

danroth27 commented 1 year ago

@mihaimyh I believe you mentioned elsewhere that you're interested in a solution to Blazor WebAssembly apps getting blocked by proxy or antivirus software. Have you had a chance to give the new webcil packaging a try in .NET 8 Preview 5? Did it resolve any issues you may have been seeing?

mihaimyh commented 1 year ago

@mihaimyh I believe you mentioned elsewhere that you're interested in a solution to Blazor WebAssembly apps getting blocked by proxy or antivirus software. Have you had a chance to give the new webcil packaging a try in .NET 8 Preview 5? Did it resolve any issues you may have been seeing?

Hi @danroth27 I have tried webcil packaging with .NET 8 preview, and indeed, it solved antivirus blocking the download of the application binaries.

Looking forward for .NET 8 release date.

rffsgate commented 1 year ago

Thanks for the updates. I can also confirm that this worked for our organisation. Zscaler and AV didn't block it this time.

paulguz-datapa commented 1 year ago

I can confirm that this works with G DATA Internet Security. Thanks for your work on this.

paulguz-datapa commented 1 year ago

I can also confirm Sophos AV.

lewing commented 1 year ago

@lambdageek I think we can close this and let the transport feed issue stay open

dotnet / runtime