dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
34.59k stars 9.79k forks source link

Enhance Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle package to define a custom bundle format #36978

Closed danroth27 closed 1 year ago

danroth27 commented 2 years ago

We've built Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle as an experimental way to alter the packaging of Blazor WebAssembly apps so they work in environments that block the download of DLLs. We should consider if we want to release a stable and supported version of this package. Before we do so, we should verify that this packaging strategy is successful in these restrictive environments.

ghost commented 2 years ago

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

Andrzej-W commented 2 years ago

I think there are a lot of people who would like to develop applications in Blazor. At the same time many of them do not know that Blazor application can be blocked by a firewall and this can be an unpleasant surprise for them. In my opinion a solution like MultipartBoundle have to be supported by ASP.NET team and included in default Blazor templates.

danroth27 commented 2 years ago

@Andrzej-W Have you tried out the experimental multipart bundle functionality and does it enable Blazor WebAssembly app execution in environments were you were previously blocked?

Andrzej-W commented 2 years ago

Unfortunately my Blazor app is not ready yet, it is not published and I don't have personal experience with firewalls blocking Blazor app. The app is intended for a wide audience and I must do my best to make it accessible to everyone. I hope MultipartBoundle will be an effective solution to the firewall/antivirus problem.

Perhaps you should add to the JavaScript code responsible for loading this bundle the ability to signal problems by calling the configured URL. It would be useful for monitoring the scale of the problem. It is always possible that the proposed solution will work now and after a few days/weeks/months firewall vendors will discover it and the problem suddenly returns. This function should also be able to show an error message to the user, something similar to already available "blazor-error-ui", but of course it should be a different <div> with different message.

smartprogrammer93 commented 2 years ago

I did take a look at the approuch suggested in the blog post recently published and it seams rather complex. I believe a feature like this should be enabled by a matter of adding a nuget package to the project and adding a value to the csproject file to tell the build process to do the needful.

petertiedemann commented 2 years ago

@smartprogrammer93 I agree. I would like to do prototypes of offloading certain calculations to the client of our services by using WASM (combined with existing JS front-end code). Using .NET WASM is already something that makes front-end developers worried, and stuff like this just makes it impossible to even consider.

rmencia-isv commented 2 years ago

I did take a look at the approuch suggested in the blog post recently published and it seams rather complex. I believe a feature like this should be enabled by a matter of adding a nuget package to the project and adding a value to the csproject file to tell the build process to do the needful.

I totally agree. A default way to make the applications work in all scenarios is a must. If the applications generated in Blazor can be blocked by a proxy in a number of scenarios, then it should be implemented by default when built in Release by adding a checkbox in VS for the project.

Also, there is the issue of static web sites, in where there is no code running in the server, so the build should already do everything that is required to create the files in the final form.

What about AOT compilation? Does this process generate wasm files of the libraries that would not be blocked?

If MS had built Office Web running in Blazor webassembly, all this would be already up and running from day 1 because otherwise it would be blocked in most companies. Maybe you could work with one of the MS Teams and build something enterprise inhouse. That would give you a better understanding or the real issues not only during deploying but also bugs in the pre-release versions. Inhouse collaboration would allow you to have access to the source code also and debug issues early.

danroth27 commented 2 years ago

@smartprogrammer93 @petertiedemann @rmencia-isv Are any of you currently operating apps in environments where the download of Blazor WebAssembly apps is currently blocked? If yes, could you please try out the Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle package and let us know if it enables Blazor WebAssembly deployment in those problematic environments? Before we make a feature like this as part of the framework, we want to collect data from customers on whether it actually solves the problem.

peterblazejewicz commented 2 years ago

I think question was to @petertiedemann (but yes, I work for a company where topics here can apply as valid scenario)

rmencia-isv commented 2 years ago

Our app is in development now and still few months away. Also my app is designed to be a static web site calling APIs. Not sure how this solution would work either.

I'm in the education industry and our users will be part of schools and large universities. They are not tech savvy and every issue is a nightmare. We had heaps of issues with them blocking notification emails and I wasted weeks dealing with them to whitelist our app email. I can't even begin to imagine what it would be having to deal with reconfiguration of their proxies. I came across this issue by chance because I've created a number of issues of vs2022 and the latest update broke my blazor project due to my solution using a shared blazor library and I went to see the status and by chance I saw this issue inside 28 pages of issues. I can't imagine how many people have no idea about this and after they release the app it doesn't work and need to spend days or weeks to find a solution. Some of the architects may even get fired.

I much rather change technology all together or use Blazor server, as it's too risky. And really, it makes me nervous that you don't have real applications in MS using Blazor webassembly and collaborate with them to get early feedback.

danroth27 commented 2 years ago

@rmarinho We agree that this issue of Blazor WebAssembly apps getting blocked in certain environments is concerning, which is why we're exploring alternative packaging approaches. Please note though that it's not all environments that have this problem - it's specifically environments that block the download of DLLs. Many deployments of Blazor WebAssembly never hit this issue.

The request here is for folks that are deploying to these more restrictive environments to try out the Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle package and let us know if this alternative packaging strategy enables Blazor WebAssembly usage.

petertiedemann commented 2 years ago

@danroth27 Our own network is not restricted in such a way, but we are a software company developing software for large industrial enterprises, and I would not be surprised to see them doing this kind of blocking.

danroth27 commented 2 years ago

@petertiedemann Makes sense. We should have enough extensibility in place now that it should be possible to customize packaging to work for specific environments. Whether we can provide a generic solution that will work in all environments is what we're investigating with the multipart bundle package.

rmencia-isv commented 2 years ago

Please don't forget about static Blazor web sites when you think about that solution.

JohnGoldsmith commented 2 years ago

@danroth27 are you able to give more details about what “Some environments block…” means in terms of how wide the problem is and what those environments are likely to be (ie is it Enterprise only, Government and Defence etc)?

Also how would you know if you did hit this - is there a specific error code?

danroth27 commented 2 years ago

Hi @JohnGoldsmith. We've seen users report that some firewalls are setup to block the download of all Windows PE files. The files simply fail to get downloaded. We think this is only impact a small subset of users, not the majority, but enough users reported it that we are exploring ways to address the problem. Sometimes users can work with their IT departments to remove the restriction for their apps. When this isn't feasible, users can repackage the app with different files name extensions, which we provide instructions on how to do. In other cases the inspection of the files is deeper, so we added extensibility in .NET 6 to enable further customization of the app packaging. These extensibility points give users the flexibility to adapt to the requirements of specific environments. We're also exploring multipart bundling as a generic solution, and we're looking for verification from users that are currently blocked that this approach works for them.

JohnGoldsmith commented 2 years ago

@danroth27 that's good to know -thank you very much for the detailed explanation. It's also good to know that there's a way out if it does raises it's head (for me).

Thanks again.

CoryKoehler commented 2 years ago

This may be a dumb question but why are the .dlls from blazor apps not being turned into .wasm files?

danroth27 commented 2 years ago

@CoryKoehler They are. But the DLLs are still needed currently for certain code paths and for their metadata.

danleydmello commented 2 years ago

@danroth27 , willl this solution work for Static Blazor web app ? where there is no server to handle app.bundle

danroth27 commented 2 years ago

Hi @danleydmello. I believe this solution works for static sites as long as the site host gives you a way to setup the mime type mapping for the custom .bundle extension. For example, I think Azure Static Web Apps lets you do this. GitHub Page I think doesn't.

ghost commented 2 years ago

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

michael-andreev commented 2 years ago

Hi @danroth27 . First of all thank you for the proposed solution, it's really very important question. I have few things to consider.

I'm developing a new module for Cockpit (Admin UI for servers). My module is a Blazor Wasm App that is hosted as a set of static files by the internal Cockpit Web Server (Cockpit modules must be client-side apps). Unfortunately I can't add required HTTP headers, because it's not supported by Cockpit.

I could ask Cockpit Developers to add required HTTP Header for *.bundle files globally into the main Cockpit Project, but I predict various reasons why they might decide to reject these changes:

  1. Current Mime-Type (multipart/form-data; boundary="--0a7e8441d64b4bf89086b85e59523b7d") looks uncommon for resource files, especially with static ID in boundary field. It would be better to find a solution that works with some common Mime-Type (like application/octet-stream).
  2. .bundle file extension is quite abstract. It would be easier to register proposed Mime-Type in Cockpit for more concrete file extension (like .netpack or .blazor). In case if it would be possible to change Mime-Type to octet-stream some well-known extension like .bin might also be better.

There are few other things that I'd like to mention:

  1. Bundling gives huge speed gain for the initial loading under Cockpit (up to 12x on a very fast PC with the app running from localhost). With app.bundle file the boot time is under 3 seconds, while with .dll files there are horrible 35 seconds. I used compression in both cases. So bundling makes a big sense in terms of performance and it's a must have for my App.
  2. I had to add additional CORS for proposed bundling method in order to work under Cockpit (default-src 'self' ... data: blob:; script-src-elem 'self' ... 'unsafe-inline' data: blob:; connect-src 'self' ... blob:). It's not a blocker for me, because CORS are configurable in Cockpit, but it's better to avoid that (it's not required to configure such CORS for .DLL files approach).
  3. May be you could consider adding required Mime-Type for GitHub Pages globally. It would allow people to host static Blazor Wasm Apps in GitHub. It would also allow me to tell Cockpit Developers that this Mime-Type has already been added for GitHub Pages, so they could do the same.
  4. WebAssembly deployment layout manual proposes to consider using AOT compilation in order to exclude DLL files. Am I correct that it's currently an outdated info? Should I add a new issue regarding that?

Thank you once again for your work. Hope that my questions can be solved somehow.

CoryKoehler commented 2 years ago

I have put this into my wasm app and deployed it and google ads and other security / vpns are still flagging our blazor wasm.

I see this is being pushed back to .net 7 and this is something that we cannot wait a year on.

I have also tried AOT and other solutions that are outlined in the web assembly deployment layout manual documentation

esben2000 commented 2 years ago

I would really like to use the Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle package, because my company blocks dlls, so we cannot see Blazor WASM applications. However, after installing the Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle NuGet package I get this Error when I Publish: Could not copy "C:\Users....nuget\packages\microsoft.aspnetcore.components.webassembly.multipartbundle\0.1.0-alpha.21470.2\staticwebassets\Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle.lib.module.js" to "C:...\Client\obj\Release\net6.0\browser-wasm\PubTmp\Out\wwwroot_content\Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle\Microsoft.AspNetCore.Components.WebAssembly.MultipartBundle.lib.module.js". Exceeded retry count of 10. Failed.
Any idea what is wrong?

legistek commented 2 years ago

@CoryKoehler They are. But the DLLs are still needed currently for certain code paths and for their metadata.

@danroth27 according to the Deployment Layout documentation an "Alternative[] to consider include" is:

Use Ahead-of-time (AOT) compilation and deployment, which produces a compiled WebAssembly app that doesn't require downloading DLLs to clients.

Is this incorrect?

legistek commented 2 years ago

Maybe I'm being super naive here, but couldn't the DLLs be embedded in a base64-encoded string in a .JS file that .JS then decodes and feeds to Mono? Let their firewalls see through that.

Oh and obfuscate it too while you're at it. A simple XOR cipher should do the trick.

Update: Ok yeah, I'm trying out this package and looking at the app.bundle, and I don't think this is going to fool any firewall. It announces its Mime type as application/octet-stream (redflag) and then is immediately followed by the famous MZ.

I'll try it out with one of my clients who's the most paranoid completely justifiably conservative type you'll ever meet when it comes to this stuff, and let you know. But assuming it doesn't work I'll try out the customizations like embedding in a JS file and obfuscation. We've come too far to let this be a blocker.

Update 2: Oddly enough, my client's system is NOT blocking Blazor apps. I'll have to see if I can find another client who is. Will certainly let you guys know if I do and how it goes.

danroth27 commented 2 years ago

@danroth27 according to the Deployment Layout documentation an "Alternative[] to consider include" is:

Use Ahead-of-time (AOT) compilation and deployment, which produces a compiled WebAssembly app that doesn't require downloading DLLs to clients.

Is this incorrect?

Whoops! No, that's wrong. Even when you AOT compile, the assemblies are still needed for their metadata and other reasons. I'll get the doc corrected. Thanks for bringing this to my attention! It's a common point of confusion. @guardrex FYI.

guardrex commented 2 years ago

Doc update complete, including the additional lead-in text changes, and will go live within 24 hours.

stavroskasidis commented 2 years ago

I noticed that Bitdefender now blocks .NET 6.0 blazor wasm apps but not .NET 5.0. Maybe that is something that can be communicated with Bitdefender ?

legistek commented 2 years ago

I noticed that Bitdefender now blocks .NET 6.0 blazor wasm apps but not .NET 5.0. Maybe that is something that can be communicated with Bitdefender ?

Does it still block it when using the MultipartBundle package?

stavroskasidis commented 2 years ago

Haven't tested it yet. I will try it and report here the results

UPDATE: The multipartbundle package is still getting blocked

stavroskasidis commented 2 years ago

This needs to be taken more seriously and with more urgency. It also affects google ads and any site getting flagged is a huge mess. Also the user trust goes out the window the moment they see a site blocked by their antivirus.

This a serious matter that should be resolved out-of-the-box asap, not in a year or two.

legistek commented 2 years ago

What is the exact security product / edition you're testing this with? I'd like to try it and see exactly what happens.

As I said, and no offense intended, but I don't understand why anyone on this team thought that simple multipart bundling was going to change anything from a security perspective. The files need to be made to look like JavaScript or something else these systems have been trained to allow. "application/octet-stream" is basically announcing you're something dangerous - which of course Blazor ISN'T because it's as safe as JavaScript.

stavroskasidis commented 2 years ago

I have created a nuget package to try and circumvent false positives from antiviruses. More info on the project's page: https://github.com/stavroskasidis/BlazorWasmAntivirusProtection

Please give it a try to see if it helps.

legistek commented 2 years ago

I have created a nuget package to try and circumvent false positives from antiviruses. More info on the project's page: https://github.com/stavroskasidis/BlazorWasmAntivirusProtection

Please give it a try to see if it helps.

Thank you for posting this. What exact antivirus products are you testing against? More importantly which ones block Blazor normally? You mentioned Bitdefender but they have half a dozen products and I want to make sure I get the right edition to test with.

stavroskasidis commented 2 years ago

I have created a nuget package to try and circumvent false positives from antiviruses. More info on the project's page: https://github.com/stavroskasidis/BlazorWasmAntivirusProtection Please give it a try to see if it helps.

Thank you for posting this. What exact antivirus products are you testing against? More importantly which ones block Blazor normally? You mentioned Bitdefender but they have half a dozen products and I want to make sure I get the right edition to test with.

I can confirm that the following are blocking blazor wasm 6.0. You can test it easily with sites like mudblazor.com

legistek commented 2 years ago

Thank you! This is extremely helpful. I will download trial versions of these products and test with my live dev site and also try your solution. We will figure this out.

MartinVetsoft commented 2 years ago

image I have the problem that the blob: requests are blocked. Has anyone an idea why this happens?

EDIT: solved with "script-src blob:" in the Content-Security-Policy

MartinVetsoft commented 2 years ago

image Why does it still load DLLs?

paulguz-datapa commented 2 years ago

Re my duplicate issue #40319. We have a customer whose GData Internet Security flags up files written to the cache as Trojans. Neither bundling or the package provided by @stavroskasidis fix this.

This is our second customer who has hit this, with two different Antivirus providers. I'm with @stavroskasidis and @legistek; this needs to be addressed as a dire emergency. .Net 7 is too far away. Blazor Wasm is not fit for purpose for developing enterprise software if it is blocked by AV and firewalls.

paulguz-datapa commented 2 years ago

We've been able to work around the GData caching issue by turning off Blazor caching, as per the docs.

danleydmello commented 2 years ago

We've been able to work around the GData caching issue by turning off Blazor caching, as per the docs.

Is that mean dlls will be downloaded everytime by the browser?

paulguz-datapa commented 2 years ago

@danleydmello I expect so, yes

paulguz-datapa commented 2 years ago

@danleydmello I expect so, yes

Kruki commented 2 years ago

I think this whole problem is way bigger than most people think. I am a developer from Austria, developing a B2B application with blazor since the release of .NET5.

From my experience with (Austrian) customers, i can confidentially say, that every 4th or 5th customer is experiencing this issue.

I dont know if IT-staff is not updating their firewalls or if they just use very restrictive systems, but this is definitly not a problem where we can say we wait until every firewall adapts blazor. That will just not happen.

We can luckily handle the problem very well because it is a pure B2B application, so we can tell each customer individually to whitelist our application in their firewall.

Next week i would like to start developing our corporate website using blazor, but i think i will instead go with angular or vue, because i can just not risk a lot of potential customers to skip our product because the website is not loading.

I have to say that i really fell in love with blazor, but i think this issue is really threatening the future of it...

danroth27 commented 2 years ago

From my experience with (Austrian) customers, i can confidentially say, that every 4th or 5th customer is experiencing this issue.

@Kruki Can you confirm you are seeing this issue even when using the multipart bundling approach? Have you also tried https://github.com/stavroskasidis/BlazorWasmAntivirusProtection? Do you have any additional details on what specific firewall software your hitting issues with and what specific check it is doing that is causing blocking issues?

Kruki commented 2 years ago

I will implement these approaches with the next update!

I cannot tell specifically which systems customers are using, because we dont want them to bother to talk with us more about the error, we are glad that it works after they whitelist our software.

I will ask our support team to find out what firewall systems are used by the troubled customers next time!

legistek commented 2 years ago

I've found in the past with firewall issues and customers that it's impossible to get any information out of them as to why the system blocked it. We're lucky to get it whitelisted and usually leave it at that.

But @danroth27 it's pretty easy to see what's going on - many systems block executables wholesale. Whether they are looking for the EXE/DLL extension, "MZ", or something more sophisticated, it's impossible to know without trial and error. The only thing to do here is make it look as little like an executable as possible.

rmencia-isv commented 2 years ago

The Blazor team could implement some code in the loader that when the file doesn't load (it's blocked), it logs the problem. It' could be integrated with Azure Application Insights and the company could gather all that information and find the affected customers. That information can be shared with Blazor team and help find solutions.

Also, at MS, it wouldn't be too difficult to set up a few of the most common Antiviruses and Firewalls in a couple of VMs in Azure. I'm sure that if you contact the AV/Firewall companies they can even give you a free license for testing purposes and try all these scenarios.

Also, encrypting the DLLs during the build should work. Then, the loader can decrypt them on loading. The key would be public, but it's not a security risk, it's only to bypass the firewalls.

Maybe it's worth thinking about compiling the whole lot of DLLs and dotnet as webassembly code and run them directly instead of using mono and bypass the whole problem.