Support Platform-independent single-file app DLLs

Logerfo commented 4 years ago

Currently, single executable applications (with or without trimming) packages both framework and third party libraries. Because of the former, the result is very large sized if compared to non-packaged builds. There should be an option to only package third party libraries, still relying on the installed framework. An alternative is Costura, but: 1) It's not free; 2) It's being deprecated in favor of single executables, as for Fody/Costura#442.

I don't think it would be hard to implement this, since the current feature is more complex than the requested alternative approach.

jkotas commented 4 years ago

cc @swaroop-sridhar

swaroop-sridhar commented 4 years ago

@Logerfo I think what you need to do is to publish the app as a framework-dependent single-file. That is, dotnet publish -r <runtime> --self-contained=false /p:PublishSingleFile=true

This will embed all dependencies of the app except the framework binaries.

Logerfo commented 4 years ago

@swaroop-sridhar I didn't know about that possibility, but it still doesn't make sense to me. Why does it require a runtime identifier to be specified if it's not self contained? The default runtime environment is supposed to be framework dependent...

swaroop-sridhar commented 4 years ago

@Logerfo the single-exe app includes the managed components and the native host. The result is a native binary that must be built per target OS/architecture. Therefore, PublishSingleFile option requires that a RID be specified.

The framework-dependent aspect just means that we don't package the framework binaries into the app -- thus making it smaller. That is, the resultant app is still a platform specific native binary containing all dependencies except for framework binaries (which need to be installed elsewhere).

The current support for framework dependent single-file apps seems to fulfill your original objective -- embedding third-party dependencies and not including the framework.

What you're probably also asking is to embed all dependencies into the (portable) managed app, which can then be launched via the dotnet host. We don't support embedding dependencies into managed assemblies yet. This is similar to supporting single-file plugins -- which may be considered in further stages.

swaroop-sridhar commented 4 years ago

Adjusted the title to refer to the issues discussed in the previous comment

Logerfo commented 4 years ago

Another problem: I have a console application which has references to System.Windows.Forms in order to create a notification icon in the task bar. This worked fine with .NET Framework and Mono, since it would only crash on Linux when the method was called (and of course my application only allows it if it's being run on Windows). While migrating to .NET Core, I noticed a new file called <MyApplication>.runtimeconfig.json. This file has a reference to Microsoft.WindowsDesktop.App, which makes the application unlaunchable on Linux. Manually changing it to Microsoft.NETCore.App after building solved the problem, making the application behave just like Mono would (crash only when encounters unavailable API). Publishing as a single file embeds this json file inside the published file, so I cannot edit it anymore. I don't know if it's safe (and if it would work) to open the file in a text editor and change it nevertheless.

devedse commented 4 years ago

@swaroop-sridhar ,

When I tried running your command I saw the following error:

C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF>dotnet publish -r win-x64 -c Release --self-contained=false /p:PublishSingleFile=true /p:PublishTrimmed=true
Microsoft (R) Build Engine version 16.3.0+0f4c62fea for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF\DeveImageOptimizerWPF.csproj : warning NU1701: Package 'MvvmLightLibs 5.4.1.1' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETCoreApp,Version=v3.0'. This package may not be fully compatible with your project.
  Restore completed in 951.03 ms for C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF\DeveImageOptimizerWPF.csproj.
C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF\DeveImageOptimizerWPF.csproj : warning NU1701: Package 'MvvmLightLibs 5.4.1.1' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETCoreApp,Version=v3.0'. This package may not be fully compatible with your project.
  DeveImageOptimizerWPF -> C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF\bin\Release\netcoreapp3.0\win-x64\DeveImageOptimizerWPF.dll
C:\Program Files\dotnet\sdk\3.0.100\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ILLink.targets(113,5): error NETSDK1102: Optimizing assemblies for size is not supported for the selected publish configuration. Please ensure that you are publishing a self-contained app. [C:\XGitPrivate\DeveImageOptimizerWPF\DeveImageOptimizerWPF\DeveImageOptimizerWPF.csproj]

swaroop-sridhar commented 4 years ago

@Logerfo editing the single-file app once built is not a good idea. The single-file app contains a directory of contents, with hard-coded offsets and sizes. So, there's a good chance your edit will break the knowledge encoded in this directory.

The edit to change the framework seems like a hack to me. My advise would be:

To have two different projects which target the correct frameworks on Windows and Linux.
I don't recommend this -- but if you must edit the configuration file, do it during the build by inserting targets into the publish step (something that runs before BundlePublishDirectory target in the SDK).

swaroop-sridhar commented 4 years ago

@devedse You've added the /p:PublishTrimmed=true setting. Trimming assemblies for framework-dependent apps is not yet supported (because there's no framework assemblies to trim).

But you can still publish a framework-dependent app as a single-file.

devedse commented 4 years ago

@swaroop-sridhar ,

The startup times of DotNet Core Single File WPF apps is a lot slower then the original ILMerge-ed WPF application build on .net 4.7. Is this to be expected or will this improve in the future?

Builds come from my ImageOptimizer: https://github.com/devedse/DeveImageOptimizerWPF/releases

Type	Estimated First Startup time	Estimated second startup time	Size	Download link
.NET 4.7.0 + ILMerge	~3 sec	~1 sec	39.3mb	LINK
dotnet publish -r win-x64 -c Release --self-contained=false /p:PublishSingleFile=true	~10 sec	~3 sec	49mb
dotnet publish -r win-x64 -c Release /p:PublishSingleFile=true	~19 sec	~2 sec	201 mb
dotnet publish -r win-x64 -c Release /p:PublishSingleFile=true /p:PublishTrimmed=true	~15 sec	~3 sec	136mb	LINK
dotnet publish -r win-x64 -c Release	~2.5 sec	~1.5 sec	223kb for exe (+400mb in dlls)

swaroop-sridhar commented 4 years ago

@devedse, to make sure, is the "second startup" the average of several runs (other than the first)? I'm curious, but lacking any explanation for why the /p:PublishSingleFile=true /p:PublishTrimmed=true run should be slower than `/p:PublishSingleFile=true run.

So, before investigating, I want to make sure the numbers in the "second startup" are stable numbers and that the difference in startup is reproducible,

Also, this issue is about single-file plugins, can you please move the perf discussion to a new issue, or to https://github.com/dotnet/coreclr/issues/20287? Thanks.

ghost commented 4 years ago

Tagging subscribers to this area: @swaroop-sridhar Notify danmosemsft if you want to be subscribed.

Serentty commented 4 years ago

Would these DLLs be openable by double clicking them, similar to the old CIL EXEs or Java JARs, or would they still require the user to pass them to the dotnet command manually?

Serentty commented 4 years ago

Single-file platform-independent DLLs would be great, but from a UX perspective for a GUI user, they would still need some sort of shell script or LNK shortcut to launch them with the dotnet loader, no? This is easy on the command line where you can just run dotnet Application.dll, but it's harder when you've just unzipped an application and are opening it from the GUI file browser. That's certainly far from an insurmountable issue, since it's easy to cover nearly every platform with just those two tiny files (.lnk and .sh), but it means that things are back up to three files total. What could be done for a portable GUI launching solution?

vitek-karas commented 4 years ago

Would these DLLs be openable by double clicking them, similar to the old CIL EXEs or Java JARs, or would they still require the user to pass them to the dotnet command manually?

Short answer: no.

In order to support double-click like action on anything but executables there would have to be one of two things:

change to the OS itself - this is how .NET Framework does it - the managed .exe is recognized by the OS loader itself and it redirect execution to the .NET Framework runtime to start the application. One of the main design goals of .NET Core is to not have any component in the OS itself - so we have no plans to support this (not counting what it would mean to do something like this on other platforms)
use a different extension (at least on Windows, since .dll already means lot of other things) and register some app (dotnet.exe possibly) to open that extension. Not sure if we would be able to do this correctly on all supported platforms (not counting that it makes little sense on platforms without GUI) - that said, this could be done by external software - there's nothing in the systems to prevent anybody from registering a file extension handler.

Additionally for GUI applications (at least on Windows) - running app.exe and running dotnet app.dll is not completely identical. On Windows the .exe contains native resources like the icon, manifest, version and so on. These are things handled by the OS itself, some of them even before the app gets to run any code (icon and to some degree manifest for example). dotnet.exe can't do these (for example the icon), so by running the app via dotnet.exe looses some of this functionality. Also dotnet.exe is currently a CUI (Console UI) application, which means that when started OS will automatically open a console window for it (unlike GUI apps which don't get console window) - this is pretty distracting. It's technically possible for dotnet.exe to somehow detect that it's running a GUI app and close the console, but it console window would still popup/close for a short time, there's no way to avoid that. (In theory dotnet.exe could be marked as GUI and open console when it needs to, but that is actually a very tricky thing to do correctly in all situations, it's highly unlikely we would go this route).

I don't know enough about GUI systems on Linux/Mac to be able to tell if there's a more elegant solution on those platforms.

Serentty commented 4 years ago

I don't know enough about GUI systems on Linux/Mac to be able to tell if there's a more elegant solution on those platforms.

On Linux (and I think it's the same on Mac), every executable is fundamentally a command-line one, and just has the option of opening a GUI window. If you double click on an executable which doesn't open a GUI, it will just run and close without anything visible happening. So I don't think associating a file with dotnet would be an issue there. The issue of the fact that Windows distinguishes between CUI and GUI executables didn't occur to me, and it's certainly a bit of a conundrum.

I think a good cross-platform solution could be to have a loader which is itself a .NET program, and which includes the native loader as usual (and is therefore platform-specific). Then, it could be associated with a file extension for single-file .NET applications, and when you open one of those applications, hopefully it would be able to call the entry point without having to create an entire new process. Such a tool could be made and distributed separately from .NET itself, but depending on it would probably be too risky at that point, since users would be very unlikely to have it, so the file would just be unrecognized. So I think for such a format to be useful it would have to be part of .NET itself.

Ultimately though, I can see why this is not as large of a priority as it used to be. Opening loose executables off a disk is becoming a lot less common in a world of package management, so having a platform-specific wrapper script to run dotnet Application.dll isn't hard for a package maintainer to add even for a closed-source application that they don't have the source to.

dotnet / runtime

Support Platform-independent single-file app DLLs #13677