dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.98k stars 4.66k forks source link

Single exe self contained console app #19227

Closed mattjohnsonpint closed 4 years ago

mattjohnsonpint commented 7 years ago

Publishing a self-contained console application in .net core, as described here results in a directory with a lot of files, even one with "smaller footprint".

Is it possible to bundle all the dependencies so we get a single executable? One large-ish myapp.exe when targeting Windows, for example.

I suppose I'm looking for something similar to ILMerge, but for .net core. Is it possible by some option flag on dotnet publish or some other command? If not, please consider as a feature request. Thanks.

mellinoe commented 7 years ago

This scenario isn't supported right now. Keep an eye on the https://github.com/dotnet/corert repo for work that will lead down this path.

karelz commented 7 years ago

CoreRT is the right place to search for this solution.

mattjohnsonpint commented 7 years ago

Thanks. It looks like CoreRT is exactly what I was looking for.

For others that may stumble on this, read here

jnm2 commented 7 years ago

I specifically want this feature without ahead of time compilation. I don't want corert– I want the same jit and debugging and runtime experience as a normal self-contained app– but I also don't want to distribute four files per utility. That's inconvenient for whoever wants to use the utility. I just want it packed and run from a single exe. This is causing me to choose .NET Framework over .NET Core for many utilities.

karelz commented 7 years ago

.NET Framework is "cheating" as you have plenty of files already present in the OS. You can achieve the same with .NET Core, by installing machine-wide shared .NET Core if that is what you want.

Refactoring CoreCLR to merge all its files into one (incl. native + managed) is non-trivial work and I don't think the scenario is that interesting for too many customers.

jnm2 commented 7 years ago

The difference is that every Windows machine is guaranteed to have .NET Framework but not .NET Core, so unless there's a special reason that is worth making deployment more complex, I guess I'll end up staying there. I guess customers don't typically write single exe utilities. Maybe one day. :-)

mellinoe commented 7 years ago

unless there's a special reason that is worth making deployment more complex

There are MANY reasons to prefer .NET Core over .NET Framework. Performance, modularity, serviceability, isolation, etc. And that is ignoring cross-platform-ness.

Having a single exe utility is mostly a cosmetic concern. There's no real functional difference (for anyone involved) if the executable is a single file or a single folder with a handful of files in it.

jnm2 commented 7 years ago

@mellinoe Oh I agree for projects in general, that is certainly true. In the small utilities where this cosmetic is desirable, .NET Framework has thus far done a good enough job. I agree it's cosmetic in the same sense as being able to specify an assembly company and copyright message. But being a cosmetic thing doesn't make me like it less. Obviously not a priority here, that's quite understandable.

ferventcoder commented 7 years ago

Possibly related: https://github.com/dotnet/core/issues/600

thefringeninja commented 6 years ago

It's not mostly cosmetic. Imagine that I want to distribute an application that has several components embedded inside, like say Idsrv4 or RavenDb + my own stuff. They take a dependency on the aspnet core bits. This all works fine as long as they depend on the same version of aspnet core. As soon as one of them rolls forward to the next version it probably will no longer work. If OTOH I could ilmerge each component down to a single .dll the problem goes away.

mellinoe commented 6 years ago

One thing to keep in mind is that there are more technical problems than just ILMerge not working. The runtime itself is fundamentally split into many files, and there are also auxiliary files like the CLR host, the runtime dependencies text file, and more. The only reason a single-file executable has worked in the past is because the .NET Framework, a global system-wide installation, was relied upon. Comparing the two scenarios is a bit unfair, because the .NET Framework doesn't really have a "self-contained" option.

This all works fine as long as they depend on the same version of aspnet core. As soon as one of them rolls forward to the next version it probably will no longer work.

This isn't really true. You can publish several different applications against different runtime versions and then bundle them into the same "uber-app". You just cannot mix their dependencies into a single folder. Even if you use the framework-dependent publish option, you can still mix and match versions. I understand that this is less convenient than having literally a single file, and might prevent things like embedding an exe as a resource in the PE. I'm not sure if that's what you're referring to, though, since you'd probably still need to extract the file to execute it, which you could still do with a directory.

thefringeninja commented 6 years ago

I didn't mean runtime version, I know that isn't going to work. What I mean is, what happens in my scenario when each component takes a dependency on LibXYZ@1.0.0. Component a then updates to LibXYZ@1.0.1. Normally that's fine, but in this hypothetical the authors of LibXYZ don't know / don't care about semantic versioning. Ilmerge made this problem go away.

edblackburn commented 6 years ago

As well as @thefringeninja pertinent point, I would suggest that a single deployable binary unit reduces the ops overhead. It reduces the likelihood of something going wrong with a deployment, lowers the burden of cognitively appreciating what you've shipped and reduces config for ops tooling be it scripting, or desired state.

I appreciate the dotnet tooling is nascent and has only just reached 2.0, but compared to alternatives such as golang this feature is sorely missing. Please consider adding it to the roadmap?

danbarua commented 6 years ago

As a lib developer, I'd appreciate having ILMerge back in Core. 👍

dazinator commented 6 years ago

Dotnet.exe is a single file. NuGet.exe is a single file. I think cosmetics are important, and a single file exe shouts simplicity in a way that a folder with 100 files in doesn't! Would be great to find a way to achieve this with dotnetcore

PRIMETSS commented 6 years ago

I assumed this was possible, and after several weeks of work, came to publish CLI console tool to a single .exe ... ummm seems at the moment I cant just get a single *.exe ?? I thought that was the point of self contained core apps?

svick commented 6 years ago

@PRIMETSS

I thought that was the point of self contained core apps?

The point of self-contained apps is that they are self-contained, that is, they don't require .Net Core to be installed on the target machine, because the app itself contains everything it needs to run.

To deploy such an app, you have to copy an entire directory of files. Having just a single file would make the deployment easier, but that's it, it's still a self-contained app even without that.

lashchev commented 6 years ago

For large and service-like apps this maybe not that important, but it is essential for command-line tools and small apps to be as simple as possible and COPY-deployment friendly.

I regularly write command line tools in .NET and having them as 1 EXE makes upgrades/deployments/packaging WAY simpler. I always pack all .config and other required files inside EXE.

Think of that single-EXE as a mini-container. Almost all benefits of container-based deployments apply here.

Scellow commented 6 years ago

This is just horrible i wonder why this was approved during design process, want to distribute your application to one of your client, then good luck to him to find what to open: image

This is just ugly and a complete mess

runtime/ MyProgram.exe MyLib.dll

This would have been a way better idea

Even Java handle that better: explorer_2018-04-21_16-42-33

PRIMETSS commented 6 years ago

Yes I was thinking along the same lines, since the self contained apps need to have the runtime files included with it with it (obviously!) (and we dont have a way to merge those), then I also thought just a folder separation would be neater, and yes I also think the runtime files should be stored off in a runtime folder, so anyone trying to use a Command Line Tool written in core, would at least find it a bit more intuitive as what to run..

imgen commented 6 years ago

@Scellow @PRIMETSS I have to agree

AnalogMan151 commented 6 years ago

How is this even still an issue? The demand is through the roof for this.

hkoelewijn commented 6 years ago

yes to this separate folder with the addition to zip that into a bin and have two files, the exe and the bin. Easy distribution.

CumpsD commented 6 years ago

Another +1 to re-open and have as a feature.

FlsZen commented 6 years ago

I was expecting self-contained deployment was self-contained and found my way here. I'd also like to see this happen.

FreeApophis commented 6 years ago

How can anyone say this is just a cosmetic issue? (Only Framework developers ...) Nobody thought of tools which are not installed? (.NET solved dll-hell, and .net core brings it back)

I just want my Command Line tool to be be self-contained.

This is bad, really bad!

ghost commented 5 years ago

+1

stansw commented 5 years ago

+1 I've just tried to run custom activity in Azure Batch on a cluster without .net core runtime and it cannot handle the high number of files the "self-contained" version creates.

Total size of resourceFiles cannot be more than 32768 characters
dgiagio commented 5 years ago

This isn't an official solution, but it might be useful. I'm the author of warp, a tool that allows you create self-contained single binary applications: https://github.com/dgiagio/warp

Piedone commented 5 years ago

I think warp is basically what everybody is looking for here! It's just about the ability to have a single exe file, whatever be in the background, nothing more. Think about NuGet packages: it's a single .nuget file and that's it; it's not something more complex than a file packaging other files.

Piedone commented 5 years ago

Warp produces an App.exe... How is it hacky? Obviously it's not part of the framework, but achieves the "one exe file" goal.

derekforeman commented 5 years ago

Warp works a treat. Thanks for your work @dgiagio!

loodakrawa commented 5 years ago

+1

I'd very much like to see some kind of single-file archive that bundles the executable and all the dependencies. Doesn't have to include the native ones - I think it's perfectly fine to require dotnet core do be installed on the system. Something very similar to Java's jar files. A dar file?

Something like: dotnet publish --self-contained dotnet run --self-contained MyApp.dar

jnm2 commented 5 years ago

It looks like there's a proposal up for this now: https://github.com/dotnet/designs/pull/52/files?short_path=28dba55

Here's the tracking issue: https://github.com/dotnet/coreclr/issues/20287

mattjohnsonpint commented 5 years ago

Also, @shanselman has a blog post on the Warp solution:

https://www.hanselman.com/blog/BrainstormingCreatingASmallSingleSelfcontainedExecutableOutOfANETCoreApplication.aspx

swaroop-sridhar commented 5 years ago

@RUSshy Are you asking about native-compiling all the managed code and linking everything together to get one executable (as in CoreRT)? This is not a goal for the current single-file work in .Net Core.

However, there are proposed solutions for reducing disk extraction and startup time (ex: load certain file types directly from the bundle, reuse extracted files). The stages of development of single-file solution are described here.

CC: @jeffschwMSFT

phillip-haydon commented 5 years ago

Why do the files need to be extracted? We used to get away with this in .NET with ILMerge.

swaroop-sridhar commented 5 years ago

@phillip-haydon, as described here, ILMerge and single-file solutions achieve different objectives.

ILMerge combines the IL from many assemblies into one, but in the process, loses the original identity of the merged assemblies. So, things like using reflection based on original assembly names will fail. Apps can still use ILMerge if they can tolerate this change.

thefringeninja commented 5 years ago

@swaroop-sridhar do you have an example of this working for dotnet core on linux? according to https://github.com/dotnet/ILMerge/blob/master/ILMerge/ILMerge.csproj#L13 this is a windows / mono only project.

swaroop-sridhar commented 5 years ago

@thefringeninja ILMerge is a windows tool, I don't know of any plans to port ILMerge to other platforms. The Single-file solution will be implemented cross-platform. This is in the process of development.

sgf commented 5 years ago

@karelz @mellinoe Deploy Deploy Deploy ! if Single or less like 1-3 files thats will be better for Deploy (Inlcude Publish,Update)!

sgf commented 5 years ago

CoreRT is AOT but ppl want AOT ? the project(CoreRT) always not Readly to use(its from 2014). Now is 2019!(about take 5 years) What a precious 5 years for a pragram language! the CoreRT even not any plan for next release !!! let ppl wait for more another 5 years?

thefringeninja commented 5 years ago

@swaroop-sridhar That's unfortunate. ILMerge/ILRepack is great for library developers to avoid the diamond dependency problem.

karelz commented 5 years ago

There are plans to support it in .NET Core 3.0 - @richlander can update you on status. It's been mentioned in .NET Core 3.0 blog posts - e.g. here: https://devblogs.microsoft.com/dotnet/net-core-3-and-support-for-windows-desktop-applications/ in "Side-by-side and App-local Deployment" section. Also read: https://www.hanselman.com/blog/BrainstormingCreatingASmallSingleSelfcontainedExecutableOutOfANETCoreApplication.aspx

sgf commented 5 years ago

.net shouldn't need all these hacks to produce small single self-contained executable.. it should be default behavior, you get eaten alive by GO, Rust is coming too, focus on CoreRT or ILMerge, .net core apps are already too long to start because of all the files to load, don't add more overheard to this with zip/unzip like mentioned in the proposal, this is not the way to go

As you like .net shouldn't need alive,because ppl has java. go or rust or dlang shouldn't support build target to Single executable program,because they have the Fuck Zip/Unzip !

terrajobst commented 5 years ago

@RUSshy

.net shouldn't need all these hacks to produce small single self-contained executable

Agreed. At the same time, .NET provides certain features over other languages that are challenging to reconcile with the requirements of a static linker (for example, on the fly code generation, reflection, data binding etc). We've experimented with excluding these features in scenarios where single file / static linking is important. We've learned from customer feedback that the result of this doesn't feel like .NET anymore, partially because the number of libraries you can consume from ecosystem drastically drops.

So doing a linker properly that doesn't compromise on the fidelity of the features that we've all taken a dependency and love in .NET, is a hard problem. As part of solving this, we're experimenting with various approaches. You call this hacks, I call this open minded and willing to think outside the box. But just blindly turning .NET into Go/Rust/C++ isn't the right answer.

kjkrum commented 5 years ago

Things like this are why Windows remains a toy OS.

Cryru commented 5 years ago

@kjkrum Things like what?

phillip-haydon commented 5 years ago

@cryru it’s what people say when they don’t know what they’re doing.

kjkrum commented 5 years ago

@Cryru Things like not being able to build a standalone executable, and the oblivious dismissal of the very real need to do so. Pretty much every other build system in existence will not only build a statically linked executable, it will strip unused parts of dependencies. Stripping dependencies has been a standard feature for at least a decade; static linking since the invention of compliers.