search

microsoft / winget-cli

WinGet is the Windows Package Manager. This project includes a CLI (Command Line Interface), PowerShell modules, and a COM (Component Object Model) API (Application Programming Interface).
https://learn.microsoft.com/windows/package-manager/
MIT License
23.37k stars 1.45k forks source link

Proposal to drop .EXE support #407

Closed jantari closed 3 years ago

jantari commented 4 years ago

Description of the new feature/enhancement

This is a suggestion / spec / discussion on how I believe winget should NOT support any .exe Setups/Installers and instead I propose a (more or less - see below) strict MSIX-only approach.

This is intended to be an open discussion about #223, specifically continuing on this comment by @VincentJoshuaET

@aclinick replied there that the adoption of MSIX is not far enough along yet and it would significantly reduce the amount of available apps.

This proposal is the formal version of replying "so what?" to this. We want a package manager for Windows. A real one. Microsoft has MSIX, it's open-source, and they have winget, it's open-source. I want to see how many others would prefer a tiny starting software selection of REAL packages with all the goodies over a larger software selection of glorified one-way install-scripts.

Let us build the right thing now for the future, instead of reinventing the past we're tring to get away from so desperately

What is wrong with supprting .EXE (not a complete list by ANY means):

  1. .EXE Setups are often not "transactional" packages, aka changes they make are not reverted on uninstall
  2. They complicate the manifest format a lot with varying silent arguments and/or answer files
  3. There are no guarantees in install behavior and they all vary a lot:
    • Even with silent arguments, some installers invoke hidden windows that steal focus
    • With silent arguments, some installers may choose undesirable defaults
    • Even with silent arguments, an installer may show a MessageBox on Error
    • Even with silent arguments, some installers do not run/work when there is no user logged in or no graphical user session available (which breaks remote winget installs via ssh/WinRM when nobody is physically logged in to the machine)
  4. It is unclear to the user of winget whether an .EXE Setup installs per-user or per-machine
  5. Since the programs installed can update on their own and do not have a truly reliable way of fetching their version number - and even if they are not forced to have a version number that is comparable numerically (they might use a version string like "v10-0-pre-Build3002") winget cannot accurately keep track of them after the initial install
  6. .EXE Setups may bundle and install dependencies such as MSVC++ Redists or Microsoft ODBC Driver that winget is unaware have been installed (dependency resolution is not possible and neither is keeping track of the patch levels of libraries or dependencies)
  7. .EXE installers are just dirty, supporting them will cause so many issues -> even now, in this super early beta, already there are tons of open issues on this GitHub that all just relate to something weird or unexpected in Nullsoft, Squirrel, InstallShield, FlexeraInstallAnywhere, INNO etc. etc. etc. These programs, once installed, can be uninstalled, tampered with, moved, have their Uninstall-registry-key deleted by anything at any moment. There are no guarantees they can make to winget.

Proposed technical implementation details (optional)

Three options:

1. Slim down the scope of winget-cli and the manifest format.

Only allow .MSIX (or .MSIXBUNDLE) application packages, because these are true "OS-manageable" packages in the sense one would expect from a package manager. This would provide the best possible user experience and communicate to OEMs, developers and partners the importance of MSIX.

2. Only allow .MSIX, .MSIXBUNDLE and "legacy" .MSI

.MSI packages aren't perfect, but they are a lot better than the wide variety of .EXE Setup packagers. They have consistent silent arguments, they are registered by the Windows Installer database and as such can be queried programmatically. They have required information fields, such as Publisher, ProductName, ProductVersion and a GUID for each package so while they permit a Version-String that may be hard to parse we could beef up winget-clis ability to parse "weirdo" non-numeric version strings such as "v10", "2.1.0-pre", "2.1.0-rc2", "3.0-alpha3" etc. past the capabilities of .NET System.Version so we could cover >96% of version numbers. Alternatively the package manifest could include a transcribed, machine readable version number for winget to use internally such as "2.1.0-rc2" -> "2.0.9999.2"

3. Only allow .MSIX and .MSIXBUNDLE packages, but provide re-packaging CI for .EXE and .MSI

The MSIX Packaging Tool can execute .EXE and .MSI Setups and repackage them into .MSIX - most of the time. There could be special "winget-repackager"-as-a-Service tool set up by Microsoft (GitHub Actions/Cloud-based) where any .EXE and .MSI packages submitted for winget will be automatically repackaged into .MSIX by the MSIX Packaging Tool under the hood - and when no errors or warnings occurred, the package is good to go and will be added to the repositories. If the Installer tries something that .MSIX does not support yet, such as installing a Windows service, the packager will error out and refuse. This tool could additionally validate that the produces .MSIX file works by deploying it to itself or a fresh VM/Container and seeing whether that succeeds and whether the package installs.

doctordns commented 4 years ago

Any soft of package manager really would need to handle .EXE packages if said packer manager is to gain traction. Yes, it is a hard problem, but declining to deal with it is unlikely to be helpful, certainly not in the short term. This is not Field of Dreams. :-)

lietu commented 4 years ago

To gain traction the package manager needs to be a "package manager", and just running random .EXEs with unpredictable results is not what a package manager does.

jantari commented 4 years ago

Any soft of package manager really would need to handle .EXE packages if said packer manager is to gain traction. Yes, it is a hard problem, but declining to deal with it is unlikely to be helpful, certainly not in the short term. This is not Field of Dreams. :-)

I disagree. I think investing the time and effort into trying to support .EXE installers - mind you, this is futile in the end - is much better invested in improving the MSIX Packaging Tool so it can more reliably repackage these Installers into TRUE packages. It's already great, I've used it myself. I repackaged Thunderbird to MSIX no problem the day the tool came out and it's even better now. This is not as big of a problem as you think.

This way, winget can deliver value on three fronts:

  1. Software availability today is not bad due to automatic repackaging
  2. Software developers (will) use winget, the fact that the MSIX packaging is easy and open-source helps spread the practice and awareness
  3. Native MSIX packages directly from the developers will increase in popularity

By supprting .EXE this tool slows down the adoption of MSIX

chrpai commented 4 years ago

I've been devoted to this space for 24 years. I love MSI but the industry has totally failed to embrace it and Microsoft has stopped enhancing it. The only way to really solve this is move everyone over to Windows-S and only support MSIX packages. That is never going to happen.

I am passionate about MSI and dislike EXE installers but they are never going to go away. Heck people around here don't want to stop there. They want to be able to extract random ZIP files and somehow "install" them.

lietu commented 4 years ago

You can never make quality software by listening to what everyone wants and complains - you instead figure out the real problem behind their complaints, and find a solution for that, then make it real easy to adopt, and then educate people about it.

  1. Windows has no proper packages, just various unrelated unmanaged ways of installing things. MSIX might be close, but it comes with so many issues that it's not an option - more on that later.
  2. Windows has no centralized repository for software and the store is a massive disaster in many ways.

MacOS solved the problem by bunding all the application files in one standardized container and stuffing it in a known location, that's kind of ok.

Other *nix systems have solved that by providing package managers like rpm, dpkg, and so on, and then have filesystem layout standards (pretty bad ones, hard to figure out what's current, etc., but they exist) their main problem is the massive fragmentation especially within Linux - everyone has to invent their own one.

The packages define e.g.:

Microsoft has tried solving this in some really bad ways, among them the MSIX format - so if you're wondering why MSIX has not taken off, maybe look at the flaws of MSIX and come up with a real solution?

The MSIX containers or whatever you call them don't work in Windows 7 (Edit: apparently not true), which is still very widely in use, so people cannot rely on them, that makes them "just another package" to build, which takes extra effort, and does not bring any benefits - so why would I as a vendor bother with it?

They get installed in some magically protected folder that blows up if I try to go edit any permissions to see what's up, and e.g. blocks screen capture software from capturing them. This is a very big and unexpected "gotcha".

They also apparently mess around with various things in the filesystem and registry and how they're presented to the applications, making it difficult to trace problems, and tweak things to work the way you want them to.

I personally always hope people have NOT used MSIX because of all the nonsense they come with.

The Microsoft Store on the other hand constantly nags at me to log in, then tries to push me to use the Microsoft login to log into my computer - I'm just trying to install software here, don't push your unrelated nonsense to me ok? It's also constantly complaining to me because I'm in Estonia and I like U.S. English, so my region is set to U.S. - apparently it's real important that this setting is followed, and that the Store can not give me a setting to change it for the Store instead of my entire OS. Similarly there's issues with customizing install location, which causes problems when this same model is used on the Xbox app and now instead of installing a couple of small apps of a few MB each, you're installing tens of GB.

Why do you need a centralized repo (or repositories)? To ensure that random developers don't do what random developers do in all too open ecosystems, they release their buggy patch with the same version number as the previous release, and suddenly my "v1.0.0" install blows up on my face, because the package manager for that ecosystem decided that GitHub tags are a reasonable way of accurately determining versions.

It's real sad seeing Microsoft pushing these broken not-even-close-to-what-people-wanted non-solutions and then wondering why nobody adopts them. These problems are not that hard to solve, they have been solved to a much more satisfactory degree by many others, and the solutions are not patented and hidden secrets.

What Windows needs is a package manager and software repositories without any extra caveats and gotchas, something that does exactly what people expect from a package manager, and that works on the systems users actually have so software vendors can support it.

The existing code signing support combined with trustworthy repositories with maybe some automatic security scans, easy way to test it on all different versions of Windows, some clear and easy certification process, a clear storefront without any of the nonsense, and a package manager capable of the fairly commonplace "install these packages", "uninstall those packages", "update all my software", "tell me what I have installed", and "install security updates automatically" -functions would make Windows a much better platform. As a cherry on top, allow users (not Administrators) to install things under their own user account, without modifying the system's protected folders, and you're approaching something decent. If you really want to kick ass with the best of them, make these reproducible and reliable like https://nixos.org and many of us will just stop looking for alternatives to Windows forever.

Then make it real easy to make these packages, if you want to support these old .EXE installers, or .ZIP packages - write converters. "Take your existing FoobarInstaller package as input and this will spit out a package with install and uninstall scripts". Maybe in the future support submission to the package repository in these formats and do the conversion automatically, but only store those real uniform packages. Set up a team to support conversion from various installers to the better universal format (but make sure it actually is better and able to do all the things people need from installers, incl. e.g. install device drivers).

Next up use all that statistical data you're stealing from everyone's Win10 computers (gee, wonder why they don't want to upgrade again) and figure out what applications they are using that aren't supporting the new package structure, and go through every one of the vendors and help them convert. Microsoft is sitting on .. what, $100-$150 billion USD in bank, it's not like they can't do actually this, they just don't want to.

Instead it seems people are building tools to use a GitHub repository with links to .EXEs to download and run them automatically from the CLI, one at a time, without a way to uninstall them, without a way to check what is installed, without a way to install any updates, without a way to ensure the version I've installed today on a new computer is the same it was yesterday on another computer (or maybe there's some magic around the PR system of that GitHub repo that I've not yet seen, who knows).

jantari commented 4 years ago

The MSIX containers or whatever you call them don't work in Windows 7, which is still very widely in use, so people cannot rely on them, that makes them "just another package" to build, which takes extra effort, and does not bring any benefits - so why would I as a vendor bother with it?

.MSIX packages work on Windows 7 SP1 and newer

They get installed in some magically protected folder that blows up if I try to go edit any permissions to see what's up, and e.g. blocks screen capture software from capturing them. This is a very big and unexpected "gotcha".

NTFS permissions are not a new concept. This also applies to ssh-keys and many many other things - if you set the wrong permissions, you may have to restore/repair them to get it working again. icacls /save and icacls /restore makes this task absolutely trivial, so this is a non-issue.

I personally always hope people have NOT used MSIX because of all the nonsense they come with.

Such as?

The Microsoft Store on the other hand constantly nags at me to log in, then tries to push me to use the Microsoft login to log into my computer - I'm just trying to install software here, don't push your unrelated nonsense to me ok? It's also constantly complaining to me because I'm in Estonia and I like U.S. English, so my region is set to U.S. - apparently it's real important that this setting is followed, and that the Store can not give me a setting to change it for the Store instead of my entire OS. Similarly there's issues with customizing install location, which causes problems when this same model is used on the Xbox app and now instead of installing a couple of small apps of a few MB each, you're installing tens of GB.

This discussion about MSIX and winget is completely unrelated to the Windows Store. I get why you may not like the Store, but please realize that that has nothing to do with the topic at hand.

What Windows needs is a package manager and software repositories without any extra caveats and gotchas, something that does exactly what people expect from a package manager, and that works on the systems users actually have so software vendors can support it.

Exactly what I'm arguing for in this issue.

The existing code signing support combined with trustworthy repositories with maybe some automatic security scans, easy way to test it on all different versions of Windows, some clear and easy certification process, a clear storefront without any of the nonsense, and a package manager capable of the fairly commonplace "install these packages", "uninstall those packages", "update all my software", "tell me what I have installed", and "install security updates automatically" -functions would make Windows a much better platform.

Exactly what I'm arguing for in this issue.

As a cherry on top, allow users (not Administrators) to install things under their own user account, without modifying the system's protected folders, and you're approaching something decent.

That is what MSIX does.

f you really want to kick ass with the best of them, make these reproducible and reliable like https://nixos.org and many of us will just stop looking for alternatives to Windows forever.

Not in the scope of this project, winget, but we could get pretty close to this with winget + MSIX + PowerShell DSC / ansible. Honestly probably good enough for most people.

Then make it real easy to make these packages, if you want to support these old .EXE installers, or .ZIP packages - write converters. "Take your existing FoobarInstaller package as input and this will spit out a package with install and uninstall scripts". Maybe in the future support submission to the package repository in these formats and do the conversion automatically, but only store those real uniform packages. Set up a team to support conversion from various installers to the better universal format (but make sure it actually is better and able to do all the things people need from installers, incl. e.g. install device drivers).

Exactly what I'm suggesting in "3. Only allow .MSIX and .MSIXBUNDLE packages, but provide re-packaging CI for .EXE and .MSI". This is also my preferred option by the way.

Next up use all that statistical data you're stealing from everyone's Win10 computers (gee, wonder why they don't want to upgrade again) and figure out what applications they are using that aren't supporting the new package structure, and go through every one of the vendors and help them convert. Microsoft is sitting on .. what, $100-$150 billion USD in bank, it's not like they can't do actually this, they just don't want to.

Maybe a good idea, maybe not, but either way not directly related to this projects scope - remember, let's keep the discussion focused on the winget tool for now.

Instead it seems people are building tools to use a GitHub repository with links to .EXEs to download and run them automatically from the CLI, one at a time, without a way to uninstall them, without a way to check what is installed, without a way to install any updates, without a way to ensure the version I've installed today on a new computer is the same it was yesterday on another computer (or maybe there's some magic around the PR system of that GitHub repo that I've not yet seen, who knows).

Exactly the problem I'm describing in this issue, and the reason I want Microsoft to ditch this approach entirely and focus exclusively on MSIX.

Honestly, it feels a little bit like you basically agree with everything I said but just didn't read it past the title. If we take out some misinformation, like how MSIX supposedly doesn't work on Windows 7 or how you went off topic and talked about the Microsoft Store - then we're basically in near perfect agreement.

lietu commented 4 years ago

The MSIX containers or whatever you call them don't work in Windows 7, which is still very widely in use, so people cannot rely on them, that makes them "just another package" to build, which takes extra effort, and does not bring any benefits - so why would I as a vendor bother with it?

.MSIX packages work on Windows 7 SP1 and newer

Neat, I was misinformed then.

They get installed in some magically protected folder that blows up if I try to go edit any permissions to see what's up, and e.g. blocks screen capture software from capturing them. This is a very big and unexpected "gotcha".

NTFS permissions are not a new concept. This also applies to ssh-keys and many many other things - if you set the wrong permissions, you may have to restore/repair them to get it working again. icacls /save and icacls /restore makes this task absolutely trivial, so this is a non-issue.

No, permissions are not a new concept. Installing applications in magically protected folders with permissions that I as the sole owner and Administrator of the machine have difficulty bypassing, and when I change those permissions the software blows up and refuses to run - these are new concepts, bad, definitely non-trivial, and not non-issues. Additionally these packages are implemented in ways that interfere with other software as mentioned. Try and install OneNote from Microsoft Store and then use OBS Studio to window capture it. You can't. Why? Because someone sprinkled some magic dust around these things. The same issues have applied to games installed via Xbox app, a more common use-case for OBS studio.

I personally always hope people have NOT used MSIX because of all the nonsense they come with.

Such as?

Everything you just called "trivial non-issues".

The Microsoft Store on the other hand constantly nags at me to log in, then tries to push me to use the Microsoft login to log into my computer - I'm just trying to install software here, don't push your unrelated nonsense to me ok? It's also constantly complaining to me because I'm in Estonia and I like U.S. English, so my region is set to U.S. - apparently it's real important that this setting is followed, and that the Store can not give me a setting to change it for the Store instead of my entire OS. Similarly there's issues with customizing install location, which causes problems when this same model is used on the Xbox app and now instead of installing a couple of small apps of a few MB each, you're installing tens of GB.

This discussion about MSIX and winget is completely unrelated to the Windows Store. I get why you may not like the Store, but please realize that that has nothing to do with the topic at hand.

No, they are not unrelated. The Store is Microsoft's best "software repository" for Windows at the moment, and as such is inadequate. For an actual Package Manager to be useful, it requires a software repository.

f you really want to kick ass with the best of them, make these reproducible and reliable like https://nixos.org and many of us will just stop looking for alternatives to Windows forever.

Not in the scope of this project, winget, but we could get pretty close to this with winget + MSIX + PowerShell DSC / ansible. Honestly probably good enough for most people.

Well maybe it should be in the scope. Any Package Manager is only as good as its packages. The package system + package manager can together make this easily possible when properly designed.

Next up use all that statistical data you're stealing from everyone's Win10 computers (gee, wonder why they don't want to upgrade again) and figure out what applications they are using that aren't supporting the new package structure, and go through every one of the vendors and help them convert. Microsoft is sitting on .. what, $100-$150 billion USD in bank, it's not like they can't do actually this, they just don't want to.

Maybe a good idea, maybe not, but either way not directly related to this projects scope - remember, let's keep the discussion focused on the winget tool for now.

It's very much related to the claims about what is required for widespread adoption.

Instead it seems people are building tools to use a GitHub repository with links to .EXEs to download and run them automatically from the CLI, one at a time, without a way to uninstall them, without a way to check what is installed, without a way to install any updates, without a way to ensure the version I've installed today on a new computer is the same it was yesterday on another computer (or maybe there's some magic around the PR system of that GitHub repo that I've not yet seen, who knows).

Exactly the problem I'm describing in this issue, and the reason I want Microsoft to ditch this approach entirely and focus exclusively on MSIX.

Honestly, it feels a little bit like you basically agree with everything I said but just didn't read it past the title. If we take out some misinformation, like how MSIX supposedly doesn't work on Windows 7 or how you went off topic and talked about the Microsoft Store - and we're basically in near perfect agreement.

I read every word in this issue. My replies are not aimed only at you. I do not agree with MSIX being the solution when it literally is the cause of many of my headaches with software on Windows. I would just stop using Windows completely if it was the only way software was distributed.

jantari commented 4 years ago

Try and install OneNote from Microsoft Store and then use OBS Studio to window capture it. You can't. Why? Because someone sprinkled some magic dust around these things. The same issues have applied to games installed via Xbox app, a more common use-case for OBS studio.

That is an issue with OBS and something they have to resolve then. Other open-source screen capturing tools such as Captura can capture OneNote or other "UWP" apps just fine. There is no magic dust, these apps are just built on a different framework - maybe OBS has to add support for that, I get that. But that doesn't mean these apps automatically suck. Also, this is once again unrelated to MSIX. MSIX packages can contain either modern UWP apps, or more traditional Win32 programs - it's just a package format and unrelated to what is packaged inside.

and when I change those permissions the software blows up and refuses to run

Again, a slight adjustment from what you're used to, but by just adding icacls /save to the beginning of your current workflow and then icacls /restore at the end you can continue messing with permissions to your hearts content without ever messing anything up. This is perhaps not what some Windows users are used to, but it's not a reason to refuse MSIX entirely - especially when it comes with such a truckload of benefits in exchange for the tiny inconvenience of running just two commands whenever you need to mess with a file.

No, they are not unrelated. The Store is Microsoft's best "software repository" for Windows at the moment, and as such is inadequate. For an actual Package Manager to be useful, it requires a software repository.

There are two scenarios here:

  1. The winget repositories will stay forever independent of the Microsoft Store, like it is now.

In that case, whatever the Store does or doesn't do is completely unrelated to this discussion.

  1. winget will add support for installing apps from the Microsoft Store

In that case, the Store GUI will not be invoked - the usability complaints you have will not matter because you're using the Store as just a dumb file hosting service for your apps. You'll be using winget, and not the Store - or you could decide to remove the Store as a repository and only install from manifests or your own custom repository etc etc.

lietu commented 4 years ago

I think you just fundamentally do not understand the problem with the permissions - I do not want to "restore them", I've bumped into this issue and found those commands before, the fact that I need to for things to run is wrong, and the way they are set up and how Windows actively tries to hide the installed applications is flat out wrong. My suspicion is that they don't want people prying into all the garbage they are installing on Windows machines without my consent, like Candy Crush, but that's a bit beside the point.

Similarly the way UWP/MS Store/MSIX or something is packaging the software is causing issues. Captura produces pretty bad results when I tested it, so I wouldn't consider it as proof of a solution existing. The OBS community has worked on the problem for years, without finding a reliable solution. This is still beside the point, I was primarily highlighting issues that MSIX and/or Microsoft Store (very related) have caused.

I've really not seen many benefits to MSIX (can't name any right now), just a lot of problems caused by it. People just saying there are ways to restore the permissions and then ignoring all the problems others are having, is a part of the problem with adoption.

Any serious Package Manager must include a serious Package Repository. Right now the only attempt for Microsoft to provide one is the Store, and it's deeply flawed. Using a GitHub repository with links is also deeply flawed as previously mentioned, unless some extraordinary ways to protect end-users from bad developer practices are in place.

Right now what I see is links to files hosted on 3rd party servers (at least with SHA-256 sums, so that's at least a start). I've yet to see any GitHub based software discovery process working well - for one, GitHub itself limits the browsing of repositories once they get large enough, and e.g. https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types is an example of a kind of important repository having their usability reduced because they're relying on GitHub, good luck guessing if it includes the thing you are interested in.

I would be somewhat surprised if today it was not already a problem that some of these registered download links to 3rd party sites would just geoblock downloads from the EU or other areas they don't like. There is likely no checks of the contents of those installers either, no verification that they actually contain somewhat functioning installers and uninstallers. Similarly download speeds are going to be up to the independent vendor's finances and capability to provide with geographically distributed release distribution infrastructure. These kinds of problems could be avoided with proper Package Repositories where you have multiple global mirrors.

Anyway, it seems @jantari and I just fail to agree, and that part of the discussion isn't moving anywhere, so there is little point in continuing this back and forth.

Summa summarum, imo for winget to be a truly great solution, you need 3 pieces working together:

  1. Package Format - it needs to support dependencies, installation, uninstallation, metadata, preferably customizable installation prefixes (i.e. default to C:\Program Files but allow me to choose ANY folder I want anywhere on any of my drives), reasonable levels of proof of integrity and origin - signatures and certificates could be fine. Many smaller software vendors will have trouble buying certificates for software distribution, and safely managing them, but there could be optional levels of certification, and different means of verification, etc. for different needs. This format needs to support literally every single use-case, including installing drivers, libraries, header files for Boost C++, applications, games, and so on, and it needs to be possible to build those packages on non-Windows build machines.

  2. Package Repositories - reliable trusted globally distributed CDN of all the available packages (with a way to add more custom sources, e.g. for internal software or software Microsoft does not approve of for some reason), with clear open API specification, powerful search, and ability to deliver me unchanged versions of software to all my machines when I so choose, at the maximum speed that my internet connection can handle.

  3. Package Manager - tool to install and uninstall the packages in the chosen Package Format, from the enabled Package Repositories, with full scripting and headless remote execution support, install critical security updates to all software automatically, and to update software when asked. Oh, and of course be able to resolve and install dependencies, and uninstall unnecessary packages previously installed as a dependency.

If the decision is that any of the 3 is "not in scope", the end result is going to be sub-par, and will fail to solve the real-world needs.

MathiasMagnus commented 4 years ago

@doctordns With all due respect: upholding the current status quo isn't the remedy we need (but the one we deserve). If MSIX has a shortcoming that hinders adoption, the solution isn't to let the old creep in, but extend MSIX in a way that enables new use cases. (MSIX not being able to install drivers was mentioned as a problem but hey: maybe I don't want some less known screen recorder installing a driver on my system without it going through some screening like the Store or just regular WHQL to make it into Windows Update.) If we want to enable SW installing drivers, perhaps we should, but extend the manifest to not only warn users for Free+ (microtransaction), but also Free+ (installs a driver that can do whatever the hell it wants with your system).

@lietu From your comments I see you revolving around two issues:

In the "not a package manager issue" I already outlined the way even the existing repo of YAML package recepies could be mostly reused to consume legacy stuff, convert them to MSIX and install those. It would bring legacy over to the "brave new world" through a roundtrip that with time will get tedious and ever harder to maintain. It will work in the short term, and will point in a better direction.

If MSIX/Store/UWP/Windows/etc. lacks a crucial feature, the solution isn't to default to compatibility with 20 year old tech. Tainting the new with the old for the sake of a few shortcomings is... meh.

lietu commented 4 years ago

I don't want some less known screen recorder installing a driver on my system without it going through some screening like the Store or just regular WHQL to make it into Windows Update.)

I very much do - with proper warnings and confirmations of course, if they are not certified - and these should be possible to bypass with CLI flags.

Tamper protection is against cracking games

I very much like to decide for myself what I crack - if someone sells me a broken product, I need to have the right to repair it.

I believe the gains outweight the benefits.

You may believe so, I do not. I believe the flaws greatly outweigh any minimal benefits.

Simple as that.

It's not simple, unfortunately the world has a desperate need for perfect developers who never make a mistake and their software always works in every environment, not to mention the company/publisher/whatever in between the developer and me.

I regularly need to replace DLLs for software I run with versions I know to work because their own search gives them the wrong ones. I regularly need to replace an .EXE with a version I want because e.g. I want mods on a game. I regularly need to modify content the developers might not have wanted me to change, but I really don't care if they wanted it or not.

Intentional attempts to block me from doing what I want on my system are unquestionably wrong and unwanted.

I'm sorry if you disagree but it's my computer and my way should be how it works, not the way some rando wanted it to be. It's like next you'll propose that I can't use my firewall to block always-online games and applications, because that's not how the developer wanted it to work.

Some screen recorder not working.

Far from my main concerns, just one of the issues I've mentioned. You might have some cases where AMD's software built using e.g. their own staff's knowledge of undocumented direct hardware access methods might work differently, but that doesn't mean there aren't issues caused by this.

It's there to make it obvious for eg. the Zoom client actually recording which window on your system is in focus while a call is conducted.

In that case it should work like it does on MacOS, there's a screen capture permission that needs to be enabled per application. Now there is no permission, it just gets blocked. And I cannot fix it. Because someone decided that if I try to fix things, the applications blow up. "But don't worry, you can restore permissions to their original state" - which still leaves me with a broken system I cannot do what I want, as the sole owner and administrator of the computer I've bought, built, and installed myself.

Solution may be to actually convert screen recorders to UWP

The way things are, the world needs less UWPs, there is basically nothing I like about them, and they're a huge cause of my daily headaches again.

Oh don't want applications running in the background? Unfortunately that also disables Search Indexing - why? Because it just does.

Don't want Mic access given to random applications by default? That's fine, but we'll also deny the software you want to use from using it, and not tell you why when you scream at the computer wondering why it "can't detect" your perfectly functioning mic.

Tainting the new with the old for the sake of a few shortcomings is... meh.

There are not a few small shortcomings, minor hindrances, some unpleasant looking corners to polish .. there are massive significant headaches caused by MSIX and UWP, that Microsoft refuses to acknowledge, fix or in any way solve. All the while ignoring their users screaming on all their available feedback channels, and then continues to shove these things down their throat as the "solution".

peter-dolkens commented 4 years ago

Hiding installed content. I can understand why the OS insists on that. Tamper protection is against cracking games, injecting malware post-install, etc. I believe the gains outweight the benefits. You can still read installed content, but not modify it. If something is meant to be modified (config files), they should not be part of the app package. Simple as that.

This is a very shallow view of the mods and patches that are in common use today.

Even utility apps benefit from these things: "BetterDiscord". There used to be an MSN one that prevented ping spam and removed ads. The list could go on, but to assume there's never a legitimate reason to want to locate the executables is naive

denelon commented 4 years ago

@jantari great discussion here. I wonder how the community would react if we were to add configuration options that prefer one installer type over another, or give the ability to ignore "non preferred" installer types. This could be a per machine config, or possibly part of policy manager options. We're looking at settings now with #396 Settings command.

Poopooracoocoo commented 4 years ago

304

lietu commented 4 years ago

Imo if I run winget install foo on any machine, the result should not significantly depend on some per-machine configuration (except maybe if there were additional repositories configured with newer versions available or such).

Otherwise people will have to start writing their setup guides something like: "If your winget is using defaults, run ..., if you've disabled .exe installers, run ..." or some such.

driver1998 commented 4 years ago

One potential problem is that MSIX is not like your normal rpm or deb packages, it is a snap/appimage-like approach, with registry and filesystem virtualization.

which means:

And this just scratches the surface of MSIX issues... Just like snap and appimage, developers might need to intentionally adapt to MSIX quirks for things to work properly.

lietu commented 4 years ago

Yep, Snap suffers from these issues - impossible to debug and fix issues that you run into with them (and you do run into them regularly), really confusing system for your average users, and lots and lots of limitations on what can be done with it.

niks255 commented 4 years ago
  • active request of admin privileges is banned (no way to get admin other than right click -> run as admin)

Not really, there's allowElevation capability. Other than that, yeah. You are right.

jantari commented 4 years ago

I agree with you all, MSIX lacks some features and has its inconveniences - however, among its advantages is that the packages are managed by the OS through APIs so its properly compatible with a package management system.

I'm not saying to ban exe Installers from Windows entirely, but let's keep them out of winget because it just doesn't work together. Besides, keep my second suggestion in mind - we could allow MSI in addition to MSIX.That way we get all the support for services, installing to Program Files, mod-and hackability etc. that is needed for some programs whilst keeping the package management aspect at least 90% reliable and sane.

MathiasMagnus commented 4 years ago

@driver1998 & @lietu If the things you listed are blockers to adopting MSIX, then make sure that you voice them to the MSIX & Windows teams respectively. Where there's a will, there's a way. Like @niks255 mentioned: likely someone was direly missing an "allowElevation" capability, so it was introduced. Same is true for everything else.

Note that Windows itself is likely being componentized and I'd bet a larger amount that Windows 10X is making that conversion in small-medium steps. That work is paving the way to likely a dozen non-public extensions required to put an OS together that in time when baked properly will make their way to the public to consume. Win10 likely isn't easy to componentize, many interdependencies (that's why there's the big jump from Server Core to Nano Server both in size and in capabilities, because features cannot be peeled off atomically), hence Win10X shoves the entire legacy part into a container and call it a day. The Win32 container even has it's own kernel separate from the system. 10X is likely being built ground-up.

I've met my share of restrictions both with MSIX and Snap alike. (PowerShell as a Snap package cannot be made a login shell for SSH, likely due to the reparse points mentioned before). I put my +1 on the relevant issue and should Canonical or some distro maintainer (or in the case of Linux, myself) care to implement it, I'd hook it into the kernel to allow that. In the case of Windows, we're at the mercy of MS to make that happen. Bug John Vintzel with stuff like this, especially if you're an ISV or maintainer of a project and say that " is a blocker for my users who want me to adopt MSIX".

mimi89999 commented 4 years ago

If exes where to be dropped I think that winget would become niche software. There are many other 'package managers' that are doing exactly the same that winget is doing currently. They are downloading installers and running them. They are also complaining about installers that are always in some kind interactive or that require unusual CLI parameters or silent config files or many other issues you know. Yet, nobody started a huge repackaging project and they are all applying their workarounds around those installers. GNU/Linux distros have packages, but they have many maintainers that are packaging software. It is rare for developers to create proper packages. They either distribute software in .tar.gzs or in containers like AppImages, Flatpaks, etc. Even if winget or any other Windows 'package manager' manages to gather such a huge community, I'm not sure if they would be able to repackage everything because of license restrictions, DRM likes in installers, etc. Containers are nice, but they have advantages and disadvantages. Isolation is good for security and if packages are not done correctly, but is horrible for app integration/communication, etc. and you end up with Android/iOS but with a desktop UI.

I also wounder when/if offline installations in images will be supported. With Debian I can just mmdebstrap and entire system or mount and chroot and apt install anything. Windows PE never even had something like chroot nor msiexec. You can only sideload with dism. I even published some of our deployment scripts (https://github.com/xivlo-sysadmins/deployment-scripts).

Windows is used in most enterprise and education environments. Unless I don't know something (yes, I also know MSI deployments through GPO), app deployment is a nightmare everywhere. I think that many sysadmins would be interested in making that better, so there would be a huge community, but nothing like that happened. I think that we should think about all the reasons for that.

doctordns commented 4 years ago

.EXE installers, like .ZIP installation packages are not going away any time soon. They work, they install software, and they work around the world. I can't imagine that the winget team is going to change that, at least in the short term. So either the winget team embrace what the rest of the world is doing or WInget just becomes a developer power toy. Useful of course, but without exe and zip support, not enterprise-ready.

I'd like to see manifests that enable winget to run an installation exe. A manifest tells winget where to get the exe, what user to run the installation exe, what shortcuts should get create and what to do after the exe runs (ie run some app). Likewise for a zip package, the manifest can tell winget where to get the zip from, where to install it into, etc. Make the manifest useful for all steps of installation for the kinds of packages encountered in the enterprise.

lietu commented 4 years ago

You do understand that people would package their own software for Linux if there wasn't 200 distros out there, and 4 major versions of each of them "active" at any given time? ... kind of like they do for Mac, though even on Mac it's not perfect, which is why you end up with homebrew and pals.

Windows has a very different position from Linux, and Microsoft is clearly in a position to bring a new solution out there that gets rid of all the nonsense. Sure, you can keep delivering .exe installers if you want, or implement the standard format package (once a real good package format is created, I'm far from convinced that MSIX is any kind of a solution to the problems) and have it easily available for everyone without you having to figure out installers.

This would require

There are already many solutions for running remote .exe installers, we don't need another one. If winget is trying to deliver any actual value, it needs to solve problems that haven't been solved before, or at least do a better job at solving an existing problem compared to the other options.

Right now winget is not doing anything new. What it's doing is done MUCH worse than by the other options. There is also no clear plan for doing things better than how others are already doing it.

The prognosis would be pretty different if winget gave real clear value, having your packages easily discoverable to users in a marketplace, easy installation/uninstallation and dependency resolution, one universal package format that just works always, with various repositories - some with highly trustworthy software signed by Microsoft (assuming MS keeps their signing keys safe and doesn't just hand them out to various well known bad actors), some with trustworthy software from others that sign them themselves, and 3rd party repositories for trustworthy enough things - e.g. building open source software & libraries and signing them.

These are things that others cannot really do easily, Microsoft has the weight, cash and just sheer volume of developers to push others to a good solution. Or they can keep fiddling around building an inferior runner for existing flawed installers.

Nobody needs to get rid of .EXE and .ZIP distributions as they are right now, if people want to keep using those, Chocolatey, Scoop, and tools like Salt Stack can serve them just as well. What they need is a new solution, instead of trying to support every legacy thing ever invented. This new solution could then develop into something that everyone will WANT to use in the future.

mimi89999 commented 4 years ago

I fear that if you have packages without package slots and without them being centrally managed, you will have never ending dependency conflicts. That's something that probably can't be resolved without a distro.

There are already many solutions for running remote .exe installers, we don't need another one. If winget is trying to deliver any actual value, it needs to solve problems that haven't been solved before, or at least do a better job at solving an existing problem compared to the other options.

Right now winget is not doing anything new. What it's doing is done MUCH worse than by the other options. There is also no clear plan for doing things better than how others are already doing it.

I can't agree more. They don't even have automatic registry updates.

driver1998 commented 4 years ago

Actually we have a solution available in our hands: MSIX, but MSIX on anything but Windows 10, aka MSIX Core. It gets rid of the Centennial virtualization and protection stuff, but still keep the benefits like declarative install and dependency management. So you get a simple “extract, register and go” package like rpm or deb. But it is not supported on Windows 10, that should just be an artifical limitation.

mimi89999 commented 4 years ago

Let's ask app developers if they could provide msix packages for their apps and even contribute manifests and add assembling them to their build for FOSS apps. Let's see if that goes well or it there will be any major issues.

I think it's the best way to go instead of continuing this purely theoretical debate.

Poopooracoocoo commented 4 years ago

For the most part people have said that they have no reason to switch to MSIX. In the case of Calibre, I think the developer said that the MSI (un)installer is already clean. Not even MS or its partners (Spotify, Chrome, Adobe, etc) are using MSIX. Speaking of hypocrisy, Skype recently switched from RNW to Electron. facedesk\ so i don't think we can convince devs.

MarkSFrancis commented 4 years ago

Is there any reason for why the package manager couldn't support both the uncontrolled ZIP and EXE packages, and support the isolated, modern MSIX packages too? It'd be great if winget printed out a warning in the terminal when I try to install a package that uses an uncontrolled format.

For example - if I want to install pgAdmin, I could open the terminal, type winget install postgresql, and then have it print a warning that this installer is an EXE, but let me continue if I want to. The developers over at pgAdmin are then free, and encouraged, to upgrade to MSIX, but if it's not possible (or not a priority right now), then they don't have to, and the user can still use winget in the meantime.

lietu commented 4 years ago

Supporting extra things is more work. If these "uncontrolled" ZIP and EXE installers are not packages, why would a "package manager" allow you to install them? Currently the "package manager" does not support any kind of package management, and the Windows world really needs one. This proposal seems to me to be mostly about getting the "package management" part done and leaving the other non-package management parts for others to worry about.

There are existing tools to install them that do them better both for individuals and organizations than win-get, incl. Chocolatey, Salt Stack, and others.

SamB commented 4 years ago

I am passionate about MSI and dislike EXE installers but they are never going to go away. Heck people around here don't want to stop there. They want to be able to extract random ZIP files and somehow "install" them.

I'm not sure what the problem is with ZIP? It's not going to go behind winget's back like EXEs do ...

jantari commented 4 years ago

I can tell you what the problem with ZIP is, as I've had to create a deployment package for neovim not long ago so that I can roll it out to our Windows Servers at work: image

This is a screenshot of the steps I had to create during the neovim deployment package that it has to go through to install the software from a ZIP file. It's messy, and some of these steps contain small-sized scripts behind them.

Installing a software from a ZIP package is not as easy as just expanding the archive, you have to wrap it in what basically becomes an installer anyway.

denelon commented 3 years ago

I'm converting this to a discussion. At present, we do not plan on eliminating support for .exe installers. I believe we will be enabling support for filtering packages by installer type. Once that is done, we could honor settings that prefer or require not installing packages with "InstallerType: .exe". There may be other feature requests that come from this discussion. We could also look at Group Policy settings that also prevent support for .exe installers.