Squirrel.Windows - Reboot Proposal

shiftkey commented 5 years ago

I wanted to provide an update to this situation that will hopefully address some concerns.

First off, many thanks to @anaisbetts for her hard work over the years on creating and maintaining Squirrel.Windows. As issues like #1466 and #1469 show, there’s a lot of love for the project and a lot of interest in keeping this going.

For those who don’t know me, I work at GitHub and have been around since the early days of Squirrel.Windows. I’m not an active contributor, but I’m a happy user of it due to its adoption in the Electron ecosystem and want to see it continue.

I had a chat with @robmen yesterday about how we might be able to help keep things going, and we agreed upon some short-term goals. These goals are rather modest, but they feel achievable over the coming weeks and months:

Review the outstanding open issues and get the issue tracker into a state that reflects the current situation
Hold a regular fortnightly meeting to discuss the project with contributors and users
Investigate whether there is some low-hanging fruit that can be worked on
Get the project to a place where we could potentially ship an update

We’re only two people with plenty of other things on our plate currently, so we’re looking for others to help with this work to build a more vibrant community around the project.

If this sounds like something you’d like to get involved with, please comment in this thread so we can gauge the interest.

If you have any questions, feel free to comment here - or you can reach out privately to shiftkey@github.com if you don’t feel comfortable asking here.

patrickklaeren commented 5 years ago

I was saddened to see Squirrel.Windows be deprecated - just last month we managed to migrate most of our work apps to use Squirrel after having started using it late 2018. We've had a good experience. Upon the announcement me and a friend embarked on building a "spiritual-successor", if you will, that caters for our needs, both in "enterprise" and for personal applications.

I'm glad to see Squirrel.Windows still has life left and would love to contribute where possible if you elaborate what help you're after.

shiftkey commented 5 years ago

I'm glad to see Squirrel.Windows still has life left and would love to contribute where possible if you elaborate what help you're after.

The first thing that comes to mind is taking stock of the issue tracker. If anyone is able to spare some time, I'd love help with reviewing the open issues and seeing what we can close out.

Things to look out for in open issues, and my suggested action:

Has the issue been resolved?
- Helpful: Comment in the issue that you think this can be closed and mention me
Is the issue not clear? Is something missing that you think would help out?
- Helpful: Ask a question that would help with understanding the problem and ensure that you @-mention the submitter so they're aware of it
Is this an old problem or a problem with an old version of the library?
- Helpful: Ask the submitter if they are still able to reproduce the issue with the latest version of the library.
Is the issue asking a general question something the maintainers cannot really help with?
- Helpful: Ask the submitter if they are fine to close out the issue, and ensure that you @-mention the submitter so they're aware of it

I can come through and help with sorting/closing issues, but having many eyes scanning over the open issues here would be a great help to get things moving...

anaisbetts commented 5 years ago

I'm glad to see that some folx found Squirrel so useful that they want to revive it :) I wanted to take a moment and share A few Guiding Principles that have served Squirrel.Windows well over the years.

Be as Dumb As Possible - since the beginning, Squirrel.Windows has always tried to do literally the simplest, least clever thing that could possibly work. Simple things are great, because it means that people can build their own more complicated / niche things on top of it that only make sense for their scenario. Let Developers who Use Squirrel be the clever ones.

Another extremely good reason to avoid complexity is, reliably installing and updating software on Windows is already walking a tightrope on a windy day while people throw tomatoes at you, complexity will end up creating more failed updates and users getting left behind. Which leads me to....
Be extremely Conservative - Squirrel.Windows routinely turns down features, moreso than other projects that I've maintained, because breaking the updater is Catastrophic. Slack broke the Mac updater shortly before moving to the Electron-based version of the app, and it took them years to remediate this mistake, it sucks really bad. Anything that touches the core path of updating the app should start off in your head with -5 votes. It sucks to turn people down, especially when they've already put in some work for their idea, but complexity adds up, and every feature you add will make it harder to feel confident that you haven't broken the updater.
Stick to The Principles and Say No - If you buy #2 above, you're already sold on this, but for this project in particular, it is so important to Say No. People want Squirrel.Windows to be a lot of different things, and Squirrel.Windows has stayed a reliable product because it is willing to tell people, "We just don't solve that problem. Sorry." Ideas like making Squirrel an NT Service to do updating would definitely mean it would apply to more scenarios, but it would also make Squirrel much much more complicated and prone to leaving users stranded.
Focus on the User Experience - Another reason to Say No is because Squirrel.Windows has drawn a very strong Line In The Sand around user experience. The goal of an installer is, from the second the user double-clicks Setup.exe, we want to get the App in front of them, as fast as possible, and do nothing else. When the user uninstalls? Our goal is to remove ourselves, as fast as possible - that's it!

And in terms of updating, our goal is to be like Chrome - you open it and it's the latest version. No prompts, no UAC dialogs, no Wizards or Progress Bars. Every time we show an update progress bar, we say to the user, "What you're doing isn't important. Look at Me instead". Any feature that tries to do this should be met with "That's Not What We Do Here"

Hope that helps explain some of the Zen of Squirrel.Windows. Excited to see what folx come up with in the future!

peters commented 5 years ago

Thanks for all the hard work you put into this project @anaisbetts 🎉

frederikolsen88 commented 5 years ago

Yes, thank you for your work, @anaisbetts, and the best of luck on your journey!

And thank you for taking control, @shiftkey. What a relief 😄

TonyValenti commented 5 years ago

Hi All, I just saw that aparently squirrel is deprecated? If so, was there an announcement?

I'm very interested in keeping the project alive and well, especially with .NET Core 3.0 coming out soon, however, a lot of the way that code was written makes me feel fairly uncomfortable. I'm more on board with a rewrite over maintenance. Thoughts?

shiftkey commented 5 years ago

@TonyValenti

especially with .NET Core 3.0 coming out soon,

This is an interesting potential audience, and I'd love to hear more thoughts about this and how Squirrel.Windows might affect them.

however, a lot of the way that code was written makes me feel fairly uncomfortable.

Can you elaborate on this? I'd love to understand these concerns and how we can mitigate them.

I'm more on board with a rewrite over maintenance. Thoughts?

There is a lot of existing usage of Squirrel.Windows in the wild currently, and I'm very reluctant to contemplate rewriting things (and introducing breaking changes and potentially breaking these users) without more explicit goals and a plan to go about it.

TonyValenti commented 5 years ago

@shiftkey -

With .NET core 3.0, desktop apps are now a supported scenario and I think that Squirrel.Windows makes a ton of sense for it. If you deploy a desktop app, you're going to want a way to update it. Also, I believe that the self-contained nature of .NET core is great for Squirrel.
Here's a really good example. Take a look at : https://github.com/Squirrel/Squirrel.Windows/blob/master/src/Squirrel/Utility.cs

There is a ton of code in there that isn't documented and should really be broken out into multiple classes. Additionally, you'll see things like Tuple returns and methods like this which make me shudder:

  public static Task ForEachAsync<T>(this IEnumerable<T> source, Func<T, Task> body, int degreeOfParallelism = 4)
        {
            return Task.WhenAll(
                from partition in Partitioner.Create(source).GetPartitions(degreeOfParallelism)
                select Task.Run(async () => {
                    using (partition)
                        while (partition.MoveNext())
                            await body(partition.Current);
                }));
        }

When I look at the code in general, it works but it needs some good love to start looking maintainable.

robmen commented 5 years ago

Yeah, we can definitely improve Squirrel.Windows but we don't need to start with a "rewrite". Small, iterative steps that bring along everyone to a better and better experience will work out well. A big bang rewrite would require significant backwards compatiblity verification and not allow us to see improvement along the way.

aaronstine commented 5 years ago

Hey guys, my company (Identifi) recently adopted squirrel and wouldn't mind investing in its future. Our current product plan keeps our devs busy for the next few months. But after that, we could free up some folks to contribute.

anaisbetts commented 5 years ago

@TonyValenti Squirrel.Windows is an Ineffable Paragon of Software Engineering Elegance and I will hear nothing to the contrary! 😅😅

stefanolson commented 5 years ago

I think the nice thing about squrirrel is how well it has worked. We've got a couple of thousand users using it on a daily basis and really we have quite minimal numbers of problems. Certainly better than anything else out there so well worth continuing with I think. I'm still running version 1.4.4. I did try and upgrade at one point but it started signing all the DLLs and taking ages to do the build, so I gave up and went back to 1.4.4. But we are now having a couple of problems with 1.4.4 so I will be downloading the code and submitting some changes if I can figure out the problem we are having.

MeikTranel commented 5 years ago

I would be more than willing to help migrate the project files to multitargeted projects where we can move Squirrel to .NET Core while keeping support for full framework. I must say... theres some complex stuff living in the project files. Maybe the current maintainers should make an effort to improve architecture documentation - the use cases are fairly well documented from what it looks like, but i don't see much information regarding how all the pieces fit together. This is not a dig on the current maintainers... it's just me being interested to contribute but being a little overwhelmed with the architecture... and i assume we can all agree that more community participation is a factor that could really help this kind of project.

Keboo commented 5 years ago

I too would be willing to help out on keeping this project alive and moving forward.

shiftkey commented 5 years ago

Just wanted to give a shout out to @Thieum who has been investigating old issues and confirming things which can be closed out as being resolved! Thanks for your help!

robmen commented 5 years ago

@MeikTranel I am slowly working through the build process, simplifying as I can but primarily working towards a fully automated build and release process.

TonyValenti commented 5 years ago

@MeikTranel @robmen - I'm working through a fairly substantial upgrade/cleanup of the squirrel.windows DLL.

MeikTranel commented 5 years ago

@MeikTranel I am slowly working through the build process, simplifying as I can but primarily working towards a fully automated build and release process.

That'll definitely help me. Seeing how things move through the build ending up at the consumer is exactly the information i need regarding project file cleanup/migration.

I'm in a super tight spot in my education right now with finals coming up next week, but i'm hoping to join the rest of the pack starting next thursday. There's a number of open source things i've got going, but currently on ice because of this reason.

@TonyValenti That's great, but i honestly feel like the more critical pillar of the codebase is the C++ stuff. There's some self-titled "janky" unzip code here, some "stub-setup" there... Maybe we should focus first on trimming things out which should not be Squirrel's primary domain. Without knowing to much of the architecture i'd say unzipping should be something we let .NET or any library do. It just feels to me that it shouldn't be my job to look at decompression code.

TonyValenti commented 5 years ago

@MeikTranel I think that's a good idea. I'm targeting the current .NET side of things because it aligns with a near-term need for our business.

Regarding the C++ side of things, I think it would be worth exploring to see if C++ can be removed from the stack and entirely replaced with .NET Core 3.0.

robmen commented 5 years ago

@TonyValenti arguably, Squirrel would benefit if everything was written in native code. That would remove the need for multi-megabyte external dependencies (.NET Framework is fortunately normally pre-installed so .NET Framework is usually not a distribution burden). .NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

Now installing .NET Core 3.0 runtime as a dependency (like .NET Framework today) will be an important feature in the not too distant future. But making Squirrel depend on it will have to take into account the whole of the Squirrel ecosystem.

TT39 commented 5 years ago

I would be willing to help out. Squirrel is really useful software.

MeikTranel commented 5 years ago

@robmen i agree 100%. We should use .NET Framework where it can be useful. .NET Standard for the referenced libraries and .NETCore/.NETFramework (to be able to use dotnet cli for development and consumption) for the build time components. As for Native C++ i don't see a reason why we should forcibly remove it from the stack. As long as we don't try to solve problems that have been solved already by the dependencies we already pull in the only real reasons to remove it would be consistency of the stack and the potential for xplat support; both of which are really weak arguments at the moment i reckon.

anaisbetts commented 5 years ago

As for Native C++ i don't see a reason why we should forcibly remove it from the stack

You can't remove the C++ code because you can't guarantee that the version of .NET that you want is installed on the machine. The C++ code exists to install the .NET Framework.

.NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

This is actually far worse, it's hundreds of megabytes right now. Mono's linker is much better at the moment, you can get Hello World down to 4MB, but you're still not beating the C++ bootstrapper.

Thieum commented 5 years ago

.NET Core would add tens of megabytes and would dwarf many of the applications distributed via Squirrel.

Would CoreRT, when ready, be an option to adress that issue?

anaisbetts commented 5 years ago

@Thieum afaik at the moment, CoreRT is similar or slightly bigger than the Mono Linker

MeikTranel commented 5 years ago

@anaisbetts

The C++ code exists to install the .NET Framework.

Does it really install .NET Framework? Which one?

This is actually far worse, it's hundreds of megabytes at the moment. Mono's linker is much better at the moment, you can get Hello World down to 4MB, but you're still not beating the C++ bootstrapper.

That's what i meant when i said we shouldn't try to forcibly remove it from the stack for seemingly no major reasons besides convenience - especially considering what you said about the downsides of the alternatives.

@Thieum

Would CoreRT, when ready, be an option to adress that issue?

We should be looking into that, but i don't expect it to be a magic switch as of right now. When .NET Core 3.0 is released, we should investigate this - the .NET team is doing lots and lots of work regarding this, AOT compiling and tree-shaking. I don't think its worth the effort right now.

robmen commented 5 years ago

CoreRT may be an answer but chances are it'll still be in the MB range. To be clear, there is a very real possibility to get Squirrel to add no more than 128K-512Kb overhead to the overall installation. Knowing that is a lower bound, I look critically at frameworks that add overhead measured in MB.

anaisbetts commented 5 years ago

@robmen Agree, though I would try to balance the concerns of disk space vs the ability for folx to contribute - it might be worth the extra MBs to get the maintenance help (and tbh, while network bandwidth is definitely still a Thing in some places, Internet speed is A Lot Faster than it used to be)

robmen commented 5 years ago

@anaisbetts Agreed. Knowing the "lower bound" allows us to measure the cost/benefit of each dependency.

amaitland commented 5 years ago

Issues

Things to look out for in open issues, and my suggested action:

@shiftkey I'll propose a different action plan for the issues, setup Probot Stale, close everything that's older than a set date, maybe 180 days? Set the initial bot message to something like In the interest of bringing order to the repository old issues will be automatically closed unless confirmation the issue is still valid is received within x days. Set the auto close to maybe 28 days or something.

Branches

Cleanup all the stale branches, deleting the merged ones should be a quick win https://github.com/Squirrel/Squirrel.Windows/branches/stale

It appears that master and develop are the two primary branches used, looking at https://github.com/Squirrel/Squirrel.Windows/blob/develop/docs/contributing/contributing.md and I don't see any clear guidance on the branch structure for the project, I think if the plan going forward is to have automated releases then a more clearly defined structure would be of benefit. Possibly just a master and stable branch system would be enough or my preference is branch per major release where they are prefixed with something like release so a generic Branch Protection Rule can be applied to them all. e.g. release/1.9

EditorConfig

I think adding a .editorconfig file now before you get really going with new development is worthwhile, keep the codebase consistent

Issue Templates

Add the standard bug report, feature request issues templates. in an effort to improve the quality of the opened issues.

Bots

There are a few bots that might be worth considering in the effort to maintain this repository

https://github.com/rsarky/helpr https://github.com/probot/duplicate-issues https://github.com/probot/no-response

shiftkey commented 5 years ago

@amaitland thanks for the suggestions:

Re: issues, I currently I don't want to bulk close issues because I still see some value in reviewing and interacting with them:

helps me to understand how others are using the project
helps to illustrate the sorts of problems users have encountered (whether fixed or outstanding)
helps identify gaps in documentation or opportunities to improve current docs
helps signal interesting areas to dive into about improving the project

Bulk closing things out to me feels like throwing away this information. We've already been able to close out over 150 issues and review over 110 more since I left that initial comment, which I think is some great progress (thanks to everyone else who has helped out).

As things settle down and we get into the harder stuff on the issue tracker, I'll revisit this and see what things might be worth automating.

Re: branches - I cleaned up most of those stale ones just now - there's a couple of interesting ones that I'll keep handy and poke at whether they might be useful after rebasing.

I don't see any clear guidance on the branch structure for the project, I think if the plan going forward is to have automated releases then a more clearly defined structure would be of benefit

This is something @robmen and I have been chatting about. I've opened #1477 to track getting some docs together to explain what these changes mean in terms of PRs and releases.

Re: editorconfig - I've opened #1478 to track this, if someone would like to put one together

Re: issue templates - I've opened #1479 to remind myself to add this

gojanpaolo commented 5 years ago

Hey guys, just wanted to let you know that I'm interested in helping keep Squirrel.Windows alive. I primarily develop WPF apps using WiX and Squirrel.Windows for deployment solutions. I started sorting out the issues, will start submitting PRs later to fix some of it :) Cheers!

Horb commented 5 years ago

Thanks to @anaisbetts for all his work on Squirrel!

We use Squirrel for several of our applications and we intend to keep using and passing changes to upstream in the future.

damieng commented 5 years ago

I spent a fair bit of time looking into what a streamlined Squirrel next-gen might look like and agree with @anaisbetts that it really should be as dumb as can be and do the smallest amount possible.

For me that meant a Squirrel-built .exe should be able to:

Unpack the contents to a new folder
Add an entry into add/remove programs that will allow removal
Remove everything if launched from add/remove
Run a developer-specified command line to perform post-install steps
Run a developer-specified command line to perform pre-uninstall steps

I'd personally like to see app icons, auto-run, pining, file associations etc. entirely handed externally. In Atom we moved most of this logic to atom itself where the app could offer users a rich ui and options for performing these steps without having to learn different tooling or languages. (It also means you can more easily preserve/honor options like whether the files are associated or not)

I also thought that the unpacking should be pretty agnostic. I looked into basically using a 7z self-extractor for that part but the idea that Squirrel next-gen wouldn't care what compression format you are using is appealing. 7z max-compression files were significantly smaller than zip and while not as small as the Squirrel delta system removed much of that complexity.

TonyValenti commented 5 years ago

@damieng - I wanted to let you know I'm hard at work on a spiritual successor to Squirrel. I'm starting with the easy things and moving to the harder ones (like self-installing).

Miggleness commented 5 years ago

Would moving Squirrel under .NET Foundation help?

erwinwolff commented 5 years ago

I would love to maintain Squirrel. My product needs to survive and it is by far the easiest installer I used to date. I would hate it to see it go down in flames.

isatahiri commented 5 years ago

Any status on Squirrel futur ? I just discovered and tested it. I really like it and want to know if we can rely on new project ?

luis-fss commented 5 years ago

So, @shiftkey how have you been?

I didn't mean to be rude, but could you give us any sign that you're still interested in maintain Squirrel.Windows?

I noticed that issues have dropped considerably in recent months, but some new ones have come up and it's been a long time since you commented here.

Before anyone judges me, I'm just a pragraming student and have some personal (non-commercial) projects that depend on Squirrel.Windows. Unfortunately (yet) I do not have the knowledge to contribute with code updates.

I'm really hoping Squirrel.Windows can move on. Thanks for everything and God bless you.

robmen commented 5 years ago

I can't speak for @shiftkey but I still have great plans for Squirrel.

The timing of the project transition was far from ideal (it wasn't planned, obviously :smiley:). It was just before summer here. Summer means time spent on family vacations, traveling to kid's soccer tournaments, BBQs with friends, hiking and other fun stuff outside. It is not a great time to pick up an additional codebase and start making immediate progress. :smile:

But the rain is coming in a couple months and then there will be more time spent at the keyboard.

Before the summer started, I was able to massively improve the build process. Others have been doing a fantastic job triaging issues. So, we'll be in fantastic shape to roll forward.

robmen commented 5 years ago

@damieng I think there need to be "some batteries" included (auto-update, app icons, shortcuts, etc) in Squirrel. But I also believe that factoring Squirrel so you can use it without any of those batteries is a good goal as well. As I start working on v2.0 (mostly aligned with the notes @anaisbetts forwarded me before summer), I'll keep this design goal in mind.

damieng commented 5 years ago

@robmen the problem is everyone wants to do something a little different around that so you end up with a ton of customization options to deal with it which becomes a ton of work and a minefield.

Given your app comes with a full programming language it's so much easier to do exactly what you want there and just keep the installer down to install/uninstall/update.

damieng commented 5 years ago

This also allows the app to include add/remove from startup as a config option/setting, create icon on desktop/quicklaunch/start in case they removed it, add/remove exe from path....

robmen commented 5 years ago

@damieng A lot of people want the bare minimum (shortcut, app icon, ARP, auto-updates) taken care of for them. Squirrel needs to provide a solid implementation of those basics. At the same time, rather than expose lots of dials and knobs to customize Squirrel, the design will allow a developer to use Squirrel with none of the basics enabled. Then a developer can provide implementations for those basics as needed. That design provides most people what they want, avoids implementing tons of customizations in Squirrel, and does not block completely custom/advanced scenarios.

damieng commented 5 years ago

Hey, you're the one working on it! I will now bow out.

luis-fss commented 5 years ago

What a fantastic news @robmen, thank you so much for sharing it with us. I will look forward to it. After all, the Squirrel we have today, is already a magnificent tool and I would be very happy just with bug fixes. New features alone is beyond what I expected, I am really happy.

TonyValenti commented 5 years ago

Hi All, In case you're interested, I've been revamping a bunch of the code: https://github.com/mediatedcommunications/badger

I don't have the installing/packaging done but I do have a lot of the framework for other things.

mihaimyh commented 5 years ago

Any plans to support .net-core?

robmen commented 5 years ago

@mihaimyh can you open a new issue with more details about the exacts support for .NET Core you'd like to see. Are you looking for it to install the redist instead of .NET Framework? If so (or if not :smile: ) say so in the new issue.

Thieum commented 5 years ago

@mihaimyh see also #1374 for existing discussion

Squirrel / Squirrel.Windows