RFC: Remove archive download option

chillu commented 5 years ago

Overview

SilverStripe started as an "archive" download before composer was a thing. Since then, we've spent a lot of time making composer install as smooth as possible, including writing our own recipe-plugin for it. While you can still "install" SilverStripe from an archive, your website becomes practically unmaintainable without composer update since modules don't provide their own individual archive downloads. So in effect, everyone using SilverStripe is already using composer.

The archive download on silverstripe.org complicates our release process (S3 buckets, additional cow release steps). It has been a time sink (e.g. path lengths in tar.gz), and there's more outstanding issues (e.g. 300MB file size due to .git inclusion, builds for PHP 5.x). Rather than fixing those, I propose that we remove the archive download option.

There's likely some edge case uses which rely on archive downloads for some scripted purposes, which would break when we stop publishing those archives. But I highly doubt that they're part of production deployment workflows, given the existing limitations outlined above.

Validation

I've done some research on this based on our recent drive to remove the installer.

Archive download stats: According to our Google Analytics custom event setup on silverstripe.org/download, we have around 300 downloads per month. This excludes anyone who disables Google Analytics tracking during downloads, or uses other tools to download it (e.g. curl).
Packagist install stats: We have around 2000 packagist installs of silverstripe/installer. Note that roughly speaking, ~10% (~225) of these installs are based on our ci builds - ~25 builds times 9 jobs.
Download page views: We've had 1800 page views on silverstripe.org/download in the last 30 days, so with ~300 archive downloads that's a ~16% conversion rate, with the remainder either choosing not to download, or using composer.
Laravel seems to be doing fine without a download as well. Drupal has one because they're far behind in composer. CraftCMS has an archive download as an alternative to the default composer instructions.

Notes

I've confirmed that our CI counts towards those stats, see notify-on-install which is enabled by default. I can't see how projects would count towards those stats after the initial installation, even if they keep the silverstripe/installer name in their root composer.json. Tracked this down to InstallationManager->notifyInstalls() in the composer source code. When running composer create-project silverstripe/installer installer; cd installer; git init .; git add .; composer update, I don't get another stat count. Since custom projects would be "installed" via git clone <url>; composer install rather than composer create-project, they wouldn't count towards install stats either (different repo URL). I've confirmed that with some debug output in InstallationManager on composer install, here's the full payload which is sent to packagist.org/downloads/. Note that it doesn't include silverstripe/installer:

{"downloads":[{"name":"composer\/installers","version":"1.7.0.0"},{"name":"silverstripe\/recipe-plugin","version":"1.3.0.0"},{"name":"silverstripe\/vendor-plugin","version":"1.4.0.0"},{"name":"bramus\/ansi-php","version":"3.0.2.0"},{"name":"bramus\/monolog-colored-line-formatter","version":"2.0.3.0"},{"name":"composer\/ca-bundle","version":"1.2.4.0"},{"name":"embed\/embed","version":"3.4.1.0"},{"name":"ralouphie\/getallheaders","version":"3.0.3.0"},{"name":"psr\/http-message","version":"1.0.1.0"},{"name":"guzzlehttp\/psr7","version":"1.6.1.0"},{"name":"intervention\/image","version":"2.5.0.0"},{"name":"league\/csv","version":"8.2.3.0"},{"name":"league\/flysystem","version":"1.0.55.0"},{"name":"m1\/env","version":"2.1.2.0"},{"name":"marcj\/topsort","version":"1.1.0.0"},{"name":"psr\/log","version":"1.1.0.0"},{"name":"monolog\/monolog","version":"1.24.0.0"},{"name":"nikic\/php-parser","version":"4.2.4.0"},{"name":"paragonie\/random_compat","version":"2.0.18.0"},{"name":"psr\/container","version":"1.0.0.0"},{"name":"symfony\/polyfill-ctype","version":"1.12.0.0"},{"name":"symfony\/yaml","version":"3.4.31.0"},{"name":"symfony\/polyfill-mbstring","version":"1.12.0.0"},{"name":"symfony\/translation","version":"2.8.50.0"},{"name":"symfony\/filesystem","version":"4.3.4.0"},{"name":"symfony\/config","version":"3.4.31.0"},{"name":"symfony\/polyfill-apcu","version":"1.12.0.0"},{"name":"psr\/simple-cache","version":"1.0.1.0"},{"name":"psr\/cache","version":"1.0.1.0"},{"name":"symfony\/cache","version":"3.4.31.0"},{"name":"swiftmailer\/swiftmailer","version":"5.4.12.0"},{"name":"symfony\/finder","version":"3.4.31.0"},{"name":"silverstripe\/config","version":"1.0.16.0"},{"name":"silverstripe\/framework","version":"4.4.3.0"},{"name":"silverstripe\/assets","version":"1.4.3.0"},{"name":"silverstripe-themes\/simple","version":"3.2.0.0"},{"name":"silverstripe\/versioned","version":"1.4.2.0"},{"name":"webonyx\/graphql-php","version":"0.12.6.0"},{"name":"silverstripe\/graphql","version":"3.1.4.0"},{"name":"silverstripe\/admin","version":"1.4.3.0"},{"name":"silverstripe\/versioned-admin","version":"1.2.3.0"},{"name":"silverstripe\/siteconfig","version":"4.4.1.0"},{"name":"silverstripe\/reports","version":"4.4.1.0"},{"name":"silverstripe\/recipe-core","version":"4.4.3.0"},{"name":"silverstripe\/campaign-admin","version":"1.4.1.0"},{"name":"silverstripe\/cms","version":"4.4.3.0"},{"name":"silverstripe\/errorpage","version":"1.4.1.0"},{"name":"silverstripe\/asset-admin","version":"1.4.2.0"},{"name":"silverstripe\/recipe-cms","version":"4.4.3.0"},{"name":"webmozart\/assert","version":"1.5.0.0"},{"name":"phpdocumentor\/reflection-common","version":"1.0.1.0"},{"name":"phpdocumentor\/type-resolver","version":"0.4.0.0"},{"name":"phpdocumentor\/reflection-docblock","version":"4.3.1.0"},{"name":"phpunit\/php-token-stream","version":"2.0.2.0"},{"name":"sebastian\/version","version":"2.0.1.0"},{"name":"sebastian\/resource-operations","version":"1.0.0.0"},{"name":"sebastian\/recursion-context","version":"2.0.0.0"},{"name":"sebastian\/object-enumerator","version":"2.0.1.0"},{"name":"sebastian\/global-state","version":"1.1.1.0"},{"name":"sebastian\/exporter","version":"2.0.0.0"},{"name":"sebastian\/environment","version":"2.0.0.0"},{"name":"sebastian\/diff","version":"1.4.3.0"},{"name":"sebastian\/comparator","version":"1.2.4.0"},{"name":"phpunit\/php-text-template","version":"1.2.1.0"},{"name":"doctrine\/instantiator","version":"1.2.0.0"},{"name":"phpunit\/phpunit-mock-objects","version":"3.4.4.0"},{"name":"phpunit\/php-timer","version":"1.0.9.0"},{"name":"phpunit\/php-file-iterator","version":"1.4.5.0"},{"name":"sebastian\/code-unit-reverse-lookup","version":"1.0.1.0"},{"name":"phpunit\/php-code-coverage","version":"4.0.8.0"},{"name":"phpspec\/prophecy","version":"1.8.1.0"},{"name":"myclabs\/deep-copy","version":"1.9.3.0"},{"name":"phpunit\/phpunit","version":"5.7.27.0"}]}

Pull Requests

/cc @silverstripe/core-team

wilr commented 5 years ago

Given an hour spend working on archive issues is an hour not spent on other product issues I'd happily vote to ditch it. I'd imagine the developer experience without using Composer is pretty horrid.

stevie-mayhew commented 5 years ago

While a part of our history, it doesn't need to be part of the future - saying that I am one of those downloads from last month to get it up and running quickly on a friends machine who already had a MAMP running locally but no composer etc. I think if we remove the archive option it would have forced me to change that for them, which ultimately would have provided a better experience for them and forced lazy Stevie to actually teach instead of just going for the easiest option.

+1 to remove.

maxime-rainville commented 5 years ago

Let the past die. Kill it if you have to. That's the only way to become what you were meant to be.

unclecheese commented 5 years ago

My head just kind of exploded a bit because Maxime gave us his opinion in the form of “une maxime”.

robbieaverill commented 5 years ago

8 core committers in favour (@dhensby @kinglozzer @robbieaverill @ScopeyNZ @clarkepaul @flamerohr @wilr @stevie-mayhew) and presumably @chillu too (9) so that's a quorum

chillu commented 5 years ago

OK, here's what I'm planning to do on https://www.silverstripe.org/download/

Reword "Manage with Composer, or download the ZIP to your server." to "Install with Composer"
Remove "download" buttons on top, extend composer input field to full width
Change "System requirements" block to a link to https://docs.silverstripe.org/en/4/getting_started/server_requirements/ (remove noise like mentioning SQL server support). Inline with remaining "useful links"
Rewrite "installation 101" with a big link to https://docs.silverstripe.org/en/4/getting_started/. They're outdated, refer to the existence of an installer (which we've just axed). We currently have three getting started guides: ss.org, docs, lessons. Let's keep the duplication to a minimum.
Remove "video tutorial" block, the video is waaaay outdated
Remove "pre-releases" section. The pre-releases logic is broken, it shows older releases because newer stables don't have an RC. Remove "development release" section.
Remove "framework only" download option, it's not really a focus. We should add a "how to only run framework" section to the docs though.
Move "addons" and Github links further up

clarkepaul commented 5 years ago

Once you're done with all that, I'm happy to take a look if you think things might need moving around to make a better layout.

chillu commented 5 years ago

OK, I've created four pull requests, incl. removing >800 LOC from cow, making it a whole lot simpler to maintain and execute :) Credit to @tractorcow and others for creating an awesome codebase on cow, was super easy to follow and extract the relevant bits (well decoupled)

chillu commented 5 years ago

@clarkepaul The proposed changes are now up on UAT: http://sssites-ssorg-uat.sites.silverstripe.com/download/. Note that I've removed the "release archive" menu entry, since it somewhat duplicates the Github feature at https://github.com/silverstripe/silverstripe-installer/releases. This is all an effort to simplify the code we have to maintain. My long term goal is to make that download page on ss.org static, and remove all that noise around version parsing, CoreRelease data objects, packagist refresh jobs in the CMS, etc. - this is the first step towards that. Can somebody with access to the ss.org codebase please peer review the required code changes? https://github.com/silverstripeltd/silverstripe.org/pull/146. Maybe @unclecheese since he wrote a some of this? :D

clarkepaul commented 5 years ago

Page still looks pretty clean placement wise, no need for any re-shuffling. Text wise it has been stripped back a fair bit so I might be recommending some more text on the four steps (or make the text bigger) so they hold the space better e.g. adding some text about adding /admin to access the CMS. That's all CMS editable so can be done at any time I guess.

We could also redesign the top of the download page, now that the extra download buttons are gone it's quite gappy but I would think that it's good enough for now (can make recommendations separate from these changes).

chillu commented 5 years ago

That's all CMS editable so can be done at any time I guess.

Yep, 90% of it. Happy for you to have a go on UAT?

We could also redesign the top of the download page, now that the extra download buttons are gone it's quite gappy but I would think that it's good enough for now

I'm keen to separate any redesign efforts, and roll that into the larger discussion about "try vs. download" we're already having this week. Can happen after this goes live IMHO

clarkepaul commented 5 years ago

Yeah sure, that's what I was trying to say... nothing of concern blocking deployment, we can make UI updates as separate issues if we feel it's justified.

chillu commented 5 years ago

Live now: https://www.silverstripe.org/download/

Taitava commented 4 years ago

Just a note about the current download/release-archive page. I asked about this in the forum before knowing about this GitHub issue. The release archive page is a little bit confusing now that the zip releases are gone (of course removing zip releases is a good thing, I'm not blaming that).

TL;DR: I'd suggest to add a small description telling that zip releases are not offered anymore, and a link to this GitHub issue. For more: https://forum.silverstripe.org/t/what-happened-to-the-release-archive-page/2624/5

Cheddam commented 4 years ago

Hey @Taitava, thanks for the suggestion - we actually have a task in our internal backlog to do exactly that, but it hasn't been actioned just yet.

silverstripe / silverstripe-framework