[Feature]: Start releasing versioned Toodles releases

[thombruce]

Feature request

All work to this point is on v0.0.0.

Using Lerna, start versioning incremental changes. Adjust the Bug issue template with versions accordingly. Adjust the deploy job and rules for other deploy environments (Netlify) accordingly (these should release after version-bump only on release commits).

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[thombruce]

I'm conflicted by this because I enjoy having production deploys mirror the state of the main branch, and I like this being the case no matter what is merged or pushed at any given time - main should always be stable.

The sort of default approach to this would be to manually initialise a release... lerna release from the CLI or something like that. This would bump versions, commit and push a release all in one command... but requires manual initiation by the project manager.

What I might prefer is a job that runs whenever a push to main occurs or whenever a PR is merged to main. One that runs lerna release. This should not happen for lerna release commits, of course...

Rules being something like:

If an untagged commit is pushed or merged to main, run release task

Run deployment tasks on tags instead of against main branch commits

That should do the trick.

[thombruce]

No, it shouldn't...

That may work for PATCH versions, but MAJOR and MINOR versions should be incremented manually.

I still don't know how we'll automate the knowledge that a commit is a tagged commit (tags are pushed separately). Maybe by matching a regex of the form ^v\d+\.\d+\.\d+$ given that version-bumping commits should likely always match that pattern and no others will? At which point we... we know it's a commit intended for deployment and NOT intended to be released (because it's a release commit). So... strictly this means, what? It means we can automate patch releases whenever something is merged, and it means we can ignore the release job when a major or minor release commit is pushed.

Okay, but...

Not every merge or non-release push should result in a patch release. Why? Breaking changes. Y'know, the reason semantic versioning exists at all.

1. We could pull these PRs and merge locally, ensuring that the last commit then pushed back is a versioned one.
2. We could include some kind of meta in the merge commit's name or description that says it should bump minor or major.
3. We could fold on the idea of automated releases and treat the state of main as an EDGE release with actual releases handled manually.

3 is looking more and more preferable, I think... although using title meta is not a bad idea, worth looking into.

[thombruce]

Some chatter here about matching against tags using a subset of RegEx's capabilities available in GitHub workflows: https://stackoverflow.com/questions/58862864/github-actions-ci-conditional-regex

Though the questioner doesn't ask about tags, all of the answers reference this capability specifically in reference to tags. So that...

That probably solves the need to run deploys only on tagged commits. But it doesn't help us know what commits should be major, minor or patch released...

This may rule out 2 as an option. I'm sure there are other ways to do it, I just don't want to introduce too much complexity in order to achieve this. A novice developer should be able to understand this project, and that includes the workflows - let's keep those simple too.

So more and more I'm thinking... we release manually. Untagged main is always EDGE and we tag periodically with versioned releases. "Periodically" generally meaning whenever changes are made.

Maybe main isn't EDGE. Maybe... we have separate branches for MAJOR and MINOR changes, so that we can still keep up with PATCH releases that are non-breaking on whatever is used as "the release branch".

That said... sometimes you want to PATCH old releases too. A 4.0.2 might be released after a 5.0.0, just because the .2 patch contained non-breaking changes that improved the security or performance of the v4 release.

These are considerations that affect Git branch management too.

We're not there yet. We aren't there until at least v1.0.0, I would say. Keep it so much simpler than that for now, and let the decisions we make at this point inform those future decisions rather than the other way around... for now.

We CAN change approaches later, if it is reasonable to do so and provided that the semantic versioning history remains clean and un-interfered with.

[thombruce]

All of that's to say, we're just going to do it manually with either lerna version or lerna publish; probably lerna publish since it accomplishes more that, while we aren't using it now we may eventually.

So, we lerna publish and this will bump versions, commit, tag and - I believe - push these changes.

We can then... maybe use tags/releases to determine when to deploy... right? Yeah, that part should be easy. It is only a question of whether I want the Netlify deploy to mirror EDGE or latest tagged release.

We do have to be wary of usage limitations, which may indicate that deployment on tagged releases is the better option as we can stagger these:

[thombruce]

There is no way to have Netlify only build tagged commits either within Netlify's UI or within the netlify.toml config options.

You can skip a deploy by including [skip ci] or [skip netlify] in a commit message.

More info: https://answers.netlify.com/t/deploy-on-git-tags-only/43759

Probably preferable is to trigger a Netlify job via a build hook (configurable from Netlify UI) that is then triggered by a workflow with tag-awareness.

This would...

1. Need to be appropriately documented should forking projects wish to use it or not
2. Be stored in an environment variable (don't allow just anyone to trigger a Netlify build)
3. Preferably be partially configured in-repo using netlify.toml

Or wait... no. Preferably not. If someone forks the project and hasn't configured a build hook in an environment variable, then we should assume they want edge-builds and not configure this to not be the case. So...

3. Scratch 3 above - probably better to configure it in UI

[thombruce]

Verified: yarn lerna publish does indeed:

1. Bump the version: https://github.com/thombruce/toodles/commit/f4ea4f5cb6ada72e61a56b707b1451da3feb4f2c
2. Create a release tag: https://github.com/thombruce/toodles/releases/tag/v0.0.1-alpha.0

I'll have to remind myself to do this manually whenever changes are committed or merged...

This of course also triggered a Netlify build, but only because all commits to main trigger a Netlify build. This is the secondary thing I need to consider handling here, and my best option was what...

• [ ] Disable automated builds (either in UI or from netlify.toml file if possible)
• [ ] Setup Netlify build-hook
• [ ] Store Netlify build-hook in GitHub Actions environment variable
• [ ] Trigger Netlify build-hook from deploy workflow

[thombruce]

I also want to add a "Create Deploy" button to the README and I'm tempted to do this first, as it might be informative in terms of what finds its way into a netlify.toml file.

By default I still think the "Deploy to Netlify" button should result in a deployment that builds on every push... I think. This maybe partially depends on where Netlify is deploying from - does it encourage the user to create a new repo for their forked deployment? If not... how do we ensure the rules remain malleable for them? They can... disable automatic builds from their own Netlify UI... and then, provided they have setup a build hook in... where? In their own repo? Again, what if they don't have one... You kind of can't make this manageable in that case but okay...

Let's first of all see what the "Deploy to Netlify" button does. Does it encourage them to create their own repo?

[thombruce]

It does in fact imply that in the docs, yup:

While Netlify will use the specified base directory for the build and deploy, the entire repo will still be cloned to the user’s Git provider account.

https://docs.netlify.com/site-deploys/create-deploys/

That's fine. It uses the term "clone", not "fork". We expect users might clone the project and take it in a different direction. If they leave their cloned repository unchanged... they should have no problem fast-forwarding their repo to catch up with our developments. By "no problem" I mean it's probably a technical task, but one that a novice developer should have no problem with. Even if their project has diverged... well, then it's an even more technical task but a seasoned developer should be able to handle conflict-resolution. It's my job to make sure the conflict-resolution is as seamless as possible by... ultimately allowing as much to be configured from a single root file with predictable changes as possible AND by making reasonable use of semantic versioning to indicate the complexity of changes.

Preferably a fork's changes would be confined to a configuration file at the base of the project and/or plugin files that are explicitly supported somehow, though I have no intention of making that a possibility yet.

[thombruce]

By default, cloned repositories being built by Netlify... will build on every push made to the new repository (which the cloning user has total control over).

This is fine.

The repository might feature a deploy job (it already does for GitHub pages) that will be altered so that it triggers a build hook provided one is specified in an environment variable.

This is also fine. No build-hook, no problem; you're probably using automated builds.

The repository will not interfere with either of these conditions in a netlify.toml file.

This is fine. Those conditions are both fine.

---

So let's talk GitHub Pages

Presently, GitHub Page builds are instigated by a deploy job alluded to above. This might automatically create a GitHub Pages site for the cloned repository...

"Might". This is not fine.

If the user wants a GitHub Pages site, then the deploy job should be present and should do that... but we would prefer that this require some manual indication that it's what they want.

Of course the site can be built, published and then ignored (as I'm presently doing with the GH Pages build), but we would prefer the user were given an indication of what's being done for them.

Now, maybe GitHub Pages does require setup, in which case this is a moot point. But the job is there and is going to run regardless.

---

What I need to do is to reach a point at which I feel the repo is satisfactorily setup to meet the conditions, THEN test the conditions by using the deploy badge and/or cloning the repo. It should:

1. Deploy to Netlify if I click the badge
2. NOT deploy to GitHub Pages unless I set this up manually

Potential third condition being... Deploy to Netlify from a workflow if a build hook env var is present. My intention is to use this in such a manner that only tagged builds are deployed. And if that's the case, and cloning users want that too, then it should be documented how to achieve this from their own cloned repositories...

How would they achieve this from their own cloned repositories? They'd have to checkout the latest version, pull tags for the latest version and push these both to their own repo. The tag is relatively crucial.

That's kind of over-complicated for the desired function here.

---

Let's reach the point at which conditions 1 and 2 are met, and continue the discussion about versioned releases after that.

[thombruce]

As a simple test, I have forked Toodles into a repo belonging to an organisation...

First finding: When forking, workflows do not run automatically - this may differ if the repo is a "clone", however Netlify handles this.

Second finding: Even when I enable the workflows and run the deploy job, pages DOES require manual setup. The job fails! This is good, actually.

This means that a cloned or forked repo must do some prerequisite work to deploy to GitHub Pages. Preferable the job would fail more silently (skip) if those conditions aren't met, and we can look into that, but this essentially exactly what I wanted here.

Next step...

Deploying to Netlify with a "Deploy to Netlify" button.

[thombruce]

A button is going to have markup like this:

[![Deploy to Netlify](https://www.netlify.com/img/deploy/button.svg)](https://app.netlify.com/start/deploy?repository=https://github.com/thombruce/toodles&base=packages/web)

Note the inclusion of &base=packages/web.

Can I add that Markdown here and see what it will yield:

[thombruce]

Testing that button now. Notably all it asked me to configure was...

1. Connect to GitHub
2. Provide a name for the new repo (I went with 'toodles-clone')
3. That's it. Done!

Noticeably missing from those steps: The deploy script to be run and the deploy directory. Over on the main Toodles deploy I have these configured:

Base directory: packages/web
Build command: yarn build
Publish directory: packages/web/dist

Without them... Even though Netlify reports a successful build, the production deploy is blank. So... more configuration yet needed.

Also, the repo created for me is NOT a fork and therefore lacks a handy little fast-forward button present on a forked repo. It's a clone, as though the repo were brand new and wholly unique to that space.

This is fine. If the user does want to switch to a fork at a later time, they should be able to do so and to point their Netlify deployment to the new fork. The button is still otherwise very handy in that it makes new deployments a snap!

So, what do we need here...

• [ ] Configure build command and publish directory for Netlify deploy button

[thombruce]

I believe we need to add a netlify.toml file with:

[build]
  command = "yarn build"
  publish = "packages/web/dist"

That should do the trick. Let's add this and the Netlify button to the repo now...

[thombruce]

Before adding the button, I'm running another clone and deploy from the button here with the netlify.toml in place in the packages/web directory.

I want to be sure that the configurations work before I advertise the possibility of using this to others.

[thombruce]

Build did use the command specified... but failed with no clear reason. The log appears to show mostly success... and yarn build is working from the packaged/web directory locally.

Funny observation:

Netlify has created the new repo, but has not actually pushed the code to it. Seemingly it does this after build.

[thombruce]

I've hit "Retry" in Netlify. Presumably it is smart enough to know that it should still pull code from the original repo, though I fear it might attempt to find the code in the newly created but as yet still empty one...

Yeah, it failed and I'm pretty sure that's why because there's nothing in the logs. I'll have to reattempt again AGAIN from the button...

[thombruce]

After investigating the publish step logs, config modified to:

[build]
  command = "yarn build"
  publish = "dist"

Publish phase complained about missing directory "packages/web/packages/web/dist", meaning yeah... we don't need the "packages/web" part here.

Reattempting deploy with this change - build failing may be a false flag.

[thombruce]

With that change, it works.

I've added the "Deploy to Netlfiy" button to the repo and yarn lerna publish released patch v0.0.1.

Patch makes sense here since the changes are entirely trivial and non-breaking.

[thombruce]

So, users can now deploy to Netlify AND get a cloned version of the repo in practically a single click.

Alternatively, they can fork the project and manually setup GitHub Pages deployments. Speaking of which...

I didn't check whether workflows were automatically running in the Netlify-cloned repos... Lemme... I guess create yet another deploy from that button...

[thombruce]

We haven't turned off automated Netlify deploys, favouring an approach based on tagged commits...

I'll just wait on this new deploy to finish before confirming how that works... Hmm... So, the CI job is automatically running and I can be pretty assured that the deploy job is going to be attempted after that. Yep, there it is...

The deploy job will fail... probably... because GitHub Pages is not setup for the cloned project.

Yep, it failed with that reason.

Okay so, last question for this issue I think is... can we get that job to run ONLY on the condition that GitHub Pages is set up?

[thombruce]

I'm not seeing a way to configure that... but the green tick in the header says the branch is passing, and the CI badge... well, that still points to thombruce/toodles, so that's gonna show as passing while thombruce/toodles is.

The cloner can actually disable workflows manually from the Actions tab: Select the workflow from the left side-menu, then under a three-dots menu in the top right you can select "Disable workflow".

This appears to be the best way to handle this.

The job failing isn't too obtrusive, and we'd much rather maintain it as a means by which cloners/forkers can quickly setup a GitHub Pages deployment. And for anyone who finds it to be a nuisance that a workflow is failing, they can manually disable it.

Okay. This is fine.

[thombruce]

So back to whether or not to deploy to Netlify ONLY on tagged commits...

To do that, we'd need to setup a job that runs... only on certain tags, which is absolutely possible.

We'd need to switch off automated deploys in the Netlify UI (this is an unobtrusive change; doesn't change the source or anything about how it's forked).

AND we'd need to setup a build hook, to be added as an env var. Then we'd create a Netlify deploy workflow and have it run only on appropriately tagged releases.

All of which will be fine because... if the user doesn't set an env var for their own build hook, the job should quietly fail (we can also potentially check for the presence of said var before attempting).

I'm electing not to do this yet. And I'm not going to create a new issue to consider it at this time. But I am satisfied in having discussed it here.

Closing the issue.