install / update hooks

[stuartpb]

Continued from plushu/plushu-apps#5.

The main Git plugin should have install hooks for cloning Git remotes into a directory, and for cloning Git remotes into a repo. It should also have hooks for updating dirs and repos.

This would remove all the Git-specific code from plugins, and allow it to be used in plugins like apps and services, and allow it to be supplemented / replaced by plugins for alternative systems, including bare HTTP gets via wget or curl.

This functionality could be a separate plugin (as I was originally considering), but there's no real harm in including it as part of this plugin: if a user only wants Git integration for installing plugins (and not receiving Git pushes into repos), they can simply not install any plugin that creates repos.

Alternately, they could install an alternate version of this plugin that only includes the installation hooks (for dirs), or only includes the hooks for repos (like what it has now).

[stuartpb]

How does retrieve-repo (as I'm now calling it) and update-repo interact with git-apps and the like?

I think git-apps will have to listen to something like a build-repo hook that will do something similar to what its git-update-hook hook does, which will be called by apps:install if a Git repo is retrieved or updated.

The alternative would be that plushu-git's retrieve-repo, on change, emits some kind of repo-modified plushook event that plushu-git-apps listens for.

The thing to think about here: right now, git-apps' update hook calls create-app when receiving a push. (wait no it doesn't, only apps:create calls create-app) It does not create apps when creating a repo (because creating an app creates a repo, and that would cause an infinite loop).

Since git-apps is currently listening for a hook (one that is by nature of the interface that's been created, Git-specific), if a repo gets created with content, it should emit an event as well.

So, git-apps may get a top-down build-repo hook, which implies the app already exists, for some other reason in the future. However, to match its current semantics, git-apps should listen for the repo-modified event (rather, have a repo-modified hook), and then create the app (ensure its existence) the same way it does for a push.

This does mean that, in the presence of git-apps, "repos:install" (or whatever) would be equivalent to "apps:install", and so maybe it will be used instead. (Or I can just have two duplicate commands rather than force users who want.)

That kind of feels like all app management should be done via repos and then the app related stuff be handled by git-apps - but no, that would mean multiple vcs-apps plugins would fight over integrating app stuff to repos.

yes to this next thing see comment below Maybe the general apps plugin could be removed of its commands and changed to only listen for hooks emitted by the git-apps and repos plugins (perhaps including a commands hook to basically alias all of repos as apps for compatibility).

Then I guess plushu-apps would translate all the create-repo (etc) calls (by "translate" I mean have a hook for what it's "translating" from that then calls another hook for what it's "translating" into) into create-app so the apps plugins would just listen for app hooks, and not overstep if for some reason they're installed without apps installed? I guess that makes sense, you could have a plugin that has apps functionality in addition to some other stuff, and you wouldn't want the apps stuff to apply if it's just repos.

Then, that would mean git-apps... calls create-repo to initialize things that should go along with a repo when it gets interacted with? (or plushu-git does it?) That doesn't seem right. Don't forget about git-receive-init and how that's optional.

Maybe plushu-apps should listen for both create-repo and receive-repo (??) and translate them into create-app, and then (as now) it has its own create-app plugin that creates the app directory.

I think so, create-repo is just an alternative to retrieve-repo.

The thing is, it pretty much really wants to always know the results of retrieve-repo... this is one of those times filter hooks would have come in handy. But instead we can have a hook that just always expects the output from another hook.

And then apps listens for the generic events happening around repo changes, and then the version-control-app-integration plugins listen with a

(Also, a thought: should we maybe pass stdout to the repo hooks via another fd so they can print output the same way the dir hooks can?)

The thing is, some kinds of things are system-specific (like how Git receives). So some things circumvent hooks in plushu-apps altogether.

So plushu-apps doesn't listen for the post-repo-retrieve event, only git-apps and co. do, performing builds appropriate to their specific design the same way as they do for whatever special interfaces (eg. git push/receive-pack) they provide. Retrieving is repo-specific but general to all repo (vcs) systems, so that's why it's on a hook. (even curl can retrieve, but you can't push a curled tarball.)

NO WAIT plushu-apps DOES listen to the post-retrieve (by retrieve meaning the alternative to create, and some other verb means update) event to create the app (if any vcs retieved it), it listens to it earlier than eg. git-apps, which performs the build step (based on whatever line in the retrieve-repo output).

the big refactor relevant to apps is that creating apps no longer creates repos. creating repos creates apps, by way of plushu-apps listening to the events that create repos (create-repo (init-repo?) and retrieve-repo) as a tagalong with the version control stuff that creates the repos, then firing the create-app hook for other app-related plugins that care about app creation and need to init something.

this is essentially necessary to keep sanity in the world where apps can be created with pre-existing content.

(side note: maybe create-repo is a hook that always accompanies init-repo and retrieve-repo? nah, who would be in charge of that? not worth figuring the inheritance)

if it wasn't clear before, apps are now clearly fully-dependent on repos. this isn't really a change.

it does mean the repos plugin needs to be written (of course, it already did, it just wasn't because it wasn't in Dokku where repos never stood alone).

[stuartpb]

As discussed in plushu/plushu-apps#7, apps:create isn't quite the same as repos:create because app naming has different criteria for name validity and generation - also true for fork and rename. So, in apps' case, these would actually just be mostly similar to the repos commands (including calling create-repo etc hooks, which plushu-apps then listens for to handle create-app, which it also listens for to setup the base app metadata stuff) but with additional name checking and generation.

So that means repos:create can create repos at invalid app names... then yeah, that'd be part of what plushu-apps' -repo hooks check for- they don't delegate out to the apps stuff when a repo has an invalid app name.

[stuartpb]

So, to sum this up:

• app creation is now a rider that follows repo creation. apps:create creates a repo, and lets plushu-apps' create-repo hook handle the app creating part.
  • plushu-apps' subcomands throw an error when creating an app with an "invalid" name, which will include subdomains that are already in use (as apps or something else) when plushu-app-name-domain is installed.
• Apps will also be created when a repo is instantiated by other means (including via other plugins such as git-receive-init), specifically when a repo is retrieved. The details of this are still pending the implementation of retrieve-repo in plushu-git.
• Commands (or whatever) that run retrieve-repo or update-repo will pipe the results to another hook like repo-retrieved, which will decide how to handle the results of a repo retrieval/update (since plushu-git handles retrieve-repo, but plushu-git-apps wants to handle its result).
  • I might call it post-retrieve-repo, although that would mean there's a pre-post-retrieve-repo, which is crazy.
• plushu-apps will handle this "repo-retrieved" hook with a numbered hook before plushu-git-apps to create the app directory.

[stuartpb]

On creating apps with the same name as a taken subdomain:

• repos that aren't valid app names still create apps because the app can be hosted with domains, but apps:create will prevent you from creating them - you have to do it through repos:create.
• Actually, I'm not sure about that. What should app-name-domain do in that case? Not create the app's domain? How can it tell when the subdomain is being used by something else - does it count the instances of the app's name coming up with the subdomains hook and then not actually vhost the app if it comes up twice? Does that mean the other plugin using the subdomain should do the same thing? If that other subdomain goes out of use, how is apps-name-domain notified (a subdomain-removed hook)?
  • Honestly there are a few hacky things about apps-name-domain at the moment. It doesn't regenerate configs if you change the server's name.

[stuartpb]

Also just want to point out all other app manipulations (rename, fork, destroy) are similarly actually repo manipulations, with a hook for that repo action that forwards it to the equivalent app-action hook.

[stuartpb]

After revisiting #5 and giving it a little think, I'm thinking maybe apps shouldn't be coupled to repos by default. Maybe app-repo integration should be a separate plugin (ie. creating repos when an app is created, so as to permit pushing to a repo to update an app), as well as the full creating-an-app-when-a-repo-is-created-by-other-means plugin (which only really needs to be installed when plushi-git-receive-init is installed). The latter could maybe even be namespaced by itself - explicit apps on the base and implicit apps when creating repos underneath a certain repo path!

[stuartpb]

plushu/plushu#18 should maybe be considered, too (kind of an old plugins-plugin issue that wasn't moved).

[stuartpb]

What about a plugin that reads git URLs and gets tarballs when Git isn't present and it's a GitHub URL (which it rewrites)?

[stuartpb]

With the new designs around plushu/plushu-repos#1, implementing this will mean commands needs to abort early if PLUSHU_REPOS_DIR isn't defined (since plushu-git will now probably become a semi-core plugin that has functionality separate from the functionality that is now dependent on plushu-repos).

[stuartpb]

Oh nutbunnies, it looks like I forgot to post the comment I was writing a couple days ago on how this is going to be implemented.

Anyway:

retrieve-dir

takes the path to the dir as $1, and the URL as $2. Each plugin's hook checks for the existence of the dir before checking if it's possible for the plugin to retrieve it. stdout works normally (messages).

update-dir

Each system checks if its relevant dotfile or directory is in the dir specified by $1, and outputs its name, followed by any relevant metadata if the contents of the dir were updated. (If no metadata is present, it will be assumed that the dir was not updated. If you have no metadata, you still need to print "1" or "updated" to signify that an update happened.)

It is expected that only one system is going to find its metadata in the dir (and is only going to output one line, as only one "ref" refers to the checked-out files). If multiple systems control the dir, that's your problem, and you'll have to figure out how to handle it.

For plugins, for example, this metadata will be passed to the plugin's update script as args.

commands' stdout should be directed to &3 as described below.

There is no "dir-updated" hook like there is for "repo-updated", as dir updates are specific to the plugin using them (the plugin can pipe/direct to read).

retrieve-repo

Takes the repo name as $1, and the URL for $2.

Like retrieve-dir, each plugin tries to retrieve the repo. If it can, it outputs a line for each "ref" (or whatever the equivalent is for a retrieved branch or whatever in its system), starting with the system name (as described below.

commands' stdout should be directed to &3 as described below.

Any caller of this hook is expected to pipe it to the repo-retrieved plushook.

repo-retrieved

A hook that gets the results of retrieve-repo.

plushu-repos has a "repo-retrieved" hook that plushooks create-repo if it receives any input (to preserve the create-repo pattern that, if one system has a repo, all systems will have one).

The create-repo hook also serves so that systems bound to repos, such as apps when apps-repo-binding is installed, will have their relevant behaviors triggered (in other words, anything that happens when repos are created will also happen when repos are retrieved).

git-apps will trigger on create-app being triggered (plushu/plushu-git-apps#9), rather than listening on repo-retrieved.

update-repo

Basically the same thing as repo-retrieved, except it doesn't take a URL, and the system's metadata will probably be different (eg. for git, it would include the new and old values of the refs).

commands' stdout should be directed to &3 as described below.

repo-updated

A hook that gets the results of update-repo.

stdout for retrieve-repo, update-dir, and update-repo

This varies slightly between retrieve-repo, update-dir, and update-repo, but they all use a similar pattern: each plugin outputs a line (or lines) when it handles the dir/repo. (the general "stdout", for messages etc, is on file descriptor 3.)

Each line is in "read -a" format: whitespace-separated fields, with a backslash escaping non-separator space characters (or literal backslash).

First parameter: the system (eg. git) that is responsible for the action.

Anything following that indicates that action was taken: the format of the line (eg. ) is system-specific (eg. for git it's the ref retrieved, along with (for updates) its old and new values).

[stuartpb]

Wait, crap, if plushu-git-apps listens on app creation, then it won't build the app when the app already exists and only the repo is being retrieved Yes it will, because create-repo and (when app-repo-binding) create-app is called whenever the repo is retrieved.

Wait... does that mean git-receive-init will cause the app to be rebuilt on every push even when master isn't present? Maybe the create-app build should only build when there isn't an existing build. (Implemented.)