Publish packages and automate publication for installing in farmOS

jgaehring commented 2 years ago

We had a discussion the other day about organizing this repository and packaging releases of field-module-tasks so that farmOS can install the final build. @pcambra suggested Asset Pagakist, which we could use for just publishing the bundled JS file itself if we wanted. Or, since we already have a build process for all the YAML files Drupal needs to install it as a module, we could just add a composer.json file and publish it to Packagist.

I think I slightly prefer the latter option, Packagist, at this point, since it should be a more or less complete option, without leaving open the question of updating those YAML files separately if the module.config.js changes in field-module-tasks. I'm also thinking I want to eventually publish field-scripts, field-module-tasks and even field-kit itself to the npm registry eventually, so having GitHub action where we can publish simultaneously to both registries with a fully automated workflow that does all the work whenever we tag, that'd be super sweet.

I actually tried publishing field-scripts as a scoped package to our npm org as @farmos.org/field-scripts but was getting a weird 403 (Forbidden) error for some reason, running npm publish --access=public from the CLI. I'll have to investigate why at a later date, but I suspect it's either b/c I still have dependencies as direct links to the file paths (eg, { "field-kit-utils": "file:../field-kit-utils" }), or maybe I need to create the package first through the web GUI.

mstenta commented 2 years ago

Great thoughts! I have some experience with Packagist, Asset Packagist, and other options - and there are a few gotchas/considerations to be aware of, so I'll just add a few thoughts here for future reference...

FWIW we initially used Asset Packagist to host farmOS-map.js, but later moved to simply building a ZIP file via GitHub Actions and attaching it to release nodes (example), and then pull that into farmOS via Composer. It's simple and works.

There were some things about using Asset Packagist that rubbed me the wrong way. For one, it pulls ALL of the library's dependencies into the codebase as well, even though we only needed the built JS file(s), which felt weird/wrong. So with farmOS-map it also downloaded the whole OpenLayers JS repo, even though that is built into farmOS-map.js. Completely unnecessary. Another thing was that using asset-packagist required defining it as a separate repository in composer.json, and that had to be done in the ROOT composer.json, which was outside of the farmOS project... long story, and we did find a better workaround for that, so it wouldn't be an issue if we wanted to use Asset Packagist again in the future... just comes to mind...

I think I slightly prefer the latter option, Packagist

I tend to agree, if only because I had a bad experience with Asset Packagist :sweat_smile: but also because Packagist "just works" and we could include that dependency in the farmOS composer.json very easily.

However... something to be aware of:

Packagist.org requires that you point the package at the default Git branch of a repository, which contains the composer.json. So this works differently from NPM (IIRC), where you literally push a directory of files. There would need to be a repository for the Tasks module, with a default branch that has the composer.json file in it, and THAT is what the Packagist.org project would point to. It handles automatically syncing tags and branches from that repo.

So, in order to publish to Packagist.org, we'd need to create some kind of read-only artifact repo, which the field-kit repo automatically publishes the results of field-scripts output to. This would work fine, and would result in a nice packagist namespace like farmos/field-kit-tasks or something like that.

Another option would be to publish to a project on drupal.org, instead of packagist.org. Drupal.org hosts it's own Packagist repository, which allows you to run composer require drupal/[project-name]. So if we made a project at (for example) https://drupal.org/project/farm_fieldkit_tasks, then it can be added via composer require drupal/farm_fieldkit_tasks. Three reasons I bring up this options:

It would essentially mean drupal.org would be the "artifact repo" (like I described above) instead of a separate GitHub repo.
@symbioquine has already established a way to automate the process of pushing code to a drupal.org repo via GitHub Actions in his farmOS-wfs module:
- https://github.com/symbioquine/farmOS_wfs
- https://www.drupal.org/project/farmos_wfs
If a module is hosted on drupal.org, and it provides strings wrapped in the t() function, those strings are automatically made available in localize.drupal.org (maybe help with #214 / #377)

jgaehring commented 2 years ago

Ohhhh, this is all excellent background, thank you!!! This is especially good to know:

If a module is hosted on drupal.org, and it provides strings wrapped in the t() function, those strings are automatically made available in localize.drupal.org (maybe help with #377)

In fact, even taking into consideration all the other factors, that may be more than sufficient reason to host on d.o, cuz that would be HUGE!

jgaehring commented 2 years ago

So I'm looking at the Composer docs, and found the "repositories" property can have a type of "package". Based on that, I wonder if it would be possible to create a zip in Field Kit's GitHub releases, then just add something like this to farmOS's composer.json:

{
  "require": {
    "farm_fieldkit_tasks/farm_fieldkit_tasks": "2.0.0"
  },
  "repositories": [
      {
          "type": "package",
          "package": {
              "name": "farm_fieldkit_tasks/farm_fieldkit_tasks",
              "version": "2.0.0",
              "dist": {
                  "url": "https://github.com/farmOS/field-kit/releases/download/2.0.0/field_module_tasks-2.0.0-dist.zip",
                  "type": "zip"
              }
          }
      }
  ]
}

I'm sure there may be better ways to automate that, but perhaps for the interim this would suffice. What do you think, @mstenta?

mstenta commented 2 years ago

Yea that is actually the same way we include farmOS-map! So that will work!

Here is where we include farmOS-map: https://github.com/farmOS/farmOS/blob/2.x/composer.libraries.json

jgaehring commented 2 years ago

Oh perfect, and that's an excellent example to work from. Thanks!

jgaehring commented 2 years ago

Ok, so I'm trying out a published draft right here:

https://github.com/jgaehring/field-kit/releases/tag/field-module-tasks%402.0.0-alpha.1

I think the composer.json would look like this:

{
  "require": {
    "farm_fieldkit_tasks/farm_fieldkit_tasks": "2.0.0-alpha.1"
  },
  "repositories": [
      {
          "type": "package",
          "package": {
              "name": "farm_fieldkit_tasks/farm_fieldkit_tasks",
              "version": "2.0.0-alpha.1",
              "dist": {
                  "url": "https://github.com/jgaehring/field-kit/releases/download/field-module-tasks@2.0.0-alpha.1/field-module-tasks@2.0.0-alpha.1.zip",
                  "type": "zip"
              }
          }
      }
  ]
}

The module worked locally for me, so I guess the next step is to just try it out on test.farmos.dev. I'm not sure if the composer.json is needed for testing, @mstenta, but let me know if I should open a pull request to add that as a repository/dependency to farmOS, and which specific composer.json to add it to. I will probably close this issue in the next day or so, unless I hear from you that it needs to packaged some other way.

For now I'm creating the zip manually. I think it's probably best to see how that works before trying to wire up any kind of automation. I've done some preliminary investigation, however, and I'll drop a few links here for later reference:

Releasing projects on GitHub
Relevant GitHub Actions I found:
- actions/upload-release-asset (deprecated, but recomends the following)
- action gh-release
- ncipollo/release-action
- Evan You's GitHub Action for releasing Vite plugins with a similar tag naming convention
npm pack: command for creating tarballs with npm, might be useful in a workflow

No later than tomorrow I will probably tag Field Kit itself for alpha.7, which will be our "public alpha", and may also move this show back over https://farmos.app! :rocket:

jgaehring commented 2 years ago

Ok, final update...

Field Kit 2.0.0-alph.7 is tagged, and the Tasks module is now officially released in the farmOS/field-kit repo (ie, this repo, and not just my fork) as field-module-tasks@2.0.0-alpha.1.

So I'm closing this issue, and with it, the 2.0.0-alpha.7 milestone. Now on to beta! Huzzah!!! :champagne:

farmOS / field-kit

Publish packages and automate publication for installing in farmOS #497