Work out how to create multiple npm modules from a single monorepo

[mrchrisadams]

I'm not sure how we should best handle having a single repo, but multiple installable packages.

All the cool kids seem to be using LernaJS for managing stuff like this.

https://lerna.js.org/

I think this would be the most popular tool to save us a bunch of work on this, but I'd appreciate some pointers.

Questions:

We don't need all these answered.

• what npm run taskname scripts would we need?
• what would a generated leaf dataset package (i.e. the ones with a calculation js file) look like using this?
• what would the npm publishing process look like under lerna?
• how do we handle contributions, or use other datasets from other repos?

[mrchrisadams]

Okay, I've had a bit more of look at this now.

As far as I can tell, Lerna works if you have a bunch of existing modules that you would like to be able to deploy independently to something like npm. You can deploy individual packages independently, or deploy them all in bulk, giving them tags and the like.

It expects each package to be in a set directory called packages, like so:

my-lerna-repo/
  package.json
  packages/
    package-1/
      package.json
    package-2/
      package.json

We don't need to use packages as a top level directory name - we can use an array of directory names like we currently have instead.

But if we want to make an emissions model available on something like npm, we'd need to have a package.json for it, as that's what npm expects for publishing.

I think we'd still need to figure out dependencies between packages. Lerna has a thing called hoisting, which stops us duplicating the same dependencies across a bunch of packages.

Even if we were working locally with them in the same repo, if we wanted to make it possible for someone to install just one model like npm install @tgwf/defra-flight-model, that might depend on a package elsewhere so:

long1 = dataFinder.getDataItemValue('transport/plane/generic/airports/all/codes', 'IATACode=' + IATACode1, 'longitude');

in this case, to bundle just transport/plane/generic/airports/all/codes in, need a module that had that, that published on npm as well beforehand. The convention used by other projects is to use have use dashes for names like this:

npm install @babel/helper-replace-supers

Which corresponds to master/packages/babel-helper-replace-supers in the code:

https://github.com/babel/babel/tree/master/packages/babel-helper-replace-supers

It's not obvious to me how we'd keep the nesting of folders this way yet. I think we'd need this:

transport/plane/generic/airports/all/codes

to be named something like this:

transport-plane-generic-airports-all-codes

And we'd likely end up with something like each needing to be installed like so:

npm install @tgwf/transport-plane-generic-airports-all-codes

On the bright side, this flatter namespace makes it easier to include packages from anyone else following this rough convention, without needing them to be in this repo.

I consider that to be a happy side effect, TBH, as it would lower the barrier for others to make similar calcs possible, assuming they follow a similar way to define inputs and outputs.

In fact you'd be able to pull in their assumptions for that model fairly easily, in something like so:

npm install @some-other-org/transport-plane-generic-airports-all-codes

And then run the calcs in something like observable, or Nextjournal, in a fairly interactive fashion, as you aren't needing to make round trips across the network all the time.