Webpack configuration: some licenses not discovered during build when module concatenation is enabled

[jayaddison]

Describe the bug During migrating to luxon in #223, the license for that library was expected in the licenses.txt output file produced by our webpack configuration. That doesn't appear to have been the case initially - although after some investigation, it appears that disabling Webpack's module concatenation optimization allows the license to be discovered.

The site is temporarily down until further investigation can identify more precisely why this is happening.

A number of other licenses, including d3-related entries, dexie, i18next, and some -- although not all -- workbox-related items appear to be affected too.

To Reproduce

1. Check out a branch prior before 1630f8f6cd652dc54ca63af2c7842a832c8fcaf8 was applied
2. Perform a production-mode build (this bug only seems to appear during production builds)
3. Search the contents of the output licenses.txt for luxon

Expected behavior The licensing information for affected packages should be discovered and included in licenses.txt.

[jayaddison]

Commits 1630f8f6cd652dc54ca63af2c7842a832c8fcaf8 and b4d917a828e914594c3a5f383771039ecef1ac68 address the immediate and near-to-mid-future aspects of this.

It'd be nice to add some test coverage or similar, but also slightly tricky to figure that out, given that this is a build process -- and since we should likely use an entirely separate implementation of the license detection process, otherwise we won't detect variances.

Roughly speaking, modifications to package.json to add/remove dependencies should probably have associated license output changes (even that is complicated though: not all dependencies are bundled into the distributed app).

[jayaddison]

Here's a bash script one-liner using jq that identifies any top-level packages from package.json that aren't contained in the licenses.txt:

$ comm -1 -3 <(cat public/licenses.txt | sort) <(jq -r ".dependencies | keys | .[]" package.json | sort)

That's far from perfect: in particular, it doesn't check whether the licenses of transitive dependencies are included. But it does provide some basic safety-check assurance -- essentially it's using package.json to figure out a list of dependencies that should have corresponding license information published.

(in practice there are various reasons why not all of them may do in practice - usually small consistency bugs / issues with packaging, filenames, identifiers, and so on. having a (usually short) list of missing licenses is helpful to track those down)

[jayaddison]

The immediate problem (absence of some licenses from the application bundle) was resolved shortly after it was discovered, and a build-time check (that is required in production, and likely to occur frequently in development) has been added to reduce the chance that this can happen again. Future improvements to that build-time license check are possible.