Open indirectlylit opened 3 years ago
It's not a blocker, but one slight complexity in separating the linting and building code, is our use of a custom module resolver that stops the linter complaining about our Kolibri API spec imports.
What this would mean is that the linting configuration for the kolibri-plugin-tools
would have to enhance the configuration provided by kolibri-formatting-tools
in order to substitute in the custom import resolver that handles this: https://github.com/learningequality/kolibri/blob/release-v0.15.x/packages/kolibri-tools/.eslintrc.js#L63
Another possibility would be to make kolibri-plugin-tools
an unrequired peer dependency of kolibri-format-tools
and set it in the case that it is present. A little interconnection, but might be simpler in the long run.
in a thread @rtibbles mentioned that
maybe one day, we can make it happen automatically (although that might not jive with your proposed new versioning scheme)
This would be very helpful for the kolibri-plugin-tools
/kolibri-tools
/kolibri-core-for-export
package.
Whether it makes sense for the other packages is an open question we should discuss. One issue is that it would make it harder to resolve the circular dependency between KDS and Kolibri if kolibri-formatting-tools
upgrades in lock-step with Kolibri versioning.
Another minor tweak here is that we can't prefix all the packages with kolibri
, as eslint plugins and browserslist configs have very specific naming requirements, which is why they are prefixed in this way.
Status
This issue is currently a draft and open for feedback.
Current state
The Kolibri product ecosystem has a number of "internal" Node packages. These are currently tracked monorepo-style in the Kolibri repo, and some are used by other products. For example, Studio and KDS both use
kolibri-tools
.These packages are currently defined within the
packages
directory:kolibri-tools
browserslist-config-kolibri
kolibri-tools
eslint-plugin-kolibri
kolibri-tools
kolibri-core-for-export
a.k.a.kolibri
kolibri-tools
hashi
(unpublished)These are all implemented as Yarn 1.0 workspaces, along with
kolibri/core
andkolibri/plugins/*
:https://github.com/learningequality/kolibri/blob/a7d136171181893b394b89425c4ff6dcdd00403a/package.json#L5-L9
Currently, all of these plugins except for Hashi are supposed track Kolibri's version number, and get re-published to NPM on a regular basis. This occurs with the
packages/publish.js
tool. In practice, this process occurs rarely and is not documented. Currently all NPM dependencies aredev
versions.Some additional notes:
kolibri-tools
as a workspace-based package using symlinkshashi
both as a workspace-based package and also through custom webpack build tooling in hashi that targets Kolibri's static folder as the output directorykolibri-core-for-export
could be replaced by an automocker in the jest configuration ofkolibri-tools
and bundling of these four dependency modulesChallenges
kolibri-tools
->kolibri-core-for-export
-> Kolibridev
versions and use thenext
rather thanlatest
tagProposal
Overview
kolibri-plugin-tools
kolibri-tools
&kolibri-core-for-export
kolibri-formatting-tools
kolibri-tools
kolibri-i18n-tools
kolibri-tools
browserslist-config-kolibri
kolibri-plugin-builder
eslint-plugin-kolibri
kolibri-code-formatting
kolibri-iframe-sandbox
hashi
N.B. any renaming of
hashi
will have to not be a global change, as this could cause significant breakage. Gradual, minimal updates are required, starting only with the name of the package.Versioning
Most of the internal packages can be independently versioned and published because they change at different cadences and are highly decoupled from each other. Any package versioning labeled Independent above can bump its first official version to 1.0.0, and then increment minor, patch, and major versions as appropriate using semantic versioning conventions.
On the other hand, some tooling related to building and testing plugins has a tighter coupling to the Kolibri code-base, while still providing some degree of abstraction to allow plugins to be built and tested independently. Currently the
kolibri-tools
andkolibri-core-for-export
packages fall into this category, and in this proposal onlykolibri-plugin-tools
would. Here it's the versioning is labeled Tracking. It does not need to follow Kolibri patch releases, but the major and minor version numbers should always match.Having independently versioned packages within the Kolibri repo is a bit strange because we use git tags that refer to the Kolibri version, while the other package versions are specified only in package.json files. In theory (and in practice) these packages can be released from any branch including
develop
or even forked feature branch. This can make it harder to associate particular released NPM versions with a particular git commit, branch, or tag. We will need to be careful not to abuse this ambiguity, and be diligent about documentation and communication for package updates.Refactoring
There are three separable tools within
kolibri-tools
. Separating them will simplify the process of regularly publishing updates for the packages that need it while pinning stable versions of the packages that don't need regular updates.kolibri-plugin-tools
– Building and testing Kolibri plugins without needing the entire Kolibri codebase at build-time. Contains an abstract representation of the core Kolibri API (a.k.aapiSpec
), which allows plugins to be built to an interface rather than a specific codebase. Subsumeskolibri-core-for-export
.kolibri-formatting-tools
– Linting and and autoformatting code.kolibri-i18n-tools
– Managing strings, context, crowdin, and other i18n concernsDocumentation
The Kolibri Dev Docs should have a new page covering how all of this works, and each internal package should have a top-level readme.md file with a high-level description of the package's purpose within the Kolibri product ecosystem.