Eliminate the need to rebuild depndent libraries

[kirlat]

Right now many of our dependencies are chained: data-modules may be used by client-adapters which, in turn, are used by components, and components are used by webextension, embed-lib, and nemo-ui.

Each package imports its dependencies from a bundle of a package it depends upon. So components, for example, import client-adapters code from client-adapters/dist/alpheios-client-adapters.js. As a result of this, if some package up the line is updated, we have to rebuild all the packages along the way to get new functionality available in the package way down, such as webextensions.

If something if changed in, for example, data-models, we have to rebuild data-models, client-adapters, components, and only then webextensions. This is a lot of work!

I think we can eliminate all this trouble by importing from a "main" JS file of a package such as index.js. This is how it is done in LexicCS.

LexisCS exports all its dependencies in index.js, and ClientAdapters and Components import LexisCS classes from its index.js, not from its bundle. As a result, when something in LexisCS is updated, we don't need to rebuild a LexisCS bundle: during its build process Components will pull updated LexisCS classes from LexisCS source code and will include them into its own build.

This technique will also allow clients (e.g. Components) to treeshake its dependencies (e.g. LexisCS).

The only problem I've discovered in a schema like this is with tests. If they require direct usage of classes from a dependent package (i.e. if tests of Components would need to access some classes from ClientAdapters directly) the schema described above won't work: Jest does not know how to import ESM yet (there is a wip on that) so we have to create a transpiled bundle just for Jest to import. However, this can be a temporary measure until Jest will learn how to work with ESM. We might also try to find a way to transpile in such cases on the fly.

@balmas, @irina060981: what do you think about the approach listed above? Would it be advantageous for us?

[balmas]

How does this differ from the way we were doing things before we bundled everything into components? Is this going back to a prior approach, or is it taking advantage of latest improvements in ESM ?

[irina060981]

We need to be able to use Jest - it is an important part of the developing process for inflection-tables for example. Tests uses fixtures, data-models, client-adapters repo. So with your suggestion - I would have double work - to create bundle for Jest and add/controll dependencies inside index.js. So this way - I don't see advantages of "direct" approach.

As for embed-lib and webextension - all dependencies are inside components already. What would be the benefit to extract them form components and add directly to webextension and embed-lib?

I believe that webpack controlls avoiding double-dependencies here. Then why should we do it ourselves using direct import?

May be I didn't understand the idea.

[kirlat]

Then why should we do it ourselves using direct import?

The main problem we have now is that if we made changes to some library up the dependency chain (e.g. data-models) then we have to rebuild data-models, client-adapaters, components, and webextension in order to have those changes fully sourced in. If we make some fixes in data-models, we have to rebuild all those packages in order, taking care of proper versioning and branching in the process.

If we use direct imports it will break that dependency chain. If we've updated data-models and this library is imported directly by webextensions all we need to do is to rebuild webextensions. Much simpler!

[irina060981]

If we use direct imports it will break that dependency chain. If we've updated data-models and this library is imported directly by webextensions all we need to do is to rebuild webextensions. Much simpler!

I believe that it is not exactly true. Because if you changed data-models you should check if client-adapters, components and others are still working well. So you will need to rebuild and retest them, I believe.

[kirlat]

Because if you changed data-models you should check if client-adapters, components and others are still working well.

This is a valid point. I just was not touching testing because it's simpler: we just need to run tests after the update. With code updates we, in addition to rebuilding, will need to take care of proper versioning, branching, and tagging. So I was trying to find a solution for bigger problem first and then go to smaller ones (but we should not forget about them nevertheless).

[balmas]

I believe this was resolved by the switch to lerna and the mono repo. @kirlat is that correct?

[kirlat]

Yes, it is