To allow us core/extension developers to easily split chunks, our webpack configuration will have to do the following:
autoChunkNameLoader
With the given codes example:
const LogInModal = () => import('./LogInModal');
A new loader will first find all async import instances and transform those snippets by adding the appropriate chunk name/path and any other configuration we might want in the future.
This will compile the LogInModal module into dist/forum/components/LogInModal.js (+ its .map file).
RegisterAsyncChunksPlugin
When hitting the backend for the chunk, we need to tell which extension/package the chunk belongs to to avoid conflicts in paths. To do that, a new plugin will pick up on any split chunks and register the chunk ID, concaednated module ID, with its path and extension ID to the flarum export registry (#3833).
So, for example, in the module where the code above resides, a line will be added at the end:
In order to determine the URL of the chunk to import, we replace webpack's async loader function, the function takes a chunk ID as a parameter, we need to be able to map that to a path and extension ID, so that when constructing the URL we can do:
Where in our example, the namespace would be core and the urlPath would be forum/components/LogInModal. the final URL would be http://flarum.lan/assets/js/core/forum/components/LogInModal.js
Module ID
Because webpack is responsible for chunking, and it does so in its own format/way, when manually retrieving a split module from chunk X (for example extension A async importing the split LogInModal). We need to call the chunk callback that will return the module, and therefore execute the added flarum.reg.add(...) lines that were added by our loaders. This then allows us to return the actual asked for module to the caller.
We use a wrapper method to call webpack's interal chunk loader, and selectively return the wanted module like so:
async asyncModuleImport(path: string): Promise<any> {
const [namespace, id] = this.namespaceAndIdFromPath(path);
const module = this.chunkModules.get(`${namespace}:${id}`);
if (!module) {
throw new Error(`No chunk found for module ${namespace}:${id}`);
}
// @ts-ignore
const wr = __webpack_require__;
return await wr.e(module.chunkId).then(() => {
// Needed to make sure the module is loaded.
// Taken care of by webpack.
wr.bind(wr, module.moduleId)();
const moduleExport = this.get(namespace, id);
// For consistent access to async modules.
moduleExport.default = moduleExport.default || moduleExport;
return moduleExport;
});
}
Backend Compilation
Current Revision/Less/Js compiler mechanism
Before explaining the new changes, let's add some context as to how the current backend compilers work.
Each revision compiler has source files that must be compiled into one single final file, which is then saved with a revision hash to the revision map. This is very simplified, but the main idea is that each of the current compilers must output a single file.
Our new use case here is that chunk files must be copied as is. An extension/package might have many chunks, so we will be using a new JsDirectoryCompiler. It will be given a destination path and -new- directory sources. For each chunk within directory sources, it will use a JsCompiler to copy the chunk, which will reuse logic to store a revision hash and a map file.
To declare chunks for a frontend, extensions will do the following:
return [
(new Extend\Frontend('forum'))
->jsDirectory(__DIR__ . '/js/dist/forum'),
];
New common frontend for assets
Some split chunks may come from the common directory in the JS side, rather than go down a rabbit hole of letting that be duplicated into each frontend's directory (forum or admin), we now define a simple common assets factory, which compiles all common split chunks from packages into the appropriate common assets path.
Revision Hashes
The JsDirectoryCompiler uses a JsCompiler for each file, so the revision of each chunk will be taken care of there. rev-manifest.json will have the revisions we need for the chunks.
We do need these to be exposed to the frontend, rather then fetch the public rev-manifest.json file, we can inlude it in the initial payload for the app to read. Then when building the chunk URL, we can fetch the hash.
chunkUrl(chunkId: number | string): string | null {
const chunk = flarum.reg.getChunk(chunkId.toString());
if (!chunk) return null;
this._revisions ??= JSON.parse(document.getElementById('flarum-rev-manifest')?.textContent ?? '{}');
const path = `${this.forum.attribute('jsChunksBaseUrl')}/${chunk.namespace}/${chunk.urlPath}.js`;
// The paths in the revision are stored as (only a portion of jsChunksBaseUrl) + the backend path.
const assetsPath = this.forum.attribute<string>('assetsBaseUrl');
const key = path.replace(assetsPath, '').replace(/^\//, '');
const revision = this._revisions[key];
return revision ? `${path}?v=${revision}` : path;
}
Frontend usage
Splitting chunks
It is as simple as doing:
const MyModule = await import('./MyModule');
Webpack will auto compile that to the right path, which would either be js/dist/forum, js/dist/admin or js/dist/common.
The backend must be aware of this, so depending on which frontend has split chunks:
return [
(new Extend\Frontend('forum'))
->jsDirectory(__DIR__.'/js/dist/forum');
]
Splitting modals or pages
In addition to the above, the modal manager and the route resolver can both handle a callback that returns a promise.
import Extend from 'flarum/common/extenders';
export default [
new Extend.Routes()
.add('tags', '/tags', () => import('./components/MyPage')),
];
Importing a split module from another extension/core
You might need a module from another package (core or an extension) which is part of a split chunk. In that case you can just use the same import format.
// From core
const LogInModal = (await import('flarum/forum/components/LogInModal')).default;
// From another extension
const TagSelectionModal = (await import('ext:flarum/tags/common/components/TagSelectionModal')).default;
Extending & Overriding a lazy loaded module.
When using extend or override, if the target module is a lazy loaded one (a split chunk that is yet to be loaded). Instead of directly specifying the prototype, pass in the full import path:
Webpack Configuration
To allow us core/extension developers to easily split chunks, our webpack configuration will have to do the following:
autoChunkNameLoaderWith the given codes example:
A new loader will first find all async import instances and transform those snippets by adding the appropriate chunk name/path and any other configuration we might want in the future.
This will compile the
LogInModalmodule intodist/forum/components/LogInModal.js(+ its.mapfile).RegisterAsyncChunksPluginWhen hitting the backend for the chunk, we need to tell which extension/package the chunk belongs to to avoid conflicts in paths. To do that, a new plugin will pick up on any split chunks and register the chunk ID, concaednated module ID, with its path and extension ID to the flarum export registry (#3833).
So, for example, in the module where the code above resides, a line will be added at the end:
Why the chunk ID and the module ID?
Chunk ID
In order to determine the URL of the chunk to import, we replace webpack's async loader function, the function takes a chunk ID as a parameter, we need to be able to map that to a path and extension ID, so that when constructing the URL we can do:
Where in our example, the namespace would be
coreand the urlPath would beforum/components/LogInModal. the final URL would behttp://flarum.lan/assets/js/core/forum/components/LogInModal.jsModule ID
Because webpack is responsible for chunking, and it does so in its own format/way, when manually retrieving a split module from chunk X (for example extension A async importing the split LogInModal). We need to call the chunk callback that will return the module, and therefore execute the added
flarum.reg.add(...)lines that were added by our loaders. This then allows us to return the actual asked for module to the caller.We use a wrapper method to call webpack's interal chunk loader, and selectively return the wanted module like so:
Backend Compilation
Current Revision/Less/Js compiler mechanism
Before explaining the new changes, let's add some context as to how the current backend compilers work.
Each revision compiler has source files that must be compiled into one single final file, which is then saved with a revision hash to the revision map. This is very simplified, but the main idea is that each of the current compilers must output a single file.
Our new use case here is that chunk files must be copied as is. An extension/package might have many chunks, so we will be using a new
JsDirectoryCompiler. It will be given a destination path and -new- directory sources. For each chunk within directory sources, it will use aJsCompilerto copy the chunk, which will reuse logic to store a revision hash and a map file.To declare chunks for a frontend, extensions will do the following:
New
commonfrontend for assetsSome split chunks may come from the common directory in the JS side, rather than go down a rabbit hole of letting that be duplicated into each frontend's directory (forum or admin), we now define a simple
commonassets factory, which compiles allcommonsplit chunks from packages into the appropriatecommonassets path.Revision Hashes
The
JsDirectoryCompileruses aJsCompilerfor each file, so the revision of each chunk will be taken care of there.rev-manifest.jsonwill have the revisions we need for the chunks.We do need these to be exposed to the frontend, rather then fetch the public
rev-manifest.jsonfile, we can inlude it in the initial payload for the app to read. Then when building the chunk URL, we can fetch the hash.Frontend usage
Splitting chunks
It is as simple as doing:
Webpack will auto compile that to the right path, which would either be
js/dist/forum,js/dist/adminorjs/dist/common.The backend must be aware of this, so depending on which frontend has split chunks:
Splitting modals or pages
In addition to the above, the modal manager and the route resolver can both handle a callback that returns a promise.
Importing a split module from another extension/core
You might need a module from another package (core or an extension) which is part of a split chunk. In that case you can just use the same import format.
Extending & Overriding a lazy loaded module.
When using
extendoroverride, if the target module is a lazy loaded one (a split chunk that is yet to be loaded). Instead of directly specifying the prototype, pass in the full import path:As soon as the module has been loaded, your extend/override logic will be applied.