Running bin/build-js.sh should not require a database connection

[mariusklocke]

If I understand correctly, bin/build-js.sh starts the javascript build process powered by npm and webpack. The script fails if you don't have var/plugins.json committed to VCS (it is gitignored by default). The file contains a list of activated plugins. It is generated by bin/console bundle:dump which requires an active database connection to retrieve a list of activated plugins.

I think the build process should not rely on the activation state of plugins. It should build JS for every plugin in the project.

[JoshuaBehrens]

Part of the challange is having a database-less runtime for the plugins as they can change the outcome of bundle:dump which influences the way how bin/build-js.sh - as you already stated - compiles files.

[icedream]

Commenting on this issue since with #87 and #89 the file var/plugins.json has been removed and ignored in the Git tree, now causing issues for Continuous Integration with building JavaScript via the build scripts without a database.

What is the current intended way of building the JavaScript bundle with Webpack as part of Continuous Integration? On our target servers we can't and don't want to compile the JavaScript in the production environment due to a missing Node.js installation and to avoid heavy load, and #87 and #89 removed the necessary var/plugins.json file for the build to work in an environment without a database.

[mariusklocke]

It is very sad that this issue has not addressed been in over a year now! The README of this project states very clearly that NodeJS is NOT required on production servers:

NPM and Node are only required during the build process and for development. If you don't have javascript customizations it's not required at all because the storefront and admin are prebuilt.

If you are using a separate build server, consider having NPM and Node as build-only requirements. Your operating application server doesn't require any of these to run Shopware 6.

How there is still no out-of-the-box solution to solve building custom JS. The core problem is that Shopware\Core\Framework\Plugin\BundleConfigGenerator relies on booted kernel with all bundles initialized.

Our workaround in GitLab CI (not using custom webpack config): PHP-Stage:

• spinup an empty MariaDB Container from official images
• composer install
• bin/console system:install
• bin/console plugin:refresh
• Determine a list of plugins with .js files using the filesystem
• Install and activate all plugins with .js files
• bin/console bundle:dump

NPM-Stage:

• take var/plugins.json from PHP-Stage
• bin/build-js.sh
• collect the generated JS files from plugins folder

On Production:

• take the artifacts from CI
• bin/console assets:install
• bin/console theme:compile

This is a very simplified version of what we actually built in tedious months of work. It works quite well, but the plugin-installation part can use some resources. Also the whole topic of plugin dependencies is totally uncovered since the beginning of Shopware 6. Composer is now first citizen. Why does Shopware Core not check if a plugin depends on the activation state of other plugins? This leads to plugin developers not understanding how to define dependencies. You can clearly see that when using different plugin from Shopware Store.

[JoshuaBehrens]

@mariusklocke we actually have the same routine in our CI. Only difference here is the reason why we did this: Plugins from the community store might be incompatible without webpack compiled files and thus breaks within the browser later on. So we had to recompile every plugin on CI level. But eventually. They are right, that production server do not need node. Still not a very good way to handle that right now

[shyim]

We want feedback :) https://twitter.com/Shyim97/status/1417098885160439808

[shyim]

See https://developer.shopware.com/docs/guides/hosting/installation-updates/deployments/build-w-o-db