The problem: NPM packages are installed into the openedx image at /openedx/edx-platform/node_modules. So, when you mount your own edx-platform, you override that node_modules folder. Now, you must re-install node_modules yourself (tutor dev run --mount=edx-platform lms npm install), which takes a long time and is completely redundant with the node_modules that you had to download when you downloaded the openedx image. If you forget to do this, then your edx-platform frontend will be broken.
The solution: Move node_modules somewhere else in the openedx image, such as /openedx/node_modules.
Tasks
[ ] In Tutor's Dockerfile, copy package.json and package-lock.json up to /openedx. Run npm install from within /openedx, causing the packages to be installed at /openedx/node_modules.
[ ] The edx-platform asset pipeline unfortunately needs to reach into node_modules and copy stuff out of it, notably in the case of node_modules dependencies that also need to be copied into "vendor" directories for use by non-Webpack frontend code. So: We need to update the assets pipeline to tolerate the fact that node modules have moved to /openedx/node_modules. As necessary, do one or more of these:
[ ] Edit edx-platform/pavelib/assets.py to be agnostic to the location of node_modules. Keep in mind that assets.py must continue working if node_modules is located inside the edx-platform directory as to not break Devstack.
[ ] Reimplement parts of edx-platform/pavelib/assets.py upstream as shell scripts, agnostic to the location of node_modules. Point both edx-platform/pavelib/assets.py and Tutor's openedx-assets at that reimplementation.
[ ] Reimplement more parts of edx-platform/pavelib/assets.py into Tutor's openedx-assets script, agnostic to the location of node_modules, but don't upstream the changes.
[ ] Add a section to the troubleshooting section along the lines of:
My NPM packages are out of date!
Are you bind-mounting edx-platform? If so: When you bind-mount edx-platform and it has a node_modules directory, then those node_modules will shadow any matching node_modules installed into the openedx image. So, if your edx-platform/node_modules is outdated, then the outdated packages will shadow the up-to-date packages on your image. To resolve this, simply remove your edx-platform/node_modules folder.
Notes
NPM checks first for node_modules in the current working directory, then checks the parent directory, then the grandparent directory, and so on. This is why we are able to locate NPM packages at /openedx/node_modules without totally breaking edx-platform. node_modules that exist in a mounted edx-platform
kdmccormick
commented
1 year ago
Background
This is a sub-task of https://github.com/openedx/wg-developer-experience/issues/146
The problem: NPM packages are installed into the openedx image at /openedx/edx-platform/node_modules. So, when you mount your own edx-platform, you override that node_modules folder. Now, you must re-install node_modules yourself (
tutor dev run --mount=edx-platform lms npm install), which takes a long time and is completely redundant with the node_modules that you had to download when you downloaded the openedx image. If you forget to do this, then your edx-platform frontend will be broken.The solution: Move node_modules somewhere else in the openedx image, such as /openedx/node_modules.
Tasks
npm installfrom within /openedx, causing the packages to be installed at /openedx/node_modules.npm clean-installstep from Tutor's dev env setup instructions.Notes
NPM checks first for node_modules in the current working directory, then checks the parent directory, then the grandparent directory, and so on. This is why we are able to locate NPM packages at /openedx/node_modules without totally breaking edx-platform. node_modules that exist in a mounted edx-platform