Test Verdaccio plugin for build improvements

DawoudSheraz commented 3 months ago

Announced on https://discuss.openedx.org/t/local-npm-registry-for-mfe-builds/13526, Maksim from Raccoon Gang shared a PoC on using Verdaccio as a npm registry. Tutor-mfe build causes npm cache and other issues consistently. The PoC plugin is an attempt to pave to path for having a dedicated registry meant for npm operations that can be expanded to Tutor officially. In Tutor Users meet on July 29, it was mentioned that Tutor Maintainers will test the plugin out and share their feedback https://openedx.atlassian.net/wiki/spaces/COMM/pages/3583016961/Tutor+Users+Group#2024-07-29. This ticket aims to cover that testing. Once tested, the stats will be shared on the discuss link.

DawoudSheraz commented 3 months ago

I have max-parallelism=2 set for the builds. Here are the stats so far:

Verdaccio enabled, max2cpu buildx config:

Mfe build (no options, some existing cache?) — 866.7s
tutor images build mfe --no-cache --no-registry-cache — 1789.9s
tutor tutor images build mfe --no-cache — 880.1s
tutor images build mfe --no-registry-cache — 649.4s
Tutor images build mfe (cache filled) — 766.2s

Verdaccio disabled, max2cpu config

MFE build (no options, some existing cache?) — 828.2s
tutor images build mfe --no-cache --no-registry-cache — 1481.8s

I have yet to do more builds with verdaccio disabled.

The plugin did not work out of the box. I had to change tutor version constraint and remove network:host from docker-compose to make it work on Mac.

Danyal-Faheem commented 3 months ago

Here are my stats for this testing:

With Max-parallelism = 2:

tutor images build mfe --no-cache (Verdaccio empty): 1438.9 secs
tutor images build mfe --no-cache (Verdaccio filled): 1271.7 secs
tutor images build mfe --no-cache (Verdaccio disabled): Crashed with network timeout

With Max-parallelism = 1:

tutor images build mfe --no-cache (Verdaccio disabled): 2107.8s

Note: I had to remove network mode: host from the docker-compose configuration for Verdaccio to get it working on MacOS.

hinakhadim commented 3 months ago

I also had to remove "network_mode:host" from docker-compose and I was also working on MacOS. Testing on my system results in:

Verdaccio	No of CPUs (parallelism)	Command	No. of MFEs	Verdaccio Cache	Time (in seconds)	Notes
Enabled	2	build mfe --no-cache	3	Empty	816
Enabled	2	build mfe --no-cache	3	Filled	841
Enabled	2	build mfe --no-cache	7	Empty	1728.9
Enabled	2	build mfe --no-cache	7	Filled	1839.2
Enabled	2	build mfe --no-cache --no-registry-cache	7	Filled	1906.5
Enabled	2	build mfe	7	Filled	1843.6	[docker cache is empty for mfe image here. Registry cache is filled here]
Enabled	2	build mfe	7	Filled	10	[docker cache is filled for mfe image here. Registry cache is also filled here]
Disabled	2	build mfe --no-cache	7		1850
Disabled	2	build mfe --no-cache --no-registry-cache	7		1933
Disabled	2	build mfe	7		9.1	[docker cache is filled for mfe image here. Registry cache is also filled here]

My conclusion:

I did not test with different levels of parallelism because I believe parallelism is a separate factor that affects build time independently. The impact of parallelism is not related to the Verdaccio cache, as the results consistently show for a constant level of parallelism.
The differences in build times are minimal, with only about a 100-second variation between the different configurations. This suggests that the impact of Verdaccio on build performance is negligible, indicating that its benefits may be minimal in this context.

DawoudSheraz commented 3 months ago

Next steps:

Use verdaccio but do not set max-parallelism. Using verdaccio should not cause any connection errors that we have seen with mfe builds in the past.

regisb commented 3 months ago

Are you guys quite sure that you were hitting verdaccio in your tests, and not the remote NPM registry? The fact that Verdaccio is slower when its cache is filled is perplexing...

Also, the Verdaccio plugin is definitely not working for me on Linux. I get:

 > [learner-dashboard-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --registry=http://host.docker.internal:4873:
82.54 npm ERR! code ENOTFOUND
82.54 npm ERR! syscall getaddrinfo
82.54 npm ERR! errno ENOTFOUND
82.55 npm ERR! network request to http://host.docker.internal:4873/npm/-/npm-8.19.4.tgz failed, reason: getaddrinfo ENOTFOUND host.docker.internal
82.55 npm ERR! network This is a problem related to network connectivity.
82.55 npm ERR! network In most cases you are behind a proxy or have bad network settings.
82.55 npm ERR! network 
82.55 npm ERR! network If you are behind a proxy, please make sure that the
82.55 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
82.55 
82.55 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-08-14T11_17_44_546Z-debug-0.log
------
Dockerfile:397

Turns out that host.docker.internal is not supported on Linux: https://stackoverflow.com/questions/48546124/what-is-the-linux-equivalent-of-host-docker-internal

EDIT: I think that the proper command to test optimizations with/without Verdaccio is:

tutor images build --docker-arg=--no-cache-filter=authn-common,authn-prod,account-common,account-prod,communications-common,communications-prod,course-authoring-common,course-authoring-prod,discussions-common,discussions-prod,gradebook-common,gradebook-prod,learner-dashboard-common,learner-dashboard-prod,learning-common,learning-prod,ora-grading-common,ora-grading-prod,profile-common,profile-prod mfe

DawoudSheraz commented 3 months ago

Are you guys quite sure that you were hitting verdaccio in your tests, and not the remote NPM registry? The fact that Verdaccio is slower when its cache is filled is perplexing...

Also, the Verdaccio plugin is definitely not working for me on Linux. I get:

 > [learner-dashboard-common 3/5] RUN --mount=type=cache,target=/root/.npm,sharing=shared npm clean-install --no-audit --no-fund --registry=http://host.docker.internal:4873:
82.54 npm ERR! code ENOTFOUND
82.54 npm ERR! syscall getaddrinfo
82.54 npm ERR! errno ENOTFOUND
82.55 npm ERR! network request to http://host.docker.internal:4873/npm/-/npm-8.19.4.tgz failed, reason: getaddrinfo ENOTFOUND host.docker.internal
82.55 npm ERR! network This is a problem related to network connectivity.
82.55 npm ERR! network In most cases you are behind a proxy or have bad network settings.
82.55 npm ERR! network 
82.55 npm ERR! network If you are behind a proxy, please make sure that the
82.55 npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
82.55 
82.55 npm ERR! A complete log of this run can be found in: /root/.npm/_logs/2024-08-14T11_17_44_546Z-debug-0.log
------
Dockerfile:397

Turns out that host.docker.internal is not supported on Linux: https://stackoverflow.com/questions/48546124/what-is-the-linux-equivalent-of-host-docker-internal

EDIT: I think that the proper command to test optimizations with/without Verdaccio is:

tutor images build --docker-arg=--no-cache-filter=authn-common,authn-prod,account-common,account-prod,communications-common,communications-prod,course-authoring-common,course-authoring-prod,discussions-common,discussions-prod,gradebook-common,gradebook-prod,learner-dashboard-common,learner-dashboard-prod,learning-common,learning-prod,ora-grading-common,ora-grading-prod,profile-common,profile-prod mfe

It did not work out of the box. During the build, the MFE container could not connect to Verdaccio container. Once network:host was removed from docker-compose, the connection established successfully. Verdaccio container logs showed whenever the registry was hit.

DawoudSheraz commented 1 month ago

After the last discussion in Tutor Users Group, it was decided that this should be tested without max-parallelism settings. I tried it a few times but the build would always kill after a while.

Max2cpu disabled, verdaccio enabled

MFE build
- Killed on first attempt — 345s
- Killed 2nd attempt 123s

Danyal-Faheem commented 1 month ago

I tried out the verdaccio plugin to build the MFE without max-parallelism and I faced a similar scenario as @DawoudSheraz. The image build process was killed everytime due to network errors.

hinakhadim commented 1 month ago

I also tried out multiple times without max-parallelism and faced errors in the image building process.

658.8 npm WARN deprecated @babel/plugin-proposal-object-rest-spread@7.20.7: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-object-rest-spread instead.
1117.0 npm notice 
1117.0 npm notice New minor version of npm available! 10.2.3 -> 10.8.2
1117.0 npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.8.2>
1117.0 npm notice Run `npm install -g npm@10.8.2` to update!
1117.0 npm notice 
1117.0 npm ERR! code FETCH_ERROR
1117.0 npm ERR! errno FETCH_ERROR
1117.0 npm ERR! network timeout at: http://host.docker.internal:4873/has-proto/-/has-proto-1.0.3.tgz
1117.1

DawoudSheraz commented 1 month ago

@cmltaWt0 FYI.

DawoudSheraz commented 1 month ago

For now, I am closing this issue. There is good enough context on this. In the future, if we want to take another dig at verdaccio testing, we can take this ticket as a base and plan the improvements accordingly.

mbtools commented 1 week ago

Hi Everyone,

I have done quite a bit of maintenance on Verdaccio recently and am curious about your tests. It might reveal areas that could be improved in Verdaccio. Could you share some details, please?

Which version of Verdaccio did you use?
How many packages were stored in Verdaccio at the end of the test?
How many packages were scoped (@.../...)?
Could you share the Verdaccio config file and a complete log of a test run?

Best, Marc

DawoudSheraz commented 6 days ago

@cmltaWt0 FYI regarding ☝🏽

overhangio / tutor-mfe

Test Verdaccio plugin for build improvements #220

My conclusion: