ci: merge staging to master

[MatrixAI-Bot]

This is an automatic PR generated by the pipeline CI/CD. This will be automatically fast-forward merged if successful.

[MatrixAI-Bot]

Pipeline Attempt on 593247826 for 349927b1ecfd829827e19d85d7f94a59f378731d

https://gitlab.com/MatrixAI/open-source/js-logger/-/pipelines/593247826

[ghost]

👇 Click on the image for a new way to code review

- Make big changes easier — review code in small groups of related files - Know where to start — see the whole change at a glance - Take a code tour — explore the change with an interactive tour - Make comments and review — all fully sync’ed with github [Try it now!](https://app.codesee.io/r/reviews?pr=21&src=https%3A%2F%2Fgithub.com%2FMatrixAI%2Fjs-logger)

Legend

[CMCDragonkai]

This failed on windows because tag pipeline which uses its own cache was attempting to install from chocolatey. However chocolatey is currently offline: https://community.chocolatey.org/packages/nodejs.

I realised that the cache is not shared between pipelines here. Each tag has its own CI_COMMIT_REF_SLUG. This is the branch or tag name. So for a branch like staging, then all the jobs share the same cache for staging, and also subsequent pipelines do as well. However tag pipelines have their own tag name and it's unique each time. So there's an empty cache for every tag pipeline.

Ideally the tag pipelines should cache and share their cache between tag pipelines as well. It sort of depends, since a tag doesn't correspond to any branch, but conventionally we create tags off the staging branch.

[CMCDragonkai]

This also means every new branch like feature branches have their own cache as well which is not as bad because it only runs the check stage. But still it would be good to share "feature" branch caching for feature branches.

[CMCDragonkai]

At the same time different runners have different caches. I believe windows and macos have their own caching that is separate.

Ok one solution to this is to use a fallback key.

The fallback key is used whenever a cache for the main key is not found. So for a new tag, no cache will be found always at the beginning.

It can then fallback on a key designed to cache "globally", for the tag pipelines, and also for new feature branches.

Actually I'm not sure if the fallback key would work here. If we fallback to a key, does it also push cache to the fallback key or to the main key?

It would make sense to push to the main key even if the fallback key was used. That way branches can then use their own cache separately from the fallback key.

But that would mean no job would ever push to the fallback key.

[CMCDragonkai]

Hi,

I need some clarification about this.

1. I use $CI_COMMIT_REF_SLUG which is the branch or tag name as the cache key. Meaning all jobs share the same cache within the same branch or tag name. The end result is that for a given branch staging, the final key will be staging-protected.

2. I was under the impression that the windows and macos runners would have separate caches, even if they are using the same $CI_COMMIT_REF_SLUG. Am I wrong about this? Are the windows and macos sharing the same cache that the linux jobs are using as well? If this is the case, then indeed it is possible that something might be wrong in the cache, as the 3 OSes should not have the same cache data.

3. If I need to separate windows and macos out, do I need to add an extra variable to the cache key, and if so, what would you suggest to get the runner platform?

4. I recently discovered the CACHE_FALLBACK_KEY which could be used for dealing with the fact that new feature branches and tags would start off with no cache because nothing has been cached for their $CI_COMMIT_REF_SLUG. If I bring this in and make it fallback on staging like CACHE_FALLBACK_KEY: "$CI_DEFAULT_BRANCH, I can imagine a couple problems here. Firstly there's no -protected suffix appended to the fallback key. Secondly nothing is pushing to this cache, I assume that it still continues to push to the main cache key. Finally if I do set it explicitly to CACHE_FALLBACK_KEY: "staging-protected" doesn't this end up allowing unprotected branches & tags to access the cache of the staging branch which is a security risk. What is the right way to use this, if I only want protected references (new feature branches and new tags) to load their cache from staging-protected?

Thanks,

Roger

[MatrixAI-Bot]

Pipeline Succeeded on 593247826 for 349927b1ecfd829827e19d85d7f94a59f378731d

https://gitlab.com/MatrixAI/open-source/js-logger/-/pipelines/593247826

[MatrixAI-Bot]

Pipeline Attempt on 593417996 for 9ec21de9a42d65d2a65da2133acf6ef320768b17

https://gitlab.com/MatrixAI/open-source/js-logger/-/pipelines/593417996

[MatrixAI-Bot]

Pipeline Succeeded on 593417996 for 9ec21de9a42d65d2a65da2133acf6ef320768b17

https://gitlab.com/MatrixAI/open-source/js-logger/-/pipelines/593417996