feat: support git v2.25.1 using `sparse-checkout set` instead of `sparse-checkout add`

OmarIthawi commented 1 year ago

Description

Support git v2.25.1 in which git sparse-checkout add sub command isn't available. This is a big pull request that ideally should have been a one-liner but bash and unit-testing made it huge:

-git sparse-checkout set --no-cone "!*"
-git sparse-checkout add "conf/locale/ar/"
-git sparse-checkout add "conf/locale/**/ar.*"
+git sparse-checkout set --no-cone '!*' 'conf/locale/ar/' 'conf/locale/**/ar.*'

This pull request introduces git_sparse_checkout_set which uses xargs to support directories with spaces, although this hasn't been tested.

Git versions

This pull request adds a GitHub Actions matrix with "default" and specific git versions in an attempt to match the corresponding devstack images below. The "default" git versions uses the latest/default git version from GitHub, which makes it easier to detect failures as git deprecates --no-cone mode.

Docker Image	Git Version	Support Status
edxops/credentials:latest	2.40.0	Tested ✅
openedx/lms-dev:latest	2.25.1	Tested ✅
edxops/discovery-dev:latest	2.25.1	Tested ✅
edxops/ecommerce-dev:latest	2.25.1	Tested ✅
openedx/edx-notes-api-dev:latest	2.25.1	Tested ✅
node:16	2.20.1	Not supported ❌
node:17	2.30.2	Tested ✅
node:18	2.30.2	Tested ✅

Not supporting node:16 and git v2.20.1

Some MFEs still have the nodejs v16 which includes git v2.20.1.

While git v2.20.1 internals have partial support for sparse checkout, the command git sparse-checkout was introduced only in v2.25+

Therefore, atlas will not support git versions below 2.25 and will produce an error:

# sparse-checkout subcommand isn't available in git 2.20.1
$ docker run -it bitnami/git:2.20.1 sparse-checkout
git: 'sparse-checkout' is not a git command. See 'git --help'.

# sparse-checkout subcommand is available in git 2.25.1
$ docker run -it bitnami/git:2.25.1 sparse-checkout
fatal: not a git repository (or any of the parent directories): .git

`--no-cone` mode deprecation

Git documentation says that --no-cone mode would be deprecated. Nevertheless we're taking a risk here until this is a problem in real world, after which atlas needs a refactor to use the state-of-the-art sparse-checkout mode in git in future versions.

Therefore the investment in testing to ensure that whenever this is a problem, atlas can be adapted to use alternative git features.

Performance degradation on `git v2.25.1`

There's a noticeable slowdown (many seconds) when using v2.25.1 in the initial git clone --depth=1 compared to more recent git versions like v2.40.1 which executes the commands almost instantly.

The end result of sparse-checkout is identical, therefore no further attempts to improve performance were carried on.

References

This pull request is part of the FC-0012 project which is sparked by the Translation Infrastructure update OEP-58.

Up-to-date project overview and details are available in the Approach Memo and Technical Discovery: Translations Infrastructure Implementation document.

Join the conversation on Open edX Slack #translations-project-fc-0012.

Check the links above for full information about the overall project.

Internalization is being rearchitected in Open edX Python, XBlock, Micro-frontend, and other projects. There are a number of immediately visible changes:

Remove source and language translations from the repositories, hence no .json, .po or .mo files will be committed into the repos.
Add standardized make extract_translations in all repositories
Push user-facing messages strings into openedx/openedx-translations.
Integrate root repositories with openedx/openedx-atlas to pull translations on build/deploy time

openedx-webhooks commented 1 year ago

Thanks for the pull request, @OmarIthawi! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

supporting documentation
Open edX discussion forum threads
timeline information ("this must be merged by XX date", and why that is)
partner information ("this is a course on edx.org")
any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

OmarIthawi commented 1 year ago

@brian-smith-tcril this is a big update for atlas, but is probably the last one until git breaks the --no-cone mode -- hopefully not so soon.

openedx-webhooks commented 1 year ago

@OmarIthawi 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

openedx / openedx-atlas