feat: accept multiple directories with destination for each

[OmarIthawi]

Description

Remove the --directory argument in favor of the positional arguments DIRECTORY ... to allow multiple directories to be specified in a single step. This is useful for pulling a huge batch of XBlocks into edX Platform or components in MFEs in a single checkout operation.

Breaking Changes and Developer Experience

One of the primary goals of the project is to avoid breaking changes. However, this pull request breaks compatibility with current (if any) usage of atlas due to the change from --directory argument to DIRECTORY... positional argument.

The main driver behind this change is to allow breaking the command into multiple lines, in which the --directory argument wouldn't support:

$ atlas pull translations/frontend-app-learning/:learning-mfe \
            translations/frontend-component-header/src/i18n/messages:frontend-component-header

Testing

• [ ] Fix all tests after the initial review is passed
• [ ] Add additional tests for new use cases

Argument description

DIRECTORY... represents one or more directory map pair separated by a colon (:) e.g. FROM_DIR:TO_DIR:

• The first directory (FROM_DIR) represents a directory in the git repository.
• The second directory (TO_DIR) represents a local directory to copy files to.
• At least one directory pair is required.
• This syntax is inspired by docker --volume from_dir:to_dir mounting syntax.

Design

In order for this to work, an extensive "matrix" of git sparse-checkout rules are added to filter for each DIRECTORY and --filter:

# Include directories that matches the language pattern e.g. `/ar/`
git sparse-checkout add "$directory_from/**/${one_filter}/**"
# Include files that matches the language pattern e.g. `ar.*`
git sparse-checkout add "$directory_from/**/${one_filter}.*"

For example, two directories and three languages would result into 12 rules.

 1. translations/frontend-app-learning/src/i18n/messages/**/fr_CA/**
 2. translations/frontend-app-learning/src/i18n/messages/**/fr_CA.*
 3. translations/frontend-app-learning/src/i18n/messages/**/ar/**
 4. translations/frontend-app-learning/src/i18n/messages/**/ar.*
 5. translations/frontend-app-learning/src/i18n/messages/**/es_419/**
 6. translations/frontend-app-learning/src/i18n/messages/**/es_419.*
 7. translations/frontend-component-footer/src/i18n/messages/**/fr_CA/**
 8. translations/frontend-component-footer/src/i18n/messages/**/fr_CA.*
 9. translations/frontend-component-footer/src/i18n/messages/**/ar/**
10. translations/frontend-component-footer/src/i18n/messages/**/ar.*
11. translations/frontend-component-footer/src/i18n/messages/**/es_419/**
12. translations/frontend-component-footer/src/i18n/messages/**/es_419.*

Example call

$ cd frontend-app-learning/src/i18n/messages
$ atlas pull --filter=fr_CA,ar,es_419 \
            translations/frontend-app-learning/src/i18n/messages:frontend-app-learning \
            translations/frontend-component-header/src/i18n/messages:frontend-component-header

Will result in the following tree:

├── frontend-app-learning
│   ├── ar.json
│   ├── es_419.json
│   └── fr_CA.json
└── frontend-component-header
    ├── ar.json
    ├── es_419.json
    └── fr_CA.json

References

This pull request is part of the FC-0012 project which is sparked by the Translation Infrastructure update OEP-58.

Check the links above for full information about the overall project.

Internalization is being rearchitected in Open edX Python, XBlock, Micro-frontend, and other projects. There are a number of immediately visible changes:

• Remove source and language translations from the repositories, hence no .json, .po or .mo files will be committed into the repos.
• Add standardized make extract_translations in all repositories
• Push user-facing messages strings into openedx/openedx-translations.
• Integrate root repositories with openedx/openedx-atlas to pull translations on build/deploy time

[openedx-webhooks]

Thanks for the pull request, @OmarIthawi! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[openedx-webhooks]

@OmarIthawi 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.