Spike: A monorepo for multiple XBlocks

[feanil]

We have a lot of XBlocks and maintaining them all in separate repos can be expensive. Is it more efficient if they are all in one repo? What changes in tooling would we need to keep them up-to-date and deploy easily?

We can start with the following 2 XBlocks and merge them in a new GitHub repo https://backstage.openedx.org/catalog/default/component/DoneXBlock https://backstage.openedx.org/catalog/default/component/FeedbackXBlock

Tasks:

• [ ] Study the Xblock architecture
• [ ] Study DoneXblock code
• [ ] Study FeedbackXBlock code
• [ ] Figure out what can be common in both and what should kept separate
• [ ] Do some code for the POC

[feanil]

@farhan the goal is not to combine the code of the various xblocks, we want a single repo that houses multiple XBlocks but the code of each xblock is still independent. Things we would have to figure out:

• What kind of tooling would we need in terms of github actions to keep all the requiremnets for all the xblocks up to date?
• How could we publish new versions just for the xblocks we care about?

[irtazaakram]

What kind of tooling would we need in terms of github actions to keep all the requiremnets for all the xblocks up to date? How could we publish new versions just for the xblocks we care about?

In terms of changes required to manage updates & publish release we only need to make few changes

For package upgrades we only need to change working directory via GitHub actions to run make upgrade command. .github/workflows/upgrade-python-requirements.yml

call-upgrade-python-requirements-workflow:
    runs-on: ubuntu-20.04
    strategy:
      fail-fast: false
      matrix:
        dir: ['DoneXBlock', 'FeedbackXBlock']
    steps:
      - name: setup python
        uses: actions/setup-python@v5
        with:
          python-version: 3.8

      - name: make upgrade
        working-directory: ${{ matrix.dir }}
        run: |

For package releases we also just have to add directories in GitHub actions. .github/workflows/pypi-release.yml

push:
    runs-on: ubuntu-20.04
    strategy:
      fail-fast: false
      matrix:
        dir: ['DoneXBlock', 'FeedbackXBlock']
.
.
.
- name: Publish to PyPi
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          packages-dir: ${{ matrix.dir }}
          user: __token__
          password: ${{ secrets.PYPI_UPLOAD_TOKEN }}

These changes would allow separate workflows for each XBlock directory to update and release.

We have a lot of XBlocks and maintaining them all in separate repos can be expensive. Is it more efficient if they are all in one repo?

1- Every time there's a change in one xblock all ci tests needs to cleared making it more expensive then running individual repos.

2- Releasing a new version of each XBlock is going to be very problematic as we would need to release all XBlocks every time we need to release only one as GitHub's release flow only doesn't support monorepo.

Let me know if I am missing something.

Thanks,

[feanil]

Thanks @irtazaakram, it seems like the shared release numbers would complicate semantic releases for the repos but other than that this is pretty doable and we might want to follow this strategy in the future for a bunch of stable core XBlocks. If we had independent release numbers, we would have to move away from the github release mechanisms and perhaps also tag it with some xblock specific prefix to be able to parse through the tag history more easily.

From a maintainer perspective we'd want these to be pretty stable as well since it would be a lot more difficult to manage if many of these xblocks were in active development and we needed to have different groups manage different blocks.

@e0d for your reference see Irtaza's comment about with his findings. I think we could pursue this for some xblocks in the future but since we don't yet have a good perspective on what xblocks are core product/stable, it may be a bit of churn to make this change now.