[DEPR]: Dockerfiles and Docker images

kdmccormick commented 1 month ago

Proposal Date

2024-05-14

Target Ticket Acceptance Date

2024-06-14

Target Removal Date

2024-12-14

Earliest Open edX Named Release Without This Functionality

Earliest release without Dockerfiles: Teak Earliest release without hosted images: Lilac (the project stopped tagging images when Tutor became the default)

Rationale

Upstream Open edX Django services have Dockerfiles, like this one in edx-platform. These were added according to OEP-45 Configuring and Operating Open edX, accepted in 2020. These Dockerfiles are generally built upon push-to-main and uploaded to the Open edX DockerHub account.

However, OEP-45 has not become reality. Instead of switching to the new Dockerfiles in the upstream Open edX repositories, the community decided to support to Tutor, which has its own Jinja2-rendered Dockerfiles, like this one for edx-platform.

So, as of 2024, the Dockerfiles are not used and tested by the community. As far as we know, they are used by the unsupported Devstack and (in some cases) in 2U's production environment. We do not know of any other production users. If you do use these Dockerfiles in production, please let us know!

These images are currently being built using the same resource pool that checks for Open edX PRs use. They impact Open edX upgrades, and their presence gives the impression to potential users that they are supported and possibly production-ready. If community members feel that it's important to maintain and continue building these Dockerfiles, then that's a route we can take--but at the moment, it seems like it is not, so I propose removing them.

Removal

Remove all Dockerfiles in the Open edX organization.
- Open question: Some repos use Dockerfiles for CI. Should we move those repos to standard images for CI?
Remove miscellaneous settings files, artifacts, etc. which are only used by the Dockerfiles
Remove all docker-compose artifacts which depend on the Dockerfiles
Freeze or deactivate the Open edX DockerHub account.
- Open question: Would it be better to make the image repositories read-only, or actually delete them?

Replacement

Tutor provides production-ready Docker images for all supported Open edX services.
Tutor can also be used just to generate prod-ready Dockerfiles (tutor config save && cd "$(tutor config printroot)/env/build") which can be used as a reference for anyone who wants to run Open edX on Docker without Tutor.
Each Open edX service repository should contain documentation describing how it can be installed and executed, allowing anyone to write a Dockerfile that provisions the repository. For example, here are the instructions in edx-platform. If a repository does not have these instructions, that it is a documentation problem that should be brought to the attention of the repository's maintainer by opening an issue

Deprecation

The Open edX DockerHub account already describes itself as unsupported.

Migration

Folks who currently rely on the Docker images will need to:

Move the Dockerfiles they need to a repository outside of the openedx organization
Build them on a regular cadence and push them somewhere other than Open edX's DockerHub account
Update any references to docker.io/openedx images to the new location

Additional Info

This DEPR would move us further away from ever implementing OEP-45. There are other issues with OEP-45, notably that YAML-based configuration was never built out enough to be viable for all providers. Based on whether this DEPR is accepted, it seems like perhaps we should mark the OEP as obsolete.
This overlaps with the pending Devstack cleanup: https://github.com/openedx/public-engineering/issues/247

Task List

No response

kdmccormick commented 1 month ago

https://discuss.openedx.org/t/deprecation-removal-dockerfiles-and-docker-images/13003

justinhynes commented 1 month ago

Hey there! I just wanted to reach out and speak for 2U. I know this would affect our team, and speculate it would affect other 2U teams too.

Personally, I understand and agree with the removal of the Dockerfiles - especially if they have little value to the community and are primarily used by 2U. I'm more worried about the timeframe for removal. If accepted, is there any way to ensure that we get a bit of runway to figure out a path forward before the Dockerfiles are removed from the various repos?

kdmccormick commented 1 month ago

Hey @justinhynes , if this is accepted, I can certainly make sure there's a runway for you folks. 6/14 is the proposed acceptance date but that doesn't have to be the removal date. Let me know what you are thinking for a timeline! Same goes for any 2U team depending on these Dockerfiles.

justinhynes commented 1 month ago

@kdmccormick sorry for the slow reply. June 14th being the acceptance date of the proposal totally makes sense.

When it comes to the timeline, I'm trying to get you an answer. I'm talking with some folks here to try and figure this out, but I was just sharing a fear of the removal work being picked up right away after acceptance :joy:. I hope that makes sense!

I hope to get back to you with some useful information some point soon haha.

kdmccormick commented 3 weeks ago

I've updated the ticket description to include a 6-month grace period between acceptance (14 Jun) and removal of the dockerfiles and images (14 Dec). Please let me know if you feel that the timeline should be shorter or longer.

I'm also opened to a phased approach. In particular, disabling the edx-platform image build+push would have the most immediate benefit on CI speed. If that could be expedited, then I'm open to pushing back the other removal steps, especially those concerning repositories outside of edx-platform.

kdmccormick commented 2 weeks ago

This is now Accepted. Removal will begin as early as Dec 14.

openedx / public-engineering