search

EVerest / everest-demo

EVerest demo: Dockerized demo with software in the loop simulation
Apache License 2.0
14 stars 15 forks source link

Refactor docker images in everest-demo #65

Open andistorm opened 3 months ago

andistorm commented 3 months ago

In the context of the general refactoring of docker images in EVerest, it would be good to refactor the docker images in the everest-demo, too.

Proposed Changes

Further Thoughts

Versioning Strategy

There are pro and cons for both strategies. The current strategy is more flexible, as it allows to release a new version of the docker images without releasing a new version of the everest-demo repository. This can be useful if only the docker images are changed, but not other things in the everest-demo repository. On the other hand, the current strategy can lead to confusion, as the version of the docker images is not directly visible in the everest-demo repository. Using the strategy from everest-ci means to release a new version of each docker image whenever a new version of the everest-demo repository is released. This can be more consistent, but can also lead to unnecessary version bumps of the docker images.

Steve Docker Image

I believe the two images can be merged easily, since the only difference I could find is the additional README.md in everest-demo. The question is where to put the merged Dockerfile. I would suggest to put it in everest-dev-environment and use it from there in everest-demo. This would also make it easier to maintain the image, as it would only be in one place.

andistorm commented 3 months ago

@shankari, could you please review the proposed changes? I would greatly appreciate your comments and thoughts on them.

shankari commented 3 months ago

@mukuflash03 @louisg1337 can you please add your initial comments to this, including exploring the everest-dev-environment and how it is similar or different? I will get to this early next week and will use your comments to focus my feedback.

louisg1337 commented 3 months ago

@andistorm really great ideas, heres my thoughts!

  • Use Reusable Workflow/Custom Github Action from everest-ci to deploy everest-demo related docker images

I think this could definitely be a great idea to have some type of common repository for our docker build scripts. However, it may be a bit challenging as a lot of the yml build files in everest-demo are tailored specifically to whichever demo we are running. If there is a nice way to abstract these build files out while still retaining the flexibility to alter the script to our different needs, that would be great. Also, if we were to go with putting all the scripts in everest-ci, I think some more documentation in that repo would be super helpful.

  • Adapt versioning strategy to use the same versioning strategy as everest-ci Currently there is a .env file in the everest-demo repository that defines the version of the docker images. In everest-ci the docker image versions are synced to the version of the everest-ci repository. We should unify the versioning strategy across the EVerest repositories.

This totally makes sense, with the current way its done its not super apparent what version the docker images are using, so doing a release based strategy would definitely help with transparency. Although, I do feel like there are a lot of hot fixes made in this repository so doing releases could be a bit excessive, but I feel like transparency may trump that.

  • The steve docker image is very similar to the one in everest-dev-environment (Currently only visible in feature branch, moved from everest-utils). It would be good to merge these two images into one.

I think this would be a great idea, the files look identical. Here is the link to the everest-dev-environment StEVe Dockerfile.

  • Reuse the ghcr.io/everest/everest-dev-environment/mosquitto image in everest-demo instead of building it again.

I'm not too familiar with mosquitto / why we are rebuilding it, so I think @shankari is better suited to answer this.

  • Use only docker images from ghcr.io/everest in everest-demo

I agree, we should eventually end up only using everest docker images. However, I think this was a one off because @shankari had the Accenture team (US-JOET) focus in on a few smart charging functionalities so that she could showcase EVerest's capabilities at the CharIn Testival. Once those changes from Accenture get merged into everest repositories, I think this could probably go back.

MukuFlash03 commented 2 months ago

Hi @andistorm , You've come up with some good points.

The current strategy is more flexible, as it allows to release a new version of the docker images without releasing a new version of the everest-demo repository. This can be useful if only the docker images are changed, but not other things in the everest-demo repository

The current versioning strategy is what I tried understanding as well in the latest PR I am working on.

This PR #10 contains a detailed discussion on the usage of the .env file, how and when we update images on tag updates, as one of the possible workflow trigger events.

The GitHub Actions cicd.yaml workflow has been updated to rely on the .env file for version information. The intent here is to simplify our existing build processes, which were put in place to address a chicken-and-egg problem that occurs in the repository because (a) our Docker Compose files are stored in the same repository as our Dockerfiles and (b) we don't want the Docker Compose files used in our demos to require the user to go through long image build processes themselves.

A .env file is now in the root directory of the repository. For each commit on main, this is intended to contain the semver of the first release containing the changes in that commit. Example 1: In the commit tagged as release v0.0.3, the .env will contain 0.0.3. Example 2: For all subsequent commits up to and including the commit tagged as v0.0.4, .env should contain 0.0.4.

Reuse the ghcr.io/everest/everest-dev-environment/mosquitto image in everest-demo instead of building it again.

everest-demo GHCR, everest-dev-environment GHCR

That's probably a good idea. Since, I do see that the latest updates to the mosquitto image were made in the everest-dev-environment repo (1 week ago) and not in the everest-demo repo (4 months ago).

MukuFlash03 commented 2 months ago

Updating my previous comments regarding the mosquitto image.

Initially I had thought that everest-dev-environment repo had the latest changes to the mosquitto image. However, these changes were made by @andistorm in just copying over the mosquitto image from everest-utils which in fact are 3 years old.

The most recent mosquitto image is indeed the one in everest-demo. Hence we build this image and push it to ghcr.io/mqtt-server and are using this image currently.

shankari commented 2 months ago

@MukuFlash03 but IIRC the mosquitto-server is not a part of EVerest; it is pulled in from a standard image. The only changes are that we make (at least in the demo) is to add a new configuration. So we don't really need to update, unless we want to switch to the new image.

You should verify what (if anything) we change in there