CI/CD setup

[ajensen1234]

We want to be able to build on both Linux and Windows machines. Probably using GitHub actions for now.

[ajensen1234]

Need to figure out how to create a docker container that has all the libraries that we need and want for this. We also need to figure out how to set this for both windows and Ubuntu to do cross-platform development.

[ajensen1234]

@nicholasverdugo let me know if you find anything out about this \

[ajensen1234]

Actions are running nicely though! A new branch was created for testing out the automatic runners.

[ajensen1234]

https://github.com/Jimver/cuda-toolkit https://github.com/marketplace/actions/setup-opencv https://github.com/marketplace/actions/install-qt

This looks super promising for getting libraries to live in the github environemt n

[ajensen1234]

Because all the packages that we are using are open-source, maybe we could include them as submodules. This also means that we might be able to just create a new directory and build open-cv, etc at runtime with a basic script in github actions. That might be a major PITA though.

[ajensen1234]

(Sorry for all the comments in such a short amount of time)

I'm not sure exactly what the differences are for Docker for reproducible builds (https://sweetcode.io/using-docker-reproducible-build-environments/) and using docker as a GitHub actions backend for building.

Need to learn these differences because it seems like the main idea here is reproducible builds without needing to suffer through extensive library installation every time.

[ajensen1234]

Okay, I hope I've got the main idea of this down (happy to be corrected):

1. Create Docker images for both Linux and Windows that contain all the necessary libraries that we are using to build. This will actually help with local development as well because we won't need to worry about getting everything setup on all the computers every single time. Future generations won't be able to enjoy the painful process of compiling (maybe we have them do that a few times first before telling them about the docker image?).
2. Either post that docker image(s) to the hub, or have them as local Dockerfiles inside the repository. We would need to distinguish between windows and Linux files.
3. On GitHub actions, either create the Dockerfile via the action's matrix per-system, or pull the images from Docker hub per system to create the build environment that we need for each of the different OS.
4. Build normally using GitHub actions?

Some problems I'm foreseeing:

• We have some weird issues with libtorch not playing nice with slots definition, so we'd need to find a way to edit that file.
• I don't know if pushing that Dockerfile to the hub will be unreasonably large or take a long time. VTK and OpenCV both take around 30 minutes to build, seems excessive to have to wait that long to see if JTML is building correctly.

[ajensen1234]

https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/ https://stackoverflow.com/questions/36862589/install-opencv-in-a-docker-container

Another option that seems like it might work well for what we're trying to do.

[ajensen1234]

https://hub.docker.com/r/kitware/vtk https://hub.docker.com/r/opencvcourses/opencv-docker

[ajensen1234]

https://wiki.qt.io/Building_Qt_5_from_Git#Getting_the_source_code

[ajensen1234]

Here's something that @stefankiesz showed me: looks hugely promising.

https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/.github/workflows/ci.yml

[ajensen1234]

I don't think we'll be getting to this anytime soon. I considered having some build environment put into HPG, but I'd like to graduate, not take this to the moon.