github action for docker build

[jeffeb3]

I am still using custompios, and the octopi work to make the v1pi images. It is working well. I basically ignore it most of the time, and then about every three months I come back and spin up a new pair of images from the scripts on my local machine.

My setup is still a bit brittle though. I am using my own docker setup, and it works on this machine, but it would be better if it was done nightly someplace.

I was thinking about working on some github actions, or a circleci, or travisci setup. But I don't want to reinvent the wheel. Do you (or any images you know) have a CI setup like that? I've used the github actions for another project, and circle CI for another. They both seem to work pretty well. I'm not sure if it is possible with the loop devices though.

[guysoft]

So what I use is Jenkins that calls docker-compose to build the image. That is what works for me.

I saw this guy adding github actions, and even asked him to share, but didn't get an answer from him: https://github.com/bitbyt3r/FullPageOS/commit/61df6d776bcf72a7280dc9f0c001cd831c1a2916

I would love to have CI/CD guides/sources. But it would require people to share what they are doing.

I can share parts of my setup, but I am not sure what would be easy to deploy.

[jeffeb3]

OK. I have been distracted since I asked. I haven't done anything on it. That link is useful. The script is all in github, thankfully. You can see the steps it takes here:

https://github.com/bitbyt3r/FullPageOS/blob/devel/.github/workflows/main.yml#L13

The first step is running the .sh script, which is here: https://github.com/bitbyt3r/FullPageOS/blob/devel/.github/workflows/main.yml#L13

So it isn't setting up docker to do the work. Github is running it somewhere, and they must be ok with the script calling sudo modprobe loop. You can see the results here: https://github.com/bitbyt3r/FullPageOS/actions

It looks like it's working, and it is working in some kind of virtual environment. I'm not sure if that means container or more full fledged environment, but if it's working for that project, it should probably work for any of ours.

[guysoft]

Cool, please document what you do. We could add it to the wiki Perhaps even I could move some builds there.

[guysoft]

@bitbyt3r also. If you can give any insight or documentation that would be great.

[bitbyt3r]

Sorry I lost track of your earlier request! It fell to the bottom of my TODO list when things got busy at work. I created a pull request to FullPageOS: https://github.com/guysoft/FullPageOS/pull/332

That should also be fairly generic, so it should be able to build similar projects based on CustomPiOS without changes.

I also created a draft pull request to add that action to v1pi, (https://github.com/jeffeb3/v1pi/pull/24) however it isn't working yet. I haven't figured out how to set OCTOPI_INCLUDE_HAPROXY correctly.

[bitbyt3r]

You can see the successful build for FullPageOS here: https://github.com/bitbyt3r/FullPageOS/actions/runs/133097110

And the most recent failed build for v1pi here: https://github.com/bitbyt3r/v1pi/actions/runs/133106807

I'll keep poking the build for v1pi for a bit and let you know if I figure out what's missing.

I figured out the issue and updated the pull requests.

[guysoft]

@jeffeb3 Any news? There is a PR but I am not sure I like the fact that only after you tag a new version you get a build, since you want to release after you built and tested an image.

[bitbyt3r]

I can change the rules for which commits trigger a build if you like. Would you like every push to trigger a new build? It wouldn't show up on the releases page, instead you'll have to download the build from the artifacts drop down on the action page.

[guysoft]

@bitbyt3r Yes that would be better. It would be scrabble and we could in the future post it to something like: https://gnethomelinux.com/OctoPi/nightly/ (broken will fix it). The code that generated it is here: https://github.com/guysoft/OctoPi/blob/devel/src/nightly_build_scripts/generate_nightly_page.js

[bitbyt3r]

Ok, I updated the pull request on FullPageOS to run a build on every push, and disabled creating releases automatically.

[guysoft]

Cool - it might be good to have a periodic nightly build too. @foosel mentioned that is possible. Current version: https://github.com/guysoft/FullPageOS/blob/devel/.github/workflows/main.yml

[jeffeb3]

Glad you're making progress. I have been... Distracted.

[jeffeb3]

I've been experimenting with the releases/artifacts to try to find a good balance on my MarlinBuilder project. Where I landed was that on release, I would push the artifacts to the release, but on pull request or on schedule (nightlies) I would only publish the artifacts to the actions. So you have to be a bit clever to find the artifacts, and the nightly builds, and I think you also have to be logged in. But when I want to make a release, or an RC, I would just create a release, with no assets, just the description, and that will trigger the builder to build everything, and publish to the release.

You can see how I did that here: https://github.com/V1EngineeringInc/MarlinBuilder/blob/main/.github/workflows/rambo.yml#L110

It's not a perfect example, but it seems like a good balance. I don't want 200 releases clogging the release page.

[jeffeb3]

I have v1pi working the way I want it to. You can see that here:

https://github.com/jeffeb3/v1pi/blob/master/.github/workflows/main.yml

Now that I am "done" with this part, and I want to do some development on the features of my image, I wish the different action steps were all separate modules.

So, right now, the action is just calling build_dist. Which works, but if there is a problem with some new module code I am testing, I just see that the build step fails, and I can't really tell where without looking through the 5k line log.

It would be neat if the steps in the github action were actually things like: mount image, start qemu, module1 start, module 2, ... module 1 cleanup, qemu cleanup, umount image...

Then I could easily see if the raspap module I'm working on is failing, and how it is failing.

I can't see a good way to do that though, without breaking down the build_dist, and reconstructing it in a workflow. I also don't see a good way to do that programatically, like for module in $MODULES... run.... I'm also not sure if I can leave the state of qemu in a way where the next step can be done in that qemu environment, and not just on the runner's command line.

Oh well, this is working. I just thought I would drop that thought here, in case someone knew how to do that, and how much we'd have to sacrifice to do that.

[guysoft]

just see that the build step fails, and I can't really tell where without looking through the 5k line log.

Related to #1

[guysoft]

It seems like there is a working one on the OctoPi github: https://github.com/guysoft/OctoPi/actions

I am not using it for nighties because I need them uploaded over FTP and such to mirrors. But it seems this issue is solved, but documentation should be added.

I think the only thing missing here is to extract the distro name that is hardcoded to "octopi": https://github.com/guysoft/OctoPi/blob/devel/.github/workflows/build.yml

[jeffeb3]

Just pointing to an example of two is a good first step in documentation. The users are going to be advanced users anyway.

I am happy with my workflow, although I am going to copy that apt fix. From my perspective, this is fine to close anytime.

[guysoft]

Done, thanks for all the help!