Add Github CI

[Leseratte10]

This pull request adds a github action config file that enables CI for this repository. What that means is with every commit made to the repo, Github will automatically spin up a Linux container, compile WiiFlow Lite in exactly that version that has been commited, and then provides a download link to that version.

Just thought that could be helpful, 1. it confirms that every commit can be built successfully and/or displays if that's not the case, 2. it kinda documents which exact steps are necessary to compile WiiFlow Lite, and 3. it also runs on incoming pull requests (I believe, I was unable to test that yet) so you can see if merging a PR would break stuff. And 4., if you put the needed archives (devkitPPC, libogc, general-tools, gamecube-tools) that are currently loaded from my server into a github release and load them from there, that guarantees that this repository will contain everything that's needed to build WiiFlow Lite, to archive all that for the future.

Currently this CI config will download the needed binaries (devkitPPC r30, libogc 1.8.19, bin2s, elf2dol) from my devkitPro archive at wii.leseratte10.de. If you don't like that and would like to have a setup that doesn't require my server to exist / be online, you could also download the four binaries manually, upload them somewhere else (for example, as a github release in this repo) and modify the URLs in the main.yml file.

EDIT: This is how this CI would look then: https://github.com/Leseratte10/WiiFlow_Lite/runs/1397315620 And here's an example on how archiving devkitPPC and stuff on github as well - by just making a dummy release and adding the four needed archives referenced in this PR: https://github.com/Leseratte10/compile-devkitARM-r32/releases/tag/helpers

[gingerbeardman]

Beautiful! Nice work @Leseratte10

[Fledge68]

ok i'm an idiot what does CI stand for? I use windows does this do me any good? is this just for everyone else who wants to compile wiiflow lite?

[gingerbeardman]

CI = Continuous Integration

This is so GitHub will generate the builds, saving you work.

The side effects are, as described above, all good.

[Leseratte10]

@Fledge68 as gingerbeardman already said, CI stands for Continuous Integration. It means that for every single commit you make (and push to Github), Github will automatically compile WiiFlow Lite, and lets you see the status, the compile log, and download the compiled binaries. That's completely independant from which operating system you are usually using to compile the loader.

It will display on GitHub which commits compile successfully and which commits have errors, people will be able to download compiled binaries for every single commit from Github without having to compile that themselves, and if someone wants to add a feature or other small code parts they don't necessarily need to set up the whole devkitPro environment on their computer, they can just make a fork of the repo, commit their changes and it gets compiled automatically.

[Fledge68]

so does this mean i will no longer need to do releases? some times i commit 2 or 3 times a day. not sure if i would want it doing that for every commit. i've been wanting to compile wiiflow lite with the latest devkitpro and libogc would that be a problem? when users report issues on here or gbatemp how will i know what commit or version caused the issue if everyone will have different daily commits or forks of wiiflow lite.

this sounds like a neat idea i just want to make sure i understand it thats why im asking questions.

[Leseratte10]

Usually you'd still do regular releases with version numbers, just like you currently do. These automatically generated CI builds are usually not directly used by end users like a released version (except for maybe cases where someone reports a bug, you commit a fix and ask them to confirm that the bug is gone). What the CI builds can help with, depending on how you use them, is make the builds a bit more reproducible. If you want to know for a fact which version a user used if they are using these automated builds, you could add some code to WiiFlow that appends the Git commit SHA hash to the version number for these automatic builds, then when people use these it'll display a version number like "5.4.8-c629021" and then you know they used the automated build of commit c629021a44fbd571403f51d860e22b8a189b8c5b.

With the current build process (which I believe just consists of you running make on your Windows machine) the build environment isn't really documented except for "I'm using devkitPPC r30 and libogc 1.8.19". It doesn't document how you build the loader, if you have additional libraries installed, if there's other changes done to the binaries after they're compiled, and so on. When WiiFlow (or other programs, most of this explaination isn't limited to WiiFlow) is compiled using this Github feature, you know for a fact that every compile gets a new, clean virtual machine with exactly these components that you've defined in the config file, and if that compilation succeeds, you know for a fact that WiiFlow doesn't depend on other stuff that might only be installed on your machine and not included in this repository.

Ideally, when using CI, one would write that CI config file (the main.yml) in a way that the CI output archive would be identical to what you would be using in a release - so that you could download a CI build from Github and upload it as a release again without modifying it when you want to make an official release.

The builds can be useful though if there are users who want the "bleeding-edge" features (no idea if that's something that applies to WiiFlow) and want to use the most recent commit without having a devkitPro setup themselves. Or to quickly figure out which commit broke something.

But after all, all that this Pull Request adds is the config file that causes Github to automatically compile the code for each commit, mainly to confirm that nothing's broken and everything still works and compiles without errors. That doesn't mean that this should replace regular releases, nor is it intended to cause confusion with end users all using different versions. It's just an additional feature that gives users the ability to download binaries for every commit should they so choose.

I just decided to create this Pull request / this CI config since I wanted to compile WiiFlow Lite myself to test some stuff and needed to set up all these specific devkitPPC / libogc versions again, so I thought "why not automate this in a CI config so when people fork the repo to add changes everything will be compiled automatically".

If you want to change WiiFlow to compile with different versions of devkitPPC and/or libogc in the future, you can just change them. The main.yml file that's in this merge request contains the URLs to the currently used versions, and when you want to use different versions you can just change the URL. Currently these URLs point to my server (which also contains all the newer versions of devkitPPC and libogc), but if you don't want that script to depend on my server you can also host the archives elsewhere.

I hope that clears up some of the confusion; if not, let me know and I'll try to explain it differently.

[Fledge68]

where are these downloads downloaded to? my computer?

[Leseratte10]

You mean the devkitPPC and libogc versions / download URLs that are mentioned in the file? No, your computer doesn't have anything to do with that. With every commit, Github will basically create and boot a fresh Linux Virtual Machine directly on the Github servers, which will then download devkitPPC / libogc and then compile Wiiflow. After that, the virtual machine with the downloaded devkitPPC is destroyed again.

For the git client on your computer, that file is just a normal text file.

[gingerbeardman]

I was thinking this will be a real benefit for me to start contributing actual code, as I've ever been able to get a working dev environment on my Mac.

[Leseratte10]

That would be one of the advantages, yeah. Once this pull request is merged, you could just fork the repo and commit your changes to the fork, and Github would auto-build the loader and let you download the binaries.

[Leseratte10]

I just updated the Merge Request, now the loader is packaged in the same format you're using for releases. An example can be seen at https://github.com/Leseratte10/WiiFlow_Lite/runs/1406498068, when you go to that URL you can click on the "Artifacts" link in the top right corner and download a ZIP that's structured like one of your releases, that'll contain the compiled binaries. The only thing that's missing from these auto-generated archives would be the "plugins_data" and "themes_lite" folders from the Wiiflow folder, since I was unable to find them in the source code repo.

[KcrPL]

That looks really neat. I don't see why you wouldn't want to use it.

[Fledge68]

I'm assuming this is to benefit other people that want to contribute or modify the code.

I don't see it as a benefit for me. I already have devkitpro on my pc and the repo locally on my computer. i make changes and compile. if it compiles fine I test it to make sure wiiflow lite still works. if it does then i commit and push the changes to the github repo. which happens to also commit the latest boot.dol and bins to the OUT folder which people can actually download and use if they can figure out how.

your plan is for people to download the repo locally and make changes but then commit the changes which triggers your CI to compile it and if it don't compile rejects the commit, but if it does compile it adds the commit and creates a zip file similar to my release zips. then people download the zip and have the boot.dol and bins that they hope work if they know what they are doing coding and understand wiiflow. correct?

One main problem i see is I use a modified version of libogc. patched with the 480p extremes pixel patch. but i'm sure you can just patch it and upload it and have your github action use it.

the only other problem is how to make sure to differentiate my official builds/releases from the random coders builds/releases.

[gingerbeardman]

I believe you'll see personal benefits once it's in place.

One main problem i see is I use a modified version of libogc. patched with the 480p extremes pixel patch. but i'm sure you can just patch it and upload it and have your github action use it.

Yes

the only other problem is how to make sure to differentiate my official builds/releases from the random coders builds/releases.

"you could add some code to WiiFlow that appends the Git commit SHA hash to the version number for these automatic builds, then when people use these it'll display a version number like "5.4.8-c629021" and then you know they used the automated build of commit c629021."

https://github.com/Fledge68/WiiFlow_Lite/pull/222#issuecomment-727269621

[Leseratte10]

@Fledge68 that is almost correct. Usually with stuff like that (when people want to contribute changes) they'd first make a fork on Github and then clone that. Then they can do their changes in their fork, commit as often as they want (and with each commit they get the binaries made by Github), and when they're done they can make a Pull Request. Then you'll immediately see in the Pull request that it compiled successfully.

If you want, you can send me the libogc patch you are using and I'll try to add both that and the version number thing I wrote in an earlier comment that gingerbeardman just mentioned. That would result in WiiFlow versions that are being built by Github having version numbers like "5.4.8-c629021" which makes it clear that this is a modified version compiled by Github which is based on 5.4.8.

[Fledge68]

sounds good.

here is my libogc.a that hex edited with fix. https://www.mediafire.com/file/8ry25esy9itpzgn/libogc.a/file

[Fledge68]

so it all fixed right?

[Leseratte10]

Nope, not yet. I'm currently working on adding the libogc patch (EDIT: That's now done), and I still need to add the code to use the git commit ID in the wiiflow version when the loader was built through the CI. I will let you know when that's all done.

[Leseratte10]

Okay, that's now done as well. The first build after this is merged will take a couple minutes longer while Github downloads devkitPPC from my server and stores it in the cache, and after that the pipeline should be independantly from my server.

It'll take the devkitPPC and libogc from the Github cache, patch that single byte in the libogc.a that you modified for the 480p fix, then it'll compile WiiFlow Lite and then provide a download link for that build.

I also modified the Makefile so that when Wiiflow is compiled inside of this Github CI, the first 8 characters of the git commit ID are appended to the WiiFlow version number, that means, for example, that Wiiflow would display "WiiFlow WFL 5.4.9 beta 1 3a7152b2" in the settings menu. Would you mind verifying that my modified Makefile still compiles on your Windows machine, and that the resulting binary does not contain the commit ID?

[Fledge68]

but i can't test it right now. maybe tomorrow.

[KcrPL]

Honestly GitHub Actions is one of the best things that ever happened to GitHub.

Thanks Leseratte.

[Leseratte10]

Thanks for merging. When you now go to the commit list https://github.com/Fledge68/WiiFlow_Lite/commits/master you can see a green checkmark next to all new commits, and when you click on that you can see the build log and download the compiled binaries, so that seems to work just fine.

Same as for the PRs, as can be seen by the green checkmark next to #224 in the pull request list: https://github.com/Fledge68/WiiFlow_Lite/pulls?q=is%3Apr