search

gdoor-org / gdoor

Wifi adapter and bus protocol documentation for the Gira TKS Door System
https://gdoor-org.github.io/
GNU General Public License v3.0
14 stars 4 forks source link

Build firmware via GitHub Actions & publish releases #12

Closed jschroeter closed 1 day ago

jschroeter commented 2 months ago

To be able to download the firmware & upload it via Wifi web UI.

DaSchaef commented 2 months ago

:+1: I think we need an extra repo for that, + github.io page :)

link collection: https://esphome.github.io/esp-web-tools/ https://github.com/witnessmenow/ESP-Web-Tools-Tutorial

https://docs.platformio.org/en/stable/integration/ci/github-actions.html https://gist.github.com/marcovannoord/4275a25639427e13d6e74f17636bbd5f https://docs.github.com/de/actions/using-workflows/events-that-trigger-workflows#running-your-workflow-only-when-a-push-of-specific-tags-occurs

jschroeter commented 2 months ago

I think this could also be within one repo + a gh-pages branch for the deployed website. I personally would prefer that, I think it's easier to maintain. But up to you.

DaSchaef commented 2 months ago

I do not like branches that much, as they are "invisible" on filesystem level? Also I'm not familiar with Github Actions at all, in contrast to Jekyll ;-)

So I kind of feel "safer" to play in another repo. (Another benefit is the github URL, where repo name needs to match org. name) https://gdoor-org.github.io/ https://github.com/gdoor-org/gdoor-org.github.io

(I have no clue how the automatic compiling is set up)

jschroeter commented 2 months ago

I do not like branches that much, as they are "invisible" on filesystem level?

True, but I was wrong, it also works without a gh-pages branch: I gave it a try in https://github.com/jschroeter/gdoor/ and added the website to the /website folder.

Also I'm not familiar with Github Actions at all, in contrast to Jekyll ;-)

Both work very well together, just add this file and it will take the /website folder, build it with Jekyll and publishes the result to GitHub Pages: https://github.com/jschroeter/gdoor/blob/main/.github/workflows/website.yml

https://github.com/jschroeter/gdoor/actions/runs/8789883341 https://jschroeter.github.io/gdoor/ (path to CSS is broken but would be easy to fix)

Another benefit is the github URL, where repo name needs to match org. name

True, this seems to be a good argument to keep it separate. And there is no issue with it, I just thought if everything is in one repo we might be able to re-use the GDoor logo and other content to not maintain it multiple times.

jschroeter commented 2 months ago

Regarding the firmware build, here a first PR: https://github.com/gdoor-org/gdoor/pull/13

What should follow in next PRs:

jschroeter commented 2 months ago

Good news, I was able to successfully flash the adapter via esp-web-tools! Cool stuff, also the integrated terminal is quite helpful. In case you want to try it: https://jschroeter.github.io/gdoor/web-installer.html

But we need to decide: Where should we place the manifest.json and firmware file?

I thought the best would be to upload both in the release process (meaning whenever we create a release, e.g. with the GitHub release action mentioned above it would happen automatically when adding a tag) as release assets. Since the manifest.json also can include a version number, it makes sense that it gets dynamically generated during the release. On the website we could directly link the file of the latest release, e.g. <esp-web-install-button manifest="https://github.com/gdoor-org/gdoor/releases/latest/download/manifest.json">. The generated manifest could relatively point to firmware-merged.bin, this should work according to the docs. But I tried it and unfortunately GitHub doesn't seem to set the according CORS headers, meaning the browser blocks directly downloading the manifest/firmware file from the release asset. There is a discussion for it.

So I do see 2 options:

  1. manually push the manifest and firmware to the gdoor or website repository

The manifest doesn't change often, except of the version number, but probably it's not so important as you only need it for the initial installation. But the pushed static firmware would become outdated quickly.

  1. automatically push the files to the website repo during the release process, meaning the release would push a commit to e.g. the website repo so that the files end up on GitHub Pages.

Should work but is a bit of effort.

  1. add a little service (e.g. Netlify function) which proxies the request to the release assets and adds correct CORS headers. Some people are doing similar things via Cloudfront.

Would work, is probably free for our use case and only requires a few lines of code. But introduces a new platform which I would like to avoid.

DaSchaef commented 1 month ago

Sounds great! Can we make an action to push it to the homepage repo as "latest"? 2) Sounds the most simple for me?

jschroeter commented 1 month ago

Agreed, I also think 2) is the best option. Could you follow https://cpina.github.io/push-to-another-repository-docs/setup.html#ssh-deploy-key-vs-api-token-github and

Then I'll set everything up. Thanks!

DaSchaef commented 1 month ago

Done, I followed the instructions in the link.

jschroeter commented 4 days ago

Some progress on this: firmware changes on the main branch now automatically get pushed to the website branch, see https://github.com/gdoor-org/gdoor/blob/main/.github/workflows/firmware.yml#L45 and e.g. https://github.com/gdoor-org/gdoor-org.github.io/commit/6ff8995dd7883af7ae5232ad17a2a620e78eb075.

Also I tried setting the GDOOR_VERSION env variable, but it's not working yet, pretty sure the issue is in https://github.com/gdoor-org/gdoor/blob/main/.github/workflows/firmware.yml#L32, probably needs some other escaping.

If all this works we can adjust it to only push on release tags, meaning not everytime the main branch changes. How do you think we should handle hardware and firmware releases? Does every hardware release also include a firmware release? But firmware releases don't neccessarily include a hardware release?

DaSchaef commented 4 days ago

I would say separate firmware / hw releases. Great work!

jschroeter commented 4 days ago

Makes sense. How should we differentiate them in the Git tags (so I can setup the release GitHub action)? Should we pre/suffix the Git tags? E.g. v3.1-hardware and if it’s not „-hardware“, it’s a firmware. Because for sure there will be more firmware then hardware releases.

DaSchaef commented 4 days ago

Good idea!

jschroeter commented 3 days ago

Alright, I think this can be closed soon:

When pushing a tag starting with v and not ending with -hardware,

Open issue:

After we solved this and as a verification: Could you create a new release for the latest firmware? Optionally you could rename the existing tags to the new format. Thanks.

DaSchaef commented 3 days ago

For platform IO you need to set another env. variable (https://community.platformio.org/t/pass-global-define-at-run-time/1439/8)

You need to differentiate between bash level environmental variable and gcc #define which is set via gcc -D command line option, which you can extend via the env. variable PLATFORMIO_BUILD_FLAGS

MUH5678964 as example version value, i think you can put in your $GDOOR_VERSION here:

  1. export PLATFORMIO_BUILD_FLAGS="-DGDOOR_VERSION=\\\"MUH5678964\\\"" (Note the escaping, otherwise the " get lost on gcc level, may be different in github jobs)
  2. pio run
  3. Check that version was set, e.g. strings .pio/build/GDOOR_ESP32MINI/web/firmware_merged.bin | grep MUH should return MUH5678964
jschroeter commented 3 days ago

Thanks, it works now! Please give it a try :-)