search

freifunk-berlin / falter-firmware-selector

Freifunk Berlin Fork of OpenWrt Firmware Selector. Modifications are in branch 'falter_stable'
http://selector.berlin.freifunk.net/
Creative Commons Zero v1.0 Universal
1 stars 2 forks source link

Selector shows wrong checksums #3

Closed Akira25 closed 2 years ago

Akira25 commented 3 years ago

The selector shows wrong checksums for the images. In fact, it shows the checksums of only one image-type, even if we select the other one. So at least for half of the images, there get correct checksums displayed.

when this bug is fixed, we can drop e7fe664ec1b17ce04c8d47c5edf5b5778b5898fb

khanku commented 3 years ago

I started looking at this a while back but was thrown off by the missing (freifunk-berlin specific) installation/usage instructions. Maybe https://github.com/freifunk-berlin/falter-firmware-selector#installation could be updated? Or maybe you could explain how collect.py is run here @Akira25?

mwarning commented 3 years ago

@Akira25 since I wrote the collect.py script, is there something I can help with?

Akira25 commented 3 years ago

@mwarning maybe you can. Really nice, that you provide us with in that issue. :)

@khanku I just added some hacky shell-scripts, that where on the firmware-selector-machine, but no added in the versioning-system.

The whole process is a bit hacky and messy. Probably the collect.py-script can do it all alone with just minor adjustments, but I didn't have had the time to read it all through now.

First thing to notice, we kind of "misuse" the version-pulldown menue to provide different flavors of the firmware too.

For deployment we currently do at that way:

  1. checkout the falter_stable branch on deployment-machine into vhost-root.
  2. link the files from subdir www/ into webserver-root (current directory)
  3. run ./fetch_release.sh 1.2.1, which will call get_profiles_local.py. That script will get the json-files we aim for through collept.py and place them in a subdir 1.2.1/ into our vhost-root. In that subdir, theres a dir for tunneldigger and notunnel
  4. afterwards I adjust the config.js by hand to have an entry for the tunneldigger-version and notunnel-version.

Everything quite hacky, I know. I'm sorry, that I didn't had the time to make it proper.

Akira25 commented 3 years ago

The whole stuff with linking files into another direction comes from a time, where I hadn't root access to the machine, but only to that specific directory. So I was not able to adjust the webservers config accordingly. That will be better in the near future. (https://github.com/freifunk-berlin/ansible/pull/3/commits/3bb08c15a82f8076cf0ea32f619691339ccd9385 line 39 and after).

khanku commented 3 years ago

First thing to notice, we kind of "misuse" the version-pulldown menue to provide different flavors of the firmware too.

I think that is the crux of the issue. IIRC the checksum that's shown is always the one from the first flavour. The scripts you added should help troubleshooting.

khanku commented 3 years ago

It's weird: it works on my machine(tm)! Here is what I did:

./get_profiles_from.py https://firmware.berlin.freifunk.net/stable/1.2.1/
jq -r .images[0].sha256 <1.2.1/tunneldigger/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json
6de12efeebbc411ef4a4c12a3a2e2cb8e8e610db7842415012e8e713ceff83ce  #correct
jq -r .images[0].sha256 <1.2.1/notunnel/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json
cbf96b3026ab5bb3f0fbedc9cabf65fd7937eae02d273d14d8e27fefacd68d7a  # correct
# also works in my browser, locally

Those are the correct checksums that can be found at https://firmware.berlin.freifunk.net/stable/1.2.1/tunneldigger/ipq40xx/mikrotik/sha256sums and https://firmware.berlin.freifunk.net/stable/1.2.1/notunnel/ipq40xx/mikrotik/sha256sums respectively.

Now if I go to https://selector.berlin.freifunk.net/ I can see that both https://selector.berlin.freifunk.net/1.2.1/tunneldigger/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json and https://selector.berlin.freifunk.net/1.2.1/notunnel/ipq40xx/mikrotik/mikrotik_sxtsq-5-ac.json have the same checksum. And the same build timestamp (up to the second) which makes me think they're the same file.

Since the differences between get_profiles_from.py and get_profiles_local.py are minimal I'm going to guess that the data in /usr/local/src/www/htdocs/buildbot/stable/$VERSION/ is the source of the error. @Akira25 you seem to have access to that machine. Could you check the files and confirm/infirm?

Akira25 commented 3 years ago

you can find the original data freely readable beneath the images at http://firmware.berlin.freifunk.net/, which maps to /usr/local/src/www/htdocs/buildbot/.

The copied data (fetched by collect.py) is accessable at https://selector.berlin.freifunk.net/1.2.1/, which maps to /usr/local/src/www/htdocs/firmware-selector/1.2.1/

Akira25 commented 3 years ago

in result, the original data from /usr/local/src/www/htdocs/buildbot/stable/$VERSION/ holds the correct checksums. I assume that we maybe use collect.py in a different way than intended.

khanku commented 3 years ago

I believe I've found the issue: when passed a local URI collect.py will use its scan function which will always pick the first (freifunk-specific) flavour. That's what happens in get_profile_local.py and results in the checksum issue which get_profile_from.py does not exhibit.

Akira25 commented 2 years ago

This was fixed by #4 . Thanks a lot to @khanku for your contribution.