turfptax / ugit

Micropython OTA update from github
GNU General Public License v3.0
136 stars 24 forks source link

Only update if latest commit mismatches #17

Open bpmct opened 1 year ago

bpmct commented 1 year ago

Fun project, I've started using it in my IoT project.

Would be neat if ugit could handle/cache the latest commit SHA from a repo, and only run an update if there is a mismatch (indicating a new version is available). I could, of course, add this logic into my own app, but was wondering if this was something you wanted to support within the scope of ugit.

If so, I'd happily send a PR

turfptax commented 1 year ago

Yes, go ahead and send a pull request.

The boards running of a hash never matches github's so yes we will have to cache the last hash and use that.

I think it is very important to include that.

turfptax commented 1 year ago

I am not sure how best to solve this. Any suggestions would be helpful. I can snag the sha from the github API but it would need to be stored in either the backup or another file.

sp4m4r commented 1 year ago

Dears, sorry if I'm out of topic but I'm more or less a noob when it comes to this stuff.

As Ben mentioned - very nice project, I've started using it in my IoT projects as well. Currently I'm pulling a "version.json" frequently and parsing it for an version number that I manually need to increase.

My idea now, as it was about comparing hashes of the whole repo, is to generate a hash of one of the MCU's local files (like main.py) and to compare it with the hash of the respective github file. I tried doing so using https://docs.micropython.org/en/v1.15/library/uhashlib.html but I'm struggling. Maybe this helps to go on in tis point - if not, I'm sorry! ;)

turfptax commented 1 year ago

Dears, sorry if I'm out of topic but I'm more or less a noob when it comes to this stuff.

As Ben mentioned - very nice project, I've started using it in my IoT projects as well. Currently I'm pulling a "version.json" frequently and parsing it for an version number that I manually need to increase.

My idea now, as it was about comparing hashes of the whole repo, is to generate a hash of one of the MCU's local files (like main.py) and to compare it with the hash of the respective github file. I tried doing so using https://docs.micropython.org/en/v1.15/library/uhashlib.html but I'm struggling. Maybe this helps to go on in tis point - if not, I'm sorry! ;)

I have found that all internal hash's done on the microcontroller will not match to the github hash from the repo. The best solution is to just run an internal hash and check on that or to just save github's hash into memory.

If anyone has a better idea let me know. I'm working on getting the new dev code to include a restore function and to check the hash. Thank you for your feedback!

sp4m4r commented 1 year ago

I have found that all internal hash's done on the microcontroller will not match to the github hash from the repo. The best solution is to just run an internal hash and check on that or to just save github's hash into memory.

I think this would be the best option - to keep the github hash locally and then to compare. I'm looking forward to the updates, thank you very much for the work and the project.

keredson commented 1 year ago

@turfptax they do match! just the git protocol has you prefixing the data before hashing. see: https://github.com/keredson/ygit/blob/7d6c0f79b5dc625124f6818d49826bf69b1ad5b8/ygit.py#L288 for most that winds up being b'blob NNNNNN\x00'+file_data, where NNNNNN is the number of bytes (as a string).

turfptax commented 1 year ago

@turfptax they do match! just the git protocol has you prefixing the data before hashing. see: https://github.com/keredson/ygit/blob/7d6c0f79b5dc625124f6818d49826bf69b1ad5b8/ygit.py#L288 for most that winds up being b'blob NNNNNN\x00'+file_data, where NNNNNN is the number of bytes (as a string).

Your project is awesome!

I'm working on the hash function as I type.

sp4m4r commented 1 year ago

Hi,

is this issue still inbdevelopment? Would be nice if this could be implemented.

Kind regards.

turfptax commented 1 year ago

Yes, I am testing this on 1.20 s2,s3, and 8266, I will try to devote some time this week. This was a good reminder so thank you.

turfptax commented 1 year ago

Thank you for your insights on this issue. I have created a poll related to this topic in the discussion tab. Feel free to share your thoughts and vote on the proposed solutions there. You can find the poll at this link: Discussion Tab[^1^].

[^1^]: Issue with Git Hash Prefixes · turfptax/ugit · Discussion #29 · GitHub

turfptax commented 1 year ago

Hi,

is this issue still inbdevelopment? Would be nice if this could be implemented.

Kind regards.

Hello,

Yes, the development is in progress. I'm glad to share that we have a working solution on the Dev branch that I'm currently testing. The solution involves checking for updates and saving the last hashes from GitHub in a file named 'ugit.tree'.

Over the coming weeks, I will be rigorously testing this new feature to ensure its reliability and efficiency. Once the testing phase is complete and the feature proves to be stable, we will consider merging it into the main branch.

I appreciate your interest and patience as we work on this update. I'll keep you informed about our progress.

Kind regards,

sp4m4r commented 1 year ago

Dear turftpax,

I kindly wanted to ask if there are any updated to share in this issue. Best regards. :)

sp4m4r commented 6 months ago

Hi everybody, it turned very calm for some time so I'll try to bring up the topic again. Best regards ... :)

turfptax commented 6 months ago

@sp4m4r,

Thank you for your patience and ongoing interest in this project. I apologize for the delay in my response.

In the next two weeks, I will be incorporating ugit into a new prototype for the OpenMuscle project. This integration will necessitate updates and tests with the latest MicroPython version. I expect to share significant updates after this process. To future-proof the code, I have opted to maintain an internal file save check, instead of aligning with GitHub's hash protocol. This approach should minimize the need for frequent updates.

I appreciate your understanding and am dedicated to continually enhancing ugit. I'll keep you informed about our progress.