Create cross-platform single-app firmware updater for EBB

[EmbeddedMan]

It would be so so so nice to have a single app that a user could download and run, on Win, Mac and Linux, that would do the following:

0) When doubled clicked, expand the necessary pieces into a temporary directory, and run the 'main app' 1) Detect which COM port their EBB is on. If none is found, prompt them that this is a problem. 2) Place the EBB in bootloader mode (by sending the "BL" command over the CDC-ACM connection) 3) Wait until the EBB re-enumerates as a HID device in bootloader mode 4) Use the Microchip HID bootloader protocol to upload a specific version of the EBB firmware hex file 5) Reset the EBB back into normal mode 6) Connect to the EBB after it's re-enumerated as a CDC-ACM device, and confirm (using the "V" command) that the new version has been properly flashed to the EBB

Ideally this would be a GUI app that was very small and very simple. There would be a new version of this app for each version of EBB firmware released. Since I'm not a GUI guy at all, my 'easy path' to this is to write it as some type of command line application, which, when double clicked on a desktop, would open a console and do it's user I/O that way.

The https://github.com/ApertureLabsLtd/mphidflash project is the perfect thing to use for step 4) and 5), as it is already a cross platform command line tool to do the actual flashing.

So this should be a pretty straight forward little app to write. The trick is how to make it cross platform. It needs to be able to get a list of available serial ports, open them, talk to them, detect if a certain PID/VID is connected (the EBB when in HID bootloader mode). It needs to be able to be a single executable, which can, when double clicked, unpack the necessary files (HEX file, main app, mphidflash, etc.) into a temp directory and run.

Is there a cross platform installer framework that could do this? If somebody has any good ideas as to a cross platform tool to use, I'm happy to go learn the tool and see if I can write this app.

[zemosh]

Such an app would be handy. What about having only one installer that would download the current repository, and then you could pick a version you want to upload (with the latest as default)? Regarding the language, Python seems to me as a natural candidate---it's cross-platform, and there's already a library for EBB available; it can find the port automatically, connect and talk to EBB, etc. It's possible to create a simple GUI in Python as well. I found this tool http://www.pyinstaller.org, which can create a single package for the application with all dependencies, but I haven't tried it yet.

[EmbeddedMan]

I like your suggestion, but we have to make it super, super super simple and foolproof. I don't want to rely upon an active internet connection. I don't want users to have to figure out which one they want to update to.

Having another app which is 'update your EBB to the latest available' may be a second stage of this effort. But the first needs to be completely self contained app/updates that require no thought.

I will look at pyinstaller. I don't know Python yet, but it doesn't seem too hard. I wonder how small an app it can make - I'd rather not have these things be multiple megabytes.

[EmbeddedMan]

Question: if a user has multiple EBBs plugged into their computer at the time that they run the updater tool, should we update every EBB that we can find? Or just the first one?

[oskay]

Thus far, we haven't generally provided software support for more than one EBB at a time... If it's easy to make it identify every one, that's fine. However, just doing the first one is fine too.

[EmbeddedMan]

Well, there was a 'bug' in my code that made it update every EBB that it found (I originally intended it to just update the first one and then quit), and I thought maybe that wasn't such a bad thing. I'll leave it in.

On Wed, Nov 9, 2016 at 7:40 PM, Windell Oskay notifications@github.com wrote:

Thus far, we haven't generally provided software support for more than one EBB at a time... If it's easy to make it identify every one, that's fine. However, just doing the first one is fine too.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-259578961, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCJ4mGCA9237DVvtAyTizktERyBdCks5q8nXxgaJpZM4Jgs17 .

[EmbeddedMan]

I found this out when I had 10 different serial port devices on my PC (3 of which were EBBs) and ran the updater to see if it would work or not.

On Wed, Nov 9, 2016 at 7:42 PM, Brian Schmalz brian@schmalzhaus.com wrote:

Well, there was a 'bug' in my code that made it update every EBB that it found (I originally intended it to just update the first one and then quit), and I thought maybe that wasn't such a bad thing. I'll leave it in.

On Wed, Nov 9, 2016 at 7:40 PM, Windell Oskay notifications@github.com wrote:

Thus far, we haven't generally provided software support for more than one EBB at a time... If it's easy to make it identify every one, that's fine. However, just doing the first one is fine too.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-259578961, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCJ4mGCA9237DVvtAyTizktERyBdCks5q8nXxgaJpZM4Jgs17 .

[EmbeddedMan]

All known issues have been fixed with the Windows installer. There are now two exe files - one to program version v2.4.3, and one for v2.4.4. It would be very helpful if somebody other than me tested them out to see if any new problems crop up.

Just download either of the exes on a Windows system with an EBB plugged in, run it, and click Start.

https://github.com/evil-mad/EggBot/tree/master/EBB_firmware/EBBUpdater

*Brian

[ShelMi]

Have attempted to use v2.4.4 update .exe for Windows. I have a Win 7 64 bit machine. Current EBB v2.3.0. Failed in following ways:

Attempt 1: Did not place EBB in bootloader mode before running EBBUpgraderGUI_244.exe The exe aborted indicating it wanted EBB to be in bootloader mode. Interestingly, it apparently caused EBB to go into bootloader mode, as after aborting EBB was alternately flashing red and green leds.

Attempt 2: Reset EBB. Placed EBB in bootload mode with PRG/RST switches, and EBB responded by flashing red/green leds. Ran EBBUpgraderGUI_244.exe. This time upgrader aborted stating couldn't find COM port on computer.

I'm happy to try any tests you want. I'll leave my system in its current state with no further attempts to update, to leave the configuration unchanged so tests will be equivalent.

For your guidance in deciding what you can ask me to do, I'm computer-savvy. As it happens I've designed, built, and programmed a PIC 18F46J50 board myself...a lovely chip indeed! I also have a 'scope in case that's useful in any way.

Current EBB version:

Starting with EBB in ordinary run mode:

Starting from bootloader mode:

[EmbeddedMan]

The Upgrader will automatically find your EBB (which should start out in normal - not bootloader - mode when you run the upgrader) and send it a command to put it into bootloader mode.

Then, after the Upgrader does this, it runs the mphidflash program which should then find the EBB in bootloader mode and perform the update.

In your case, the Upgrader application did not properly put your EBB in bootloader mode.

Could you remove power from your EBB, then reapply power to it, then run the Updater again and see if you get the same result?

Thanks SO MUCH for trying this out!

*Brian

On Sat, Nov 26, 2016 at 9:58 PM, Shel Michaels notifications@github.com wrote:

Have attempted to use v2.4.4 update .exe for Windows. I have a Win 7 64 bit machine. Current EBB v2.3.0. Failed in following ways:

Attempt 1: Did not place EBB in bootloader mode before running EBBUpgraderGUI_244.exe The exe aborted indicating it wanted EBB to be in bootloader mode. Interestingly, it apparently caused EBB to go into bootloader mode, as after aborting EBB was alternately flashing red and green leds.

Attempt 2: Reset EBB. Placed EBB in bootload mode with PRG/RST switches, and EBB responded by flashing red/green leds. Ran EBBUpgraderGUI_244.exe. This time upgrader aborted stating couldn't find COM port on computer.

Current EBB version: [image: ebb current version] https://cloud.githubusercontent.com/assets/16483308/20645408/7258a5c2-b42b-11e6-890f-633169a2b296.jpg

Starting with EBB in ordinary run mode: [image: update fail starting from ordinary run mode ebb] https://cloud.githubusercontent.com/assets/16483308/20645420/8e1984ac-b42b-11e6-9c7a-119f3d7c164e.jpg

Starting from bootloader mode: [image: update fail starting from bootloader mode ebb] https://cloud.githubusercontent.com/assets/16483308/20645421/b2ba78d4-b42b-11e6-9304-e17c884c8bac.jpg

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263100922, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCNaq_OVBkk0q_IV9645toygblSAsks5rCP_ygaJpZM4Jgs17 .

[ShelMi]

Success! I removed both 9 v and USB cables, then connected USB cable only, then did no queries or anything on EBB, then ran installer. Worked perfectly this time.

I'll fool around a bit more with variations on the theme, to see if I can provoke the problem again.

[EmbeddedMan]

SO AWESOME! Thanks for the report.

*Brian

On Sat, Nov 26, 2016 at 10:20 PM, Shel Michaels notifications@github.com wrote:

Success! I removed both 9 v and USB cables, then connected USB cable only, then did no queries or anything on EBB, then ran installer. Worked perfectly this time.

I'll fool around a bit more with variations on the theme, to see if I can provoke the problem again.

[image: update success] https://cloud.githubusercontent.com/assets/16483308/20645515/d066caba-b42e-11e6-9b50-0ccd52d65214.jpg

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263101529, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCNtr7NWsFk5vqzwKziHeEBM5Cwavks5rCQT1gaJpZM4Jgs17 .

[ShelMi]

I've tried all combinations of "stuff", but have not been able to re-create the failure. Osmosis strikes again, but in a happy way for a change!! 8^) And thanks for making such an easy-to-use installer!!

[EmbeddedMan]

No problem. Hopefully others will try it and we can figure out if the problem happens to anyone else or not.

*Brian

On Sat, Nov 26, 2016 at 10:29 PM, Shel Michaels notifications@github.com wrote:

I've tried all combinations of "stuff", but have not been able to re-create the failure. Osmosis strikes again, but in a happy way for a change!! 8^)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263101788, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCCo3CDrCvBNWa_TIe002xKSYHzk-ks5rCQdGgaJpZM4Jgs17 .

[oskay]

Bug found when updating multiple boards in one session. The first board on which you update the firmware, it runs once. On the second, it runs twice, and so on. By about ten, it's very annoying.

[EmbeddedMan]

Oohh. Yeah, that's bad. Should be easily fixable.

On Tue, Nov 29, 2016 at 2:17 PM, Windell Oskay notifications@github.com wrote:

Bug found when updating multiple boards in one session. The first board on which you update the firmware, it runs once. On the second, it runs twice, and so on. By about ten, it's very annoying.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263686083, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCGZmXzAVejnfBVGNriZmR2AhCDGOks5rDIhhgaJpZM4Jgs17 .

[EmbeddedMan]

I am unable to reproduce this problem. I tried with four EBBs, and they each only got updated once.

What version of Windows are you running?

[oskay]

It's Windows 10. Flashed a batch of ~100 boards, restarting the program every 3-4 times.

[EmbeddedMan]

So strange. Looking through the logic of the updater, I just can't figure out how it could behave like you are seeing it behave. It only gets the list of available COM ports once at the beginning of the program, then walks through them one at a time (in the order that they were found), attempting the update once per com port.

It's just BASIC code, so it's very easy to follow, and the source code is right along side the executable in the repo, so you're welcome to do a little code review and see if you can find something to change to fix this bug. The software (ThinBasic) environment is free and very easy to use.

On Tue, Nov 29, 2016 at 7:48 PM, Windell Oskay notifications@github.com wrote:

It's Windows 10. Flashed a batch of ~100 boards, restarting the program every 3-4 times.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263758464, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCHWPFV23_0LmDtXFKXQbUlgLTQcfks5rDNXXgaJpZM4Jgs17 .

[oskay]

Any chance that it has to do with the incrementing port number? It appears to loop over available ports...

[EmbeddedMan]

If it does, I don't see how. It makes a list of all available com ports, then walks through the list, trying each one in turn. Unless you see something like this in your com port list:

COM2 COM3 COM3 COM6 COM6 COM6 COM1 COM1 COM1 COM1

I don't see how it would try to update each EBB one more time than the last one. Hmm.

The COM ports are not sorted in any way - we just make a list right from what Windows gives us. And I can't explain why it would be different on your system compared with mine. There's obviously something I'm missing here.

On Tue, Nov 29, 2016 at 8:06 PM, Windell Oskay notifications@github.com wrote:

Any chance that it has to do with the incrementing port number? It appears to loop over available ports...

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263761328, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCPfXGzJy0X7mPw8nFHIe19ksRkNTks5rDNodgaJpZM4Jgs17 .

[EmbeddedMan]

Can you try it with say, 4 EBBs, can then post a screenshot of the window when it's all done? Maybe there's a clue there . . .

On Tue, Nov 29, 2016 at 8:12 PM, Brian Schmalz brian@schmalzhaus.com wrote:

If it does, I don't see how. It makes a list of all available com ports, then walks through the list, trying each one in turn. Unless you see something like this in your com port list:

COM2 COM3 COM3 COM6 COM6 COM6 COM1 COM1 COM1 COM1

I don't see how it would try to update each EBB one more time than the last one. Hmm.

The COM ports are not sorted in any way - we just make a list right from what Windows gives us. And I can't explain why it would be different on your system compared with mine. There's obviously something I'm missing here.

On Tue, Nov 29, 2016 at 8:06 PM, Windell Oskay notifications@github.com wrote:

Any chance that it has to do with the incrementing port number? It appears to loop over available ports...

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-263761328, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCPfXGzJy0X7mPw8nFHIe19ksRkNTks5rDNodgaJpZM4Jgs17 .

[oskay]

Apparently, it does look like that. Each time that you plug in a new EBB, it seems to "find" it more than once.

[oskay]

Perhaps possible to remove duplicate names from the list before starting to update?

[EmbeddedMan]

Yes, that's absolutely something that could be done. It's just so very very strange to me that the behavior is different between your computer and mine. I'll add code to remove duplicates from the list and see if it helps. Thanks for continuing to test.

*Brian

On Tue, Dec 20, 2016 at 2:10 PM, Windell Oskay notifications@github.com wrote:

Perhaps possible to remove duplicate names from the list before starting to update?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-268345733, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCFRTw2xFtKBVdMm3AjzrH5PII9kqks5rKDZQgaJpZM4Jgs17 .

[EmbeddedMan]

A HA! I have found the problem. It lies with a use case that I had not considered.

I expected that everyone would follow the text in the window, and when an update is finished, they would click "Exit" (as the text instructs them to). However, users always find good ways to break a developer's assumptions!

When you use it, you fire up the app, plug in an EBB, click Start Update, then (when the update is finished), remove that EBB, plug in a different EBB, and click Start Update again, correct? That use case will in fact produce exactly the type of problem you describe - I have verified it here.

Your screenshot - with multiple "Click Exit" texts in the window - was the clue I needed.

As as temporary workaround, plug in ALL of the EBBs you want to update at once, then run the app, then when it's completely done updating, click exit. Then you won't get the duplicate entries in the COM port list.

Or, if you want to do them one at a time, make sure to click Exit after each EBB so that it starts fresh the next time.

In the meantime, I will try modify the code to allow the app to work properly when used the way you use it.

[EmbeddedMan]

OK, new versions of the updater EXE files have been uploaded to the GH repo. They should fix the problem you are seeing - they do for me anyway. Please give them a try, and close the issue if the problem is resolved.

[oskay]

Interesting... Isn't that how I described the issue-- updating multiple boards in one session? As I said, we had to restart the program every few times...

[oskay]

"As as temporary workaround, plug in ALL of the EBBs you want to update at once"

Oh, that's hilarious. My computer does not have 100 USB ports. :(

[EmbeddedMan]

Windell,

Yeah, neither does mine. But I just use hubs. You can get up to 128 EBBs on a single host port that way.

You are correct - you described how you were using it, but it just never 'clicked' with me that you weren't clicking the exit button after each update until I saw the screen capture.

*Brian

On Wed, Dec 21, 2016 at 11:27 AM, Windell Oskay notifications@github.com wrote:

"As as temporary workaround, plug in ALL of the EBBs you want to update at once"

Oh, that's hilarious. My computer does not have 100 USB ports. :(

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/evil-mad/EggBot/issues/53#issuecomment-268584352, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbeCFUkmequTqgFMI0n-cTs8zshjh96ks5rKWGFgaJpZM4Jgs17 .

[oskay]

Is this ticket obsolete, now that we have reasonable (but not truly cross-platform) updaters?

[EmbeddedMan]

It still might be better if we had a single updater app that worked on all platforms. I don't know how to write that kind of app though.

The Windows app for updating EBBs does appear to work pretty well now. If you have solutions for Mac and Linux, then we have updater apps for everybody - if that's good enough, feel free to close this issue.