update micronucleus

[sanbeg]

The micronucleus bootloader is bundled with a command line utility. Although this command has backwards compatibility to allow the command to work with older bootloaders, it doesn't have forwards compatibility, and thus will refuse to work with newer bootloaders.

It appears that ATTinyCore bundles an old alpha of this command from 2015, which is no longer listed in the release page https://github.com/micronucleus/micronucleus/tags (IIRC, the same version that was used by the long since unmaintained Digistump core).

When trying to flash a Digispark clone with an updated bootloader, I get this error:

Warning: device with unknown new version of Micronucleus detected.
This tool doesn't know how to upload to this new device. Updates may be available.
Device reports version as: 2.4

Although may cheap clones still use very outdated bootloaders which will work with this old tool, an updated tool would work with both old and new bootloaders.

[SpenceKonde]

Me too :-)

The specs on the parts you'll be able to upload to via USB should help take the sting out ;-)

[SpenceKonde]

@stonehippo Any updates here? There has to be a new release sooner than I'd expected due to #470, which was an embarrassingly severe PWM problem on a bunch of parts. none of which should have seen any change in related behavior, and it would be nice to get new binaries here so we can make the core again installable through board manager on platforms for which there wasn't a micronucleus tool available.

[stonehippo]

@SpenceKonde I'll get the Windows and Linux x86_64 binaries built this weekend, and have them all in my repo for you to point to.

[sanbeg]

@SpenceKonde I'll get the Windows and Linux x86_64 binaries built this weekend, and have them all in my repo for you to point to.

Are those the same as the ones in my repo? It may be nice to centralize them so we can track versions in one place.

Is the only one we're missing currently the 32 bit Linux? Is that still used enough to be concerned with?

[stonehippo]

Yes, based on what I've seen building for ARM and MacOS, it seems like the existing binaries for Linux and Windows would work. I wanted to go through the build process on the additional platforms so I'd have any documentation or build steps documented, and have new builds all in the same place.

If you wanted to move faster, we could just focus on getting various binaries in one place, and then I can catch up on process. Thoughts?

[stonehippo]

OK, I've added linux x32 and x64 binaries to my repo, and updated my fork of the micronucleus repo to compile Linux and MacOS binaries statically by default to make them easier to move around. I still need to get the Window binaries built (the one that is in the upstream repo is not static and won't work without installing libusb), then I have to package each platform correctly, as was done for the digistump builds. Probably won't get that done today, but should be set tomorrow.

When I'm done with the above, I'll send over a PR for ReleaseScripts that adds Linux ARM and updates the other archs to the 2.4 binary, @SpenceKonde.

[stonehippo]

Ugh, just took a look at what the various Digistump projects were doing. They wrapped the micronucleus CLI with launchers for the various platforms, rather that compiling it statically. Does anyone here care if we go the latter route for ATTinyCore support? Since it's going to get used by the core, I don't see any problems with a static binary over a dynamic one. If fact, it's way smaller, and easier to package up.

[sanbeg]

I was just about to comment on the launcher. I've put some effort into reverse engineering that, and I didn't see any signs of it bundling shared libraries, although it's possible I missed something there. AFAIK, the launcher does 3 things:

• translates avrdude syntax to micronucleus
• passes through to avrdude if called without -c micronucleus
• works around a few issues with micronuclues status messages.

There was some discussion on this above, though I don't recall a conclusion on that.In my repo, I've reimplemented the translation and message work around as a bash script, which should be more portable; the pass through isn't relevant to what we're doing.

As an alternative, I've also opened a PR to fix the issues it works around @ https://github.com/micronucleus/micronucleus/pull/193

Personally, I'm OK with using a launcher script if it helps; at least that better than the mystery binary we have now. But if we modify the core to use the native syntax and either merge my PR or just build binaries from my branch we may not need it.

[stonehippo]

@sanbeg the launcher is just wrapping the binary micronucleus binary, right? Looking at the packages from Digitstump, they're definitely bundling the dynamic libraries with the CLI tool and the launcher, too. Maybe it makes sense to mix your launcher re-implementation with the static CLI tool I've been building (I started on this because I found that the binaries in your repo didn't work without also installing libusb).

[sanbeg]

@stonehippo Yes, it just call micronucleus. My script and the binary are completely interchangeable on every system I've tested it on; admittedly, all of which already had libusb installed.

@SpenceKonde had expressed interest in updating the core to use micronucleus syntax instead of going through the avrdude translation layer. If that happens, the script could become much simpler.

For the message issue, I think it's more polite to push the fix upstream than quickly publish a script to work around it. @cpldcpu do you have any insight there?

So given all that, we may just need to decide which of the things the launcher currently does it should continue with, and which should be done by other means.

[stonehippo]

I've got the tool compiling under Msys2/MinGW64, but it won't compile statically due to a lib incompatibility. Can't see why, yet, but will keep at it.

[ArminJo]

@SpenceKonde I just packaged a new version with the new micronucleus binaries for all platforms - not requiring launcher any more, see platform.txt 😀- and new bootloaders. Now it's your turn...

[SpenceKonde]

Siiiiiiick!

Need to finish DxCore 1.3.1 and get that released which I hope can happen today/tomorrow (test that changing TCA PORTMUX correctly moves analogWrite pins and that the TCD PWM pins act like the PWM pins on the ATtiny x7 of ATTinyCore now, and clean up variants for the TCA change, propagate flash-write changes from DA to DB and make at least one AVR128DB board I have run and test Opamp.h on it).and then can finally come back to ATTinyCore

[SpenceKonde]

You just had to delete the old file didn't you?

Good job - now nobody can install ATTinyCore

[SpenceKonde]

https://github.com/SpenceKonde/ReleaseScripts/commit/19300de9328a3512cabc2846ef4ae894264d7e2b#diff-dba3a4c259e313a72ba4b704f3db5319f26fac51bde77dfa545d5acd993b7754 2.0a4 windows (same version) rehosted to restore functionality.

[ArminJo]

Good Morning! Oh sorry sorry!!! I changed my own json to point to the original digispark win zip for the 2.04 version, since this is more transparent. I was not aware that you linked to the file. I restored it now. Tell me when it is save to remove.

Sorry Armin

[sanbeg]

@SpenceKonde I just packaged a new version with the new micronucleus binaries for all platforms - not requiring launcher any more, see platform.txt - and new bootloaders. Now it's your turn...

I just noticed that the PR to fix the mirconucleus issue with the "please plug in" message being buffered was rejected. So it looks like we'll still need a launcher script to emit that message. https://github.com/micronucleus/micronucleus/pull/193

[SpenceKonde]

Oh, I found a clean copy - but I only had one copy of that one. Like 4 copies of the one from the official digispark repo.

wait, you were using the digispark version? But it's not the same! one of the files - I think the .inf - is slightly different. I was under the impression that yours fixed something, which is why I used yours for windows instead of the old digispark one. But I think it was relating to the digispark USB libraries, I think the CDC one, but that one never worked for me no matter what I did! (tried both installs many times. )

Were you ever able to make the libraries work? I never was able to make any USB libraries work, which is precisely why I don't include those libraries with ATTinyCore. I got some help from someone on FB - but then life got crazy.

What I was going to do is 1) geta known working hex file and from there, I should be able to work on the driver issue until I have it working on my test systems. Then, I could work on fixing the libraries so I could build my own working ones (i'd have the advantage of being able to examine the hex files of a working and non-working build, too. Maybe I'd find that if I went back enough compiler versions, finally it would work, and I'd see what the difference was and be able to prevent it from happening. (and I could dump a .lst of both versions too)..

[ArminJo]

@SpenceKonde I never did a fresh install on windows for the micronucleus USB driver in the last year. Maybe I should test it again. And I can check if the V-USB examples are still running, I never used them personally, only for issue fixing. After this you will get feedback.

[SpenceKonde]

@SpenceKonde I never did a fresh install on windows for the micronucleus USB driver in the last year. Maybe I should test it again.

Please let me know how to set it up if you manage this - I have always really wanted to be able to make thee into USB devices! It would open so many doors!

[nerdralph]

@SpenceKonde I never did a fresh install on windows for the micronucleus USB driver in the last year. Maybe I should test it again.

Please let me know how to set it up if you manage this - I have always really wanted to be able to make thee into USB devices! It would open so many doors!

And it will have you pulling out your hair. V-USB is by no means robust. Even micronucleus isn't as robust as it could be, and that's a lot easier to do when you have 100% control of the CPU and interrupts. For example I found a bug in Tim's OSCCAL tuning code. It seems the USB hubs he tested on were more like a USB switch and didn't broadcast traffic to every port on the hub after they were enumerated. On the PC I was using, with a common Intel chipset USB EHCI, the interrupt IN tokens for my low-speed mouse get broadcast to every port on the same hub, which was sometimes throwing off the tuning. This is one of the things I worked on fixing for MN: https://github.com/nerdralph/micronucleus-firmware/blob/asm/firmware/osccal.S

Then on top of the software hacks, you have the common hardware hack of running the AVR at 5V, and clipping the D+/D- signals with zeners. Even if you get the "right" zeners, you've reduced your signal margins since the host outputs 3V3 (> 2.7V from what I recall of the USB spec), and the AVR thinks 5V is high. VIH for most AVRs is between 2.6V and 2.8V with 5V power. If you run the AVR at 3V3, VIH is between 1.7 and 1.9V, making things much easier.

If you want to make a USB project that other people can build and use on Windows, Linux, MACs on their various hardware platforms, it's WAY easier with a MCU that has a hardware USB SIE. For around $1 you can get an ARM Cortex-M MCU with a full-speed USB peripheral, or for less than $3 you can get a tube of 10 8051-based CH551G MCUs.

[SpenceKonde]

BLOODY HELL! There are at least three distinct versions of each of the micronucleus tool files, with the same name, which differ by 20-50 bytes in size and have a different SHA hash.

Sorting that fucked up mess out is what's between here and 1.5.0. Should be ready tonight... and they are all getting repacked, WITH A UNIQUE NAME

I just went through generating SHA-256 hashes and file sizes for the 7 archives. because they didn't match what was in the archive. Then I looked at what I had on the webserver.... they're all different too! It looks like every place that the 7 packages are located, they were compressed in a slightly different way, yielding an ever-so-slightly different archive, and because the size is almost identical and the names are the same,

[SpenceKonde]

And the digiusb drivers for windows are missing from the archives too - we only have the digiserial (CDC) drivers, which it has been clearly established do not work on post-XP windows or recent versions of the linux kernel... instead of the DigiUSB drivers they were meant to replace, but which still work whereas the CDC ones do not.... I don't know why we have the drivers that are known to not work, and don't have the drivers that do - I generally try for the other way around....

[SpenceKonde]

Okay, 1.5.0 is getting: For all linux: Repacked micronucleus binaries (just renamed and with a readmneto make these distinct from the other near identical archives that exist...

For all windows: Repacked micronucleus binaries, with the files related to DigiUSB and DigiX, and a post_install.bat file. We know that Arduino will not run that file, but it will print a warning mentioning that - specifically, mentioning the file path. So an enterprising user could track it down and run it to install the drivers.

My interpretation of the doom gloom and begging for salvation on other forums about the serial drivers is that: digiCDC is dead. It relied on a hackjob that stopped working almost as soon as it was implemented. digiCDC is deader than a doornail in the bottom of dracula's coffin. The hackjob probably never should have worked, even on XP. forbidden eldritch magic would be needed to return it to the realm of the living. (anyone have experience writing windows drivers? crickets chirp) All who wield such magic lost interest in digispark when the crowdfunding buzz passed, and haven't thought about it in years, unless they suffer from PTSD from their time working on it, which (based on comments about what it's like working on a bitbang USB implementation in one of the other topics) would not be entirely surprising. In any event, we should expect no help from anyone of that caliber, so it falls to mediocre minds like mine to try to chart a course to an approximation of serial, as that's the most plausible, flexible, and familiar interface for two-way communication of arbitrary data with an application on the host system.

However, all is not lost, as there are widespread reports of DigiUSB continuing to work fine. Provided that can be confirmed, this leaves us in a worse place than if we had CDC, by far - but compared to having neither of them, it is greatly preferable; while it requires custom software, such software can be written in python and examples appear to be available.

My interpretation of the body of the files is that DigiX is related to DigiCDC. Thus, IF observations bear out my interpretation of available information above with 1.5.0, I will remove DigiX and DigiCDC drivers from the 2.0.0 release, include the DigiUSB library and not the non-functional CDC one. If evidence contradicts my interpretation above, plans will be adjusted accordingly. Goal is for known working libraries to be included, and known non-working libraries to not be included :-) 1.5.0 is an interim release, and therefor will include all the drivers so we can bang on this shit and see what works and what doesn't, so by 2.0.0, we will know what does and does not work.

[SpenceKonde]

~Goddamn, they have made the errors it throws over post_install.bat much nastier, can't release it like that. Ugh - why do they do that? It doesn't provide any added security! when you install a core you'd better trust the author. Like any piece of software you install on a windows PC!~ Nevermind, the awful error doesn't reproduce, and we get the behavior I wanted: the message:

Warning: non trusted contribution, skipping script execution (C:\arduino-1.8.13-portable\portable\packages\ATTinyCore\tools\micronucleus\2.5\micronucleus-2.5\post_install.bat) - 

gives users a path to follow to trace down the drivers. Maybe in future version I could even split the drivers off into an extra dependancy, so it could look like:

Warning: non trusted contribution, skipping script execution (C:\arduino-1.8.13-portable\portable\packages\ATTinyCore\tools\micronucleus_drivers\1.0\run_post_install_bat\post_install.bat)

[SpenceKonde]

Raarrrrrgghhh! Okay! it's done!

[stonehippo]

Oh man, thanks for all of this @SpenceKonde! Looking forward to trying out 1.5.0 from a RPi (upgrade from 1.4 worked smoothly). Sorry I couldn't get the damn binaries all the way there for you.

[per1234]

@SpenceKonde while adding post_install.{bat,sh} support to Arduino CLI, the point was raised by Matthijs that this is really a pointless restriction, since the user is just about to run arbitrary executables from the platform anyway. So post install script now works for 3rd party boards platforms when the user is using Arduino CLI or Arduino Pro IDE.

This will also be applied to Arduino IDE as soon as the Boards Manager business code in the IDE code base is removed and replaced with a call to Arduino CLI. That process has gradually been happening with all the non-GUI code in the IDE.

[SpenceKonde]

Wonderful news!

---

Spence Konde Azzy’S Electronics

New products! Check them out at tindie.com/stores/DrAzzy GitHub: github.com/SpenceKonde ATTinyCore: Arduino support for almost every ATTiny microcontroller Contact: spencekonde@gmail.com

On Sat, Feb 27, 2021, 01:22 per1234 notifications@github.com wrote:

@SpenceKonde https://github.com/SpenceKonde while adding post_install.{bat,sh} support to Arduino CLI, the point was raised by Matthijs that this is really a pointless restriction, since the user is just about to run arbitrary executables from the platform anyway. So post install script now works for 3rd party boards platforms when the user is using Arduino CLI or Arduino Pro IDE.

This will also be applied to Arduino IDE as soon as the Boards Manager business code in the IDE code base is removed and replaced with a call to Arduino CLI. That process has gradually been happening with all the non-GUI code in the IDE.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SpenceKonde/ATTinyCore/issues/465#issuecomment-787014107, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTXEW6RTNUCV4LXMAB4MXLTBCFQ7ANCNFSM4SNYOAIA .

[nerdralph]

It looks like those MN builds are still using Tim's osccal tuning code. https://github.com/ArminJo/micronucleus-firmware/blob/master/firmware/osccalASM.S

As I mentioned above, it has a subtle bug that will sometimes choose a bad OSCCAL value if there's another low-speed device such as a keyboard or mouse on the same root hub. It is likely to be quite frustrating to users who encounter it, as it will seem almost random since it is only a problem when one of the first couple frames is non-empty. Tim helped review and optimize my new tuning code, which in addition to fixing the bug is smaller than Tim's original asm version.

AFAIK the version of MN used by Eric on the DigiSpark didn't have this bug. I think that version used Christian's tuning code, which doesn't have this bug. I even think Christian's osctune.h code may be a better choice for a bootloader than my asm OSCCAL tuner. For a USB library, my code is better because it doesn't tie up timer0. https://github.com/obdev/v-usb/tree/master/libs-device

[cpldcpu]

Hi Ralph,

indeed, we should include your excellent code. Can you submit a pull request to the MN repository?

Btw, the issue with the hub was fixed at some time, but I had to remove the code again, because of some timing issues with windows 10 afair. So far, there wwas not really a bug report regarding USB hubs that forward all traffic. It may be that they are not that widely spread.

[nerdralph]

indeed, we should include your excellent code. Can you submit a pull request to the MN repository?

I just opened a discussion about it. https://github.com/micronucleus/micronucleus/discussions/216

Btw, the issue with the hub was fixed at some time, but I had to remove the code again, because of some timing issues with windows 10 afair. So far, there wwas not really a bug report regarding USB hubs that forward all traffic. It may be that they are not that widely spread.

I think you are underestimating the extent of the bug. If you look back our email communications from May of 2014, I was having problems with OSCCAL tuning back then, but wasn't able to nail down the specifics of the bug. It's only last year after getting a better understanding of the protocol and having a good logic analyzer and scope that I was able to confirm the bug. To identify the bug I had to: 1) capture several USB enumeration sequences in order to identify the IN token for my USB low-speed mouse. 2) capture a few failed enumerations with a logic analyzer and a high-speed scope. Since the enumeration failures are intermittent (and dependent upon the interrupt interval of the other low-speed device), this took dozens of captures. 3) once I had a capture on my scope of a failed enumeration, I was able to measure the bit times transmitted by the AVR to see that they were way off 666.7ns/bit. 4) I then ran the OSCCAL tuning code on an AVR with an external xtal, with a high-speed UART debug log function added to output the attempted OSCCAL value each time through the loop.

Since the bug is intermittent (around 10-20% of the time from my experiments), many people that encountered it may have been OK with the simple unplug and re-connect "solution".

[SpenceKonde]

What I never understood is why MN needs to re-tune every time, rather than saving the correct tuning value... could even achieve that with one of the "updater" type sketches. You've got all the flash you need to write a really careful funer, cause you're gonna delete it, then it rewrites the last page of the flash (where the bootloader resides), adding the tuning value it determined, and you upload another sketch over it, and every time it starts, MN uses that saved OSCCAL value... As long as it was being used indoors from a USB port that was outputting ~5v,, there's like a 0.3v and 10 degree C window it's gotta work in... I don;t think the dependence on T or V is that strong, is it? why would we have to re-tune it? having to re-tune every start makes the whole thing way more demanding, because it means the tuner needs to be built for size and speed since it is eating up space that the sketch could be occupying and needs to be able to tune off of USB and right away be able to communicate over USB.

[nerdralph]

I do exactly that with picoboot-lib (calibrate once and then self-modify the bootloader code with the OSCCAL value). With a 8N1 UART, you have a +5.88%/-6.25% margin of error for timing. For the T85, it looks like there's a 0.6% frequency difference from 15C to 30C. The total margin of error for V-USB 16.5Mhz is +/-1.5% at best.. I suspect there may even be hubs out there that don't fully comply with the spec and require device timing to be within 0.5%, since that's the requirement for full-speed (12Mbps). OSCCAL steps on the t85 in the 16.5MHz range are around 0.7-1%, limiting it's tuning ability.

I think a one-time calibration may have the most merit for AVRs with better RC oscillators like the t84A. The frequency drift from 15-30C is only about 0.3%, and the frequency change per OSCCAL step is about 0.5%. My tuning code is limited to 5 SOFs, and can't guarantee the best OSCCAL value is chosen. A longer-running calibration done the first time the bootloader is run, could sample dozens of SOFs to pick precisely the best OSCCAL value. The reduction in the bootloader code size is ~50 bytes, making it not a high priority target. When MN merges most of my asm branch code, the reduction in bootloader flash size should be over 100 bytes. edit: the size reduction is actually over 200 bytes.

On Sat, Feb 27, 2021 at 4:13 PM Spence Konde (aka Dr. Azzy) < notifications@github.com> wrote:

What I never understood is why MN needs to re-tune every time, rather than saving the correct tuning value... could even achieve that with one of the "updater" type sketches. You've got all the flash you need to write a really careful funer, cause you're gonna delete it, then it rewrites the last page of the flash (where the bootloader resides), adding the tuning value it determined, and you upload another sketch over it, and every time it starts, MN uses that saved OSCCAL value... As long as it was being used indoors from a USB port that was outputting ~5v,, there's like a 0.3v and 10 degree C window it's gotta work in... I don;t think the dependence on T or V is that strong, is it? why would we have to re-tune it? having to re-tune every start makes the whole thing way more demanding, because it means the tuner needs to be built for size and speed since it is eating up space that the sketch could be occupying and needs to be able to tune off of USB and right away be able to communicate over USB.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SpenceKonde/ATTinyCore/issues/465#issuecomment-787128816, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNZ6V7FF2FDS3AVAYLKTTTBFG6NANCNFSM4SNYOAIA .

[cpldcpu]

Have you checked if this fixes the bug?

https://github.com/micronucleus/micronucleus/blob/29317ea0c5b55b704a42ad318c52945cf3e2359e/firmware/osccalASM.S#L150

As stated there, it was in the codebase for a long time, but caused MN to fail when Win 10 came out. That's why I removed it, because few people actually use USB 1.1 hubs and my expectation was that this would decline further over time.

Your new solution is much better because it does not extend the osccal duration so it should still work with Windows 10. So if we can include that everything should be fine.

@SpenceKonde Well, you can activate saving of oscall in the source.

The oscillator frequency changes with temperature and voltage (there can be quite some different in actual USB voltage) and you never know how marginal you are if you only save the value once. To do this properly you would have to find a way to define when this perfact osccal value is found and only save it then. Surely doable but it's quite some work to do this properly, it may add complexity to the usability of micronucleus and we may encounter new bugs along the way.

[nerdralph]

Have you checked if this fixes the bug?

https://github.com/micronucleus/micronucleus/blob/29317ea0c5b55b704a42ad318c52945cf3e2359e/firmware/osccalASM.S#L150

As stated there, it was in the codebase for a long time, but caused MN to fail when Win 10 came out. That's why I removed it, because few people actually use USB 1.1 hubs and my expectation was that this would decline further over time.

The issue is not limited to USB 1.1 hubs. I think I mentioned in one of the MN issues that I debugged the issue using a PC with an Intel chipset and USB 2.0 high-speed (i.e. 480Mbps) EHCI hubs. It only happens when there is traffic for a 2nd low-speed device on the same hub. In my case it was when I had my mouse plugged into the same hub. Since some ports are routed to the back of the PC case and some are routed to the front, it took some time to realize that the intermittent enumeration problems were primarily happening with certain ports. Years ago I was using genuine DigiSpark devices (with RSTDSBL programmed), and therefore had to unplug and re-connect the digispark to force a re-enumeration. I now know there was a 50/50 chance I'd plug it into a port on the same hub as my mouse. So this was a very tricky bug to track down, taking much more work than your typical MN and/or V-USB user is going to bother with.

USB hubs isolate traffic for different speed devices, so if a device running MN is plugged into the same port as a high-speed flash drive, the AVR will never see any of that traffic. In fact, if you could count on there being no other USB traffic on the port, you could eliminate the code in V-USB for handling the 7-bit USB device address.

As for Win 10, someone else will have to test that. I've tested on Win7 & Linux. I had Win10 installed on a machine a few years ago, but the motherboard died. When Win10 refused to run on the replacement motherboard (something to do with the license key), I just wiped the disk and installed Ubuntu.