Breakage caused by PinStatus and PinMode types

[per1234]

This project changes LOW, HIGH, INPUT, INPUT_PULLUP, and OUTPUT from macros to enums: https://github.com/arduino/ArduinoCore-API/blob/e1eb8de126786b7701b211332dda3f09aa400f35/api/Common.h#L10-L23

I'm concerned that this will cause breakage of a significant amount of existing code.

An example is the popular Keypad library. Compilation of both the original library and the version in Library Manager fails once this change is made.

In file included from E:\electronics\arduino\libraries\Keypad-master\examples\HelloKeypad\HelloKeypad.ino:10:0:

E:\electronics\arduino\libraries\Keypad-master\src/Keypad.h: In member function 'virtual void Keypad::pin_write(byte, boolean)':

E:\electronics\arduino\libraries\Keypad-master\src/Keypad.h:81:81: error: cannot convert 'boolean {aka bool}' to 'PinStatus' for argument '2' to 'void digitalWrite(pin_size_t, PinStatus)'

  virtual void pin_write(byte pinNum, boolean level) { digitalWrite(pinNum, level); }

This commonly used code will also now fail:

digitalWrite(13, !digitalRead(13));  // toggle pin 13

toggle:2:36: error: cannot convert 'bool' to 'PinStatus' for argument '2' to 'void digitalWrite(pin_size_t, PinStatus)'

   digitalWrite(13, !digitalRead(13));  // toggle pin 13

I understand that the root cause of these errors is bad code and that any code which followed best practices will have no problems with this change. However, I fear there is a lot of bad code in widespread use that currently works fine. In the case of the Keypad library, it is unlikely it can even be fixed since Chris--A has gone AWOL. I'm sure there are other such abandoned projects.

I do like the spirit of this change (though lumping CHANGE, FALLING, and RISING into PinStatus is questionable). I'm open to being convinced that it's worth the breakage and, if so, I'm willing to help ease the transition by providing user support and submitting PRs to fix broken code. I just think this warrants some consideration before ArduinoCore-API goes into more widespread use.

Some previous discussion on the topic:

• http://forum.arduino.cc/index.php?topic=455579 (from here onward)
• https://forum.arduino.cc/index.php?topic=584322
• http://forum.arduino.cc/index.php?topic=602250
• https://forum.arduino.cc/index.php?topic=621429
• https://forum.arduino.cc/index.php?topic=627883.msg4319254#msg4319254
• https://forum.arduino.cc/index.php?topic=659624
• https://github.com/arduino/ArduinoCore-megaavr/issues/68

[PaulStoffregen]

I can confirm more code in the wild which does an ifdef check on INPUT_PULLUP (and falls back to pinMode & digitalWrite), because INPUT_PULLUP was added to Arduino later than the others.

[matthijskooijman]

I can confirm more code in the wild which does an ifdef check on INPUT_PULLUP (and falls back to pinMode & digitalWrite), because INPUT_PULLUP was added to Arduino later than the others.

One way to fix that, without sacrificing the enum, is to add #define INPUT_PULLUP INPUT_PULLUP. I think we've used this trick for other macros converted to constants in the AVR core before (or at least discussed it).

As for the convert-from-boolean point, I guess one could introduce a class type to wrap these constants and add the appropriate conversion operators on them, though that might become a bit more complex than one would like. It does keep the advantage of separate types for each of these. It would also allow splitting PinStatus and SignalFlank (or whatever), and still allow using LOW and HIGH for both by adding implicit conversions.

[Coding-Badly]

The digitalRead can be handled with an operator...

bool operator ! (const PinStatus &v)
{
  return v == HIGH ? LOW : HIGH;
}

[Coding-Badly]

The digitalWrite can handled with an overload...

void digitalWrite( unsigned pin, PinStatus value )
{
...
}

void digitalWrite( unsigned pin, bool value )
{
  digitalWrite( pin, value ? HIGH : LOW );
}

Ideally, the unsigned / bool function is placed in a header file and marked to be inlined.

[Floessie]

Maybe a stupid question: Why does digitalWrite() take a PinStatus at all? CHANGE, FALLING, and RISING make no sense, or do they? As PinStatus is a class-less enum, providing only a void digitalWrite(unsigned pin, bool value) would solve the problem IMHO.

[WestfW]

I agree with Floessie - having digitalRead/digitalWrite use pinStatus() is no better than the code that expects/passes booleans. There is no reason to overload the enum with functionality from two separate sets of functionality...

[per1234]

Today another report came in of a popular library broken by this: https://bitbucket.org/fmalpartida/new-liquidcrystal

Minimal demonstration sketch:

#include <LiquidCrystal_I2C.h>
void setup() {}
void loop() {}

E:\arduino\libraries\Newliquidcrystal_1.3.5\LiquidCrystal.cpp: In member function 'virtual void LiquidCrystal::send(uint8_t, uint8_t)':

E:\arduino\libraries\Newliquidcrystal_1.3.5\LiquidCrystal.cpp:125:48: error: cannot convert 'bool' to 'PinStatus' for argument '2' to 'void digitalWrite(pin_size_t, PinStatus)'

    digitalWrite( _rs_pin, ( mode == LCD_DATA ) );

                                                ^

E:\arduino\libraries\Newliquidcrystal_1.3.5\LiquidCrystal.cpp: In member function 'void LiquidCrystal::writeNbits(uint8_t, uint8_t)':

E:\arduino\libraries\Newliquidcrystal_1.3.5\LiquidCrystal.cpp:305:48: warning: invalid conversion from 'int' to 'PinStatus' [-fpermissive]

       digitalWrite(_data_pins[i], (value >> i) & 0x01);

                                                ^

In file included from C:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.6.210\cores\arduino/api/ArduinoAPI.h:52:0,

                 from C:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.6.210\cores\arduino/Arduino.h:23,

                 from E:\arduino\libraries\Newliquidcrystal_1.3.5\LiquidCrystal.cpp:36:

C:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\megaavr\1.6.210\cores\arduino/api/Common.h:104:6: note:   initializing argument 2 of 'void digitalWrite(pin_size_t, PinStatus)'

 void digitalWrite(pin_size_t pinNumber, PinStatus status);

      ^

Reported at: http://forum.arduino.cc/index.php?topic=595987

[facchinm]

Could something like this work for everyone? (it is from the namespace_arduino branch, hence the namespace stuff :slightly_smiling_face: https://github.com/arduino/ArduinoCore-API/blob/55cbc081bb5bcf1244743c3387b85ba56d0fe4b2/api/Compat.h#L1-L16

[per1234]

@facchinm this is probably a dumb question, but it's not clear to me exactly how this would be implemented. I get the idea of overloading the functions. But pinMode and digitalWrite are C, which doesn't support function overloading in this manner.

I was trying to modify the Arduino megaAVR Boards core code to test this solution against the known issues reported so far in the thread. I tried just replacing the api folder with the one from the namespace_arduino branch of ArduinoCore-API, but then I get the same issues as well as some new unrelated errors.

[facchinm]

@per1234 it's not a dumb question at all! In the namespace branch all implementations are C++ only, and only one of them is declared extern "C" so it can be used from any C file. In this case, the (non compact) implementation gets exposed, while the other one is only available in C++ files. Maybe it's not the most elegant approach but it should solve 99% of the problems (if the nonstandard code lives in the sketch or in a c++ library)

[per1234]

Thanks for the explanation! I agree that the amount of C code in existence affected by this issue is probably very small. I hadn't considered that the solution would be limited to C++/.ino.

[MCUdude]

I'll have to agree with @WestfW and @Floessie on this one.

I understand that @facchinm tries to provide a solution that will keep the new implementation, but what do we need this new implementation in the first place? It only leaves ut with a workaround that's not compatible with C libraries, not any benefits as far as I can see.

[JChristensen]

I'd be interested to know what problem is solved by introducing the PinStatus type. If this is about type safety, then I think there is such a thing as getting carried away. For example, it seems perfectly natural to pass a bool as the second argument to digitalWrite(). In that case, introducing PinStatus is not only confusing but seems like reinventing the wheel, as the language already provides a completely adequate type. If in fact this change is about type safety, then I'd have to wonder how big the problem was in the first place.

Perhaps worse, the current situation is inconsistent between various boards, with megaAVR architecture demanding a PinStatus type (e.g. for digitalWrite), and regular AVR being more permissive. In fact, PinStatus is not defined for the AVR architecture.

[Coding-Badly]

Jack! I hope you've been well!

Excellent question. I can imagine two possibilities.

1. bool is severely limited in the amount of information that it can carry. A pin status type can carry the same information and later, if necessary, be changed to carry additional information. It's a way of "future proofing" the interface.

2. Having a type specific to pin status allows overloading. It is possible for digitalWrite(pin, bool) to have different behaviour than digitalWrite(pin, PinStatus). (I can't think of a valid reason to do that but I also have not put much thought into it.)

[bxparks]

Question 1:

Looking at $ARDUINO_IDE/portable/packages/arduino/hardware/megaavr/1.8.1/cores/arduino/wiring_digital.c, I see that digitalWrite() now accepts 3 values of PinStatus (HIGH, LOW and CHANGE). It seems that CHANGE causes the pin to toggle, and CHANGE is defined to be an enum of 2.

However, on a normal AVR, CHANGE is defined to be 1 (see $ARDUINO_IDE/hardware/arduino/avr/cores/arduino/Arduino.h) so digitalWrite(pin, CHANGE) compiles, but has the same effect as digitalWrite(pin, HIGH).

Is this intended?

Question 2:

Is there a #define macro that identifies the megaAVR boards which use this new PinMode and PinStatus API, so that I can fix my library for the megaAVR?

[facchinm]

Hi @bxparks

1) probably implementing CHANGE as pin toggle was not the best idea ever; anyway, not using it doesn't hurt :slightly_smiling_face: What's your use case? 2) you can target any core implementing PinMode and PinStatus with #if (ARDUINO_API_VERSION >= 10000)

[bxparks]

Hi @facchinm,

I don't use digitalWrite(pin, CHANGE)... I was trying to deduce the valid range of PinStatus for the new digitalWrite() and digitalRead(). Previously both of them were documented to handle only 2 values: HIGH and LOW. Now digitalWrite() accepts 3 values. So I'm wondering, will digitalRead() ever return any other values of PinStatus?

The reason for asking is that my AceButton library stores a value of digitalRead() that represents "UNKNOWN", in other words, the state of the pin before any digitalRead() has been performed. But the documentation for digitalRead() does not define the value of HIGH or LOW, so I have a compile-time check like this:

#if HIGH != 1
  #error HIGH must be defined to be 1
#endif
#if LOW != 0
  #error LOW must be defined to be 0
#endif

Which enforces that my "UNKNOWN" value is different from either HIGH or LOW. Obviously for the megaAVR, these checks no longer work. And obviously, I cannot use PinStatus to store my digitalRead() result, because "UNKNOWN" cannot be represented by the enum.

Thanks for the pointer to ARDUINO_API_VERSION. Is it correct to assume that eventually even the normal AVRs will migrate to the new API, with the PinStatus and PinMode enums?

[facchinm]

The whole idea of HIGH and LOW is to hide the implementations details (HIGH could be 0 and LOW 0x12 in some random funky architecture, but your Arduino code will run seamlessly). Expecting them to be 0 and 1 (with no other value allowed) breaks this assumption, so maybe there's a better way to enforce the UNKNOWN field :wink:

[PaulStoffregen]

Overloading digitalWrite will break much of the Arduino ecosystem. A tremendous amount of code has been built up over the last decade equating digitalWrite & digitalRead values with the C/C++ convention of zero representing logical false to mean LOW and non-zero representing logical true to mean HIGH.

[facchinm]

@PaulStoffregen of course most (all?) the architectures are sane and HIGH is 1 while LOW is zero but since the APIs don't specify the value of those macros/enum/whatever (and never did) the code runs "by chance" :slightly_smiling_face:

[bxparks]

I guess I'm having a hard time seeing what advantages the PinStatus API offers that outweighs the cost breaking backwards compatibility. We've already seen that the one new feature that it does provide, the ability to toggle using digitalWrite(pin, CHANGE) cannot be used, since it breaks compatibility with all other platforms.

I also understand that writing:

pinStatus = (pinStatus != LOW)? LOW : HIGH;

is probably more clear to most people than

pinStatus ^= 0x1;

but it would be nice to have the option of writing the second form in places where it makes sense (e.g. deep inside a library where app developers are not expected to visit often). The XOR can save a few bytes of flash memory and CPU cycles, since the compiler cannot optimize the first into the second. (Interestingly, when I wrote the first version using ?: operator, I actually wrote it wrong initially and flipped the LOW and HIGH. That expression is not 100% easy to get correct. Hmm.)

With regards to UNKNOWN, yes it's always possible to use a second byte to store the bool isUnknown flag. But that's using 2 bytes to store 2 bits of information (LOW, HIGH, UNKNOWN), instead of just one byte. Now, if PinStatus contained a PinStatus::UNKNOWN whose value was guaranteed to be never returned by digitalRead(), then PinStatus would actually provide some value to me.

In any case, it looks like the new PinStatus API was introduced almost 2 years ago, so the ship has sailed? I will use

#if (ARDUINO_API_VERSION >= 10000)
...
#endif

to target different platforms appropriately.

[WestfW]

the APIs don't specify the value of those macros/enum/whatever (and never did) the code runs "by chance" wiring_shift.c in the arduino core ( https://github.com/arduino/ArduinoCore-avr/blob/master/cores/arduino/wiring_shift.c#L32 ) passes a boolean to digitalWrite(), and the Firmata library uses 0/non-zero: https://github.com/firmata/arduino/blob/master/Boards.h#L970

[PaulStoffregen]

That sort of code is pervasive throughout the Arduino ecosystem.

I fear these API changes could bring widespread misery.

[bxparks]

I tried to protect myself against those assumptions with my compile-time checks, e.g.:

#if HIGH != 1
  #error HIGH must be defined to be 1
#endif
#if LOW != 0
  #error LOW must be defined to be 0
#endif

But even that breaks under this new API. :-/

[WestfW]

That's presumably because HIGH and LOW are now part of an enum rather than mere constants.

You can probably do:

void unexpected_HIGH_or_LOW_value(void) __attribute__ (( error("") ));
// ...
void MyFunction (...) {
   if (HIGH != 1 || LOW != 0)
      unexpected_HIGH_or_LOW_value();
//    ...
}

Since HIGH and LOW are still compile-time constants, this won't normally produce any actual code, but they no longer need to be preprocessor-time constants. (There are a bunch of places where it might be useful to use this sort of thing in the Arduino code. But that's for another time/issue!)

[bxparks]

(This is getting slightly off topic, but hopefully it's useful to someone else facing a similar problem with compile-time checking of HIGH and LOW values)

@WestfW: Thanks for the pointer! I didn't know about the error attribute. I discovered 3 problems with your solution unfortunately: 1) The compiler complains about an unused MyFunction() if this check is the only code inside. 2) I have to create a new unexpected_HIGH_or_LOW_value() function for each compile-time check that I want to perform. 3) The error attribute is a g++ only extension, clang++ does not recognize it. (This is a problem for me because I try to write portable code, so that I can compile and run my Arduino unit tests on Linux (g++ or clang++) and MacOS (clang++)).

After reading the GCC guide on __attributte__ on the extern char [(condition) ? 1 : -1]; trick, and this blog on catching compile-time errors, I came up with the following:

#define CONCAT_(x, y) x##y
#define CONCAT(x,y) CONCAT_(x,y)
#define COMPILE_TIME_ASSERT(cond, msg) \
    extern char CONCAT(compile_time_assert, __LINE__)[(cond) ? 1 : -1];
COMPILE_TIME_ASSERT(HIGH == 1, "HIGH must be 1")
COMPILE_TIME_ASSERT(LOW == 0, "LOW must be 0")

Works on both a #define HIGH and an enum value of HIGH.

[Avamander]

Can this be fixed as soon as possible? It's unnecessarily incompatible with a large majority of other platforms, megaavr is just an annoying outlier here.

[Avamander]

@facchinm Can this god-awful change be reverted, there is really zero benefit to this breakage.

[JM-FRANCE]

Part of me would say "too bad for those who never understood that type coherence is an important concept"... but I would vote against getting that live too.

I've seen too much bad code that has been confusing a truth value (a bool) with a voltage status value (HIGH and LOW or even UNKNOWN) that I fear this change would have terrible impacts on newcomers trying to use random libraries and books that have been relying on the "expected" behavior.

And the benefit is very limited...

[MCUdude]

Since the devs aren't listening. I'm tired of this too, so I will get rid of this in the MegaCoreX repo. This core is 100% compatible with the Uno Wifi Rev2 and the Nano Every.

@SpenceKonde might do it for megaTinyCore as well.

[facchinm]

@MCUdude this commit should solve all the issues about this topic (and is being used by mbed core, Nano 33 BLE, so there's some validation of the method). The megaAVR core didn't receive any update in the meantime but I think we could release a new version of the core if needed.

[MCUdude]

digitalRead is still not returning an int, but rather PinStatus. But this might be less of a problem.

The megaAVR core didn't receive any update in the meantime but I think we could release a new version of the core if needed.

Yes, so much yes!

[bperrybap]

@facchinm That commit solves the issue for digitalWrite() and pinMode() booleans, but it does not solve the issues related to code wanting to check for the existence of INPUT_PULLUP i.e. I have a library that breaks with an error message because INPUT_PULLUP is not defined. My library code explicitly checks for the existence of that macro. Others noted this issue as well.

Common.h could resolve this by adding these defines just after the pinMode enum.

// create equivalant macro symbols for PinMode enums
#define INPUT INPUT
#define OUTPUT OUTPUT
#define INPUT_PULLUP INPUT_PULLUP
#define INPUT_PULLDOWN INPUT_PULLDOWN

Could we add this to Common.h

[mohdrais]

@bperrybap Will it be safe to modify the common.h in the system? By the way,I cannot find the right common,h file in my computer, i.e C:\Program Files (x86)\Arduino\hardware\tools\avr\avr\include\avr\common.h

[bperrybap]

@mohdrais , It should be. This is what I have done for INPUT_PULLUP until it is fixed. It isn't "common.h" but "Common.h" and it is down under the packages directory. No clue where that is on Windows. On linux, with the current megaavr core release it is: ~/.arduino15/packages/arduino/hardware/megaavr/1.8.6/cores/arduino/api/Common.h

I added the macros just below the pinMode enum

// create equivalant macro symbols for PinMode enums
#define INPUT INPUT
#define OUTPUT OUTPUT
#define INPUT_PULLUP INPUT_PULLUP
#define INPUT_PULLDOWN INPUT_PULLDOWN

To provide full backward compatibility for the previously existing macros, there are other enums that should have symbols/macros created as well.

[bxparks]

I maintain a handful of Arduino libraries which are moderately popular. I continue to get bug reports that my libraries don't work the MegaAVR, but they work for many other Arduino-like boards (AVR, ESP8266, ESP32, Teensy, SAMD). I don't have the time or motivation to debug the backwards incompatible breakages caused by the MegaAVR tool chain. I don't own a MegaAVR, nor do I plan on getting one in the future.

I am inclined to make the MegaAVR explicitly unsupported by my libraries. I'm curious to hear the opinions of any other other libraries maintainers who were/are bitten by these MegaAVR breakages.

[SpenceKonde]

Instead of doing that, refer users to the https://github.com/MCUdude/MegaCoreX which supports the ATmega4809 (including Uno WiFi Rev. 2 and Nano Every), and doesn't have this mess (I think it also adds a bunch of pwm pins, and does a variety of things better too). I have likewise reverted my https://github.com/SpenceKonde/megaTinyCore to do pinMode/etc the old way as well. Marking megaavr as unsupported architecture will also impact users of these third party cores on which they would otherwise work.

[bxparks]

Ah ok, I'm not familiar with those other megaAVR cores (MegaCoreX and megaTinyCore). If they use the same old programming framework as all the other boards, then my libraries should work just fine with those. I guess I should narrow down my blacklist: How about I exclude the 2 entries I see under "Arduino megaAVR Boards", i.e. Nano Every and the UNO WiFi Rev2.

To be clear, I have nothing personal against those boards. But I maintain my libraries on my own time, for free, for fun. I see nothing fun in spending my limited time and patience, to track down random, unknown, undocumented, backwards incompatible breakages for these 2 megaAVR boards that I don't care about.

[SpenceKonde]

I think that would work; I would also suggest a note in the README of the relevant libraries referring users of those boards to MegaCoreX (you can use it with the Nano Every/Uno WiFi Rev. 2 - you just tell the IDE it's a 4809, and it has a pin mapping option that matches the pin numbering on the officials board).

[matthijskooijman]

I think that the suggestion from @bperrybap is actually good, just #define these values as themselves for compatibility. IIRC we already did the same thing for some other constant that was turned into a const or enum elsewhere (maybe in some Arduino core, or maybe some other library, don't remember exactly), and it seems to be a safe but effective way to allow #ifdef without messing up uses of these names in other contexts (i.e. it actually fixes some problems that occur when defining these constants directly when code tries to use the same name as a variable name, for example).

[matthijskooijman]

Also, these defines should maybe be put in Compat.h rather than Common.h to mark them as deprecated and not recommended for use? For some thoughts about phasing out such compatibility stuff, see https://github.com/arduino/arduino-cli/issues/956

[Michael-Brodsky]

You're absolutely right, the changes to the megaavr api broke tons of my code and I'm now having to sift through everything and put in conditionals. Nice of them to tell us, and there's no docs to speak of. Right on!!

[bperrybap]

You're absolutely right, the changes to the megaavr api broke tons of my code and I'm now having to sift through everything and put in conditionals. Nice of them to tell us, and there's no docs to speak of. Right on!!

While I do agree that the documentation could be better, most code that breaks is code that abused the digital i/o function API by making assumptions on the values of HIGH and LOW and the underlying implementation and used the digital pin i/o API in a way that was not assured/guaranteed by the existing minimal documentation. i.e. the code did things in a way that was not supported by the existing documentation so it was not portable and got burned by abusing the API rather than use the API the way it was documented.

I do feel for the users that get caught in the middle.

The only thing that tripped up my library was the lack of a macro define for INPUT_PULLUP. I got caught & burned by using something that was not documented. i.e. checking for the existence of macro INPUT_PULLUP I only did it in one diagnostic test example so it didn't affect the overall library or any other of its examples.

Since INPUT_PULLUP became part of Arduino in 1.0.1 ( the next release after 1.0) I now no longer support Arduino 1.0 and will just let the one example ungracefully fail should INPUT_PULLUP not exist This allows the code to run on all platforms that have the INPUT_PULLUP symbol regardless of how it is declared. So far, I haven't run into a platform that doesn't have this symbol.

A much bigger concern for me that is related to this is that the Arduino IDE allows the tools in one platform to override the tools in another platform if the tools are newer and are for the same architecture.

This new behavior caused the installation of the megaavr platform to break other unrelated AVR platforms. IMO, one platform should not be able to override something in another platform. i.e. the installations of one platform should not be allowed to modify, override, or even break things in another installed platform. Each platform should be separate and independent of the any other installed platform, even if they are the same architecture.

--- bill

[Michael-Brodsky]

My problems are much more serious. They eliminated the C std lib header , had to roll my own. There’s a reason why the Standard Library is called the Standard Library and has been around for generations. They also changed several api function signatures without even documenting it, they don’t even have the signatures in the docs, just the arg list. There are conflicts with some of the WiFi headers, and the list goes on, some functions bomb during initialization but can be called when running w/ the same params, like pinMode???

As a professional C++ OOP developer, I don’t use macros, ever. I’ve rolled my own basic C++ std lib for the Arduino and have even replaced many of their macros w/ constexpr. These never fail in a proper implementation. Also more problems emerge when I test the code for errors/warnings using the Arduino IDE. Visual Studio and Atmel Studio don’t complain nearly as much.

The problem isn’t our code, it’s that they change the api w/o documentation or workarounds, replace things w/ non-standard stuff and so on. Maybe I’m expecting too much, but this sorta stuff would never fly in the real world. “Hello, Oracle? Sorry but we changed the System V kernel and now all your stuff will be off by one. Good Luck!”

Regards, Michael B.

The Sent from my iPhone

On Apr 20, 2022, at 10:57 AM, bperrybap @.***> wrote:

 You're absolutely right, the changes to the megaavr api broke tons of my code and I'm now having to sift through everything and put in conditionals. Nice of them to tell us, and there's no docs to speak of. Right on!!

While I do agree that the documentation could be better, most code that breaks is code that abused the digital i/o function API by making assumptions on the values of HIGH and LOW and the underlying implementation and used the digital pin i/o API in a way that was not assured/guaranteed by the existing minimal documentation. i.e. the code did things in a way that was not supported by the existing documentation so it was not portable and got burned by abusing the API rather than use the API the way it was documented.

I do feel for the users that get caught in the middle.

The only thing that tripped up my library was the lack of a macro define for INPUT_PULLUP. I got caught & burned by using something that was not documented. i.e. checking for the existence of macro INPUT_PULLUP I only did it in one diagnostic test example so it didn't affect the overall library or any other of its examples.

Since INPUT_PULLUP became part of Arduino in 1.0.1 ( the next release after 1.0) I now no longer support Arduino 1.0 and will just let the one example ungracefully fail should INPUT_PULLUP not exist This allows the code to run on all platforms that have the INPUT_PULLUP symbol regardless of how it is declared. So far, I haven't run into a platform that doesn't have this symbol.

A much bigger concern for me that is related to this is that the Arduino IDE allows the tools in one platform to override the tools in another platform if the tools are newer and are for the same architecture.

This new behavior caused the installation of the megaavr platform to break other unrelated AVR platforms. IMO, one platform should not be able to override something in another platform. i.e. the installations of one platform should not be allowed to modify, override, or even break things in another installed platform. Each platform should be separate and independent of the any other installed platform, even if they are the same architecture.

--- bill

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[bperrybap]

@Michael-Brodsky Yeah that sucks. Were you around for the Arduino 1.0 release debacle? This stuff pinStatus & pinMOde API stuff hurts but nearly as bad as when they chose to intentionally break 100% of all the existing third party code & libraries when moving from the release candidate to the final release.

I'd "just say no" and drop support for the megaAVR platform until they fix a few things. And don't forget that installing that platform can break other AVR platforms which can make the problem even worse. In the earlier days of the megaAVR platform, I told my library users not only that that the library didn't support it but not to install it as it caused other issues.

While this github issue was about the PinStatus & pinMode stuff, some of the issues I'm seeing in this github issue are do to some of the other things that have been done, or not done, in the megaavr platform beyond the PinStatus and pinMode types.

I'm still not sure of the value of using stricter types like enums in the API at this point in time. Yeah it would have made sense 10+ years ago, but at this point in time it seems to have much more downside than upside, given the desire/need for a certain level of backward compatibility for existing code. There are now function overloads to get around having to use the new types so what value do the new types really bring to the table even for the long term?

IMO, the only way out of these types of issues would have been to plan for API migration and make these types of significant changes with a new Arduino version like say 2.0 And then have an option in the IDE GUI to set a compile time define to enable pre 2.0 API compatibility to allow existing code to continue to work for some period of time - which might be forever.

[Michael-Brodsky]

Hey Bill,

No I wasn’t around for v 1.0. I just started this a couple few years ago, developed some simple industrial automation apps to see how they’d work https://youtu.be/fCEVc_z0KCA. Turns out I had quite a bit of success and have kept at it. My current project is a sorta open source version of LabJack on an Arduino along with a Windoze desktop app that forms the basis of an HMI/SCADA system. I hope to post a demo on YT this coming weekend.

The problems appeared when I bought a WiFi Rev 2 to demo the system’s over-the-air performance and everything broke. I’ve managed to get everything working, but I have a lot of time invested in this project and hunting down api bugs throws me off my train of thought and the problem at ha I’ve raised these issues on the Arduino Forums with about zero luck. You’re actually the first person I’ve met that has an understanding of the issues. I like some of your ideas like the overloads thing or even going further to templates. As long as it stayed within the realm of template argument deduction, users wouldn’t see a difference.

Anyway, glad to have run into you. Would like to stay in touch and discuss this and other topics further.

Cheers

Sent from my iPhone

On Apr 20, 2022, at 1:13 PM, bperrybap @.***> wrote:  @Michael-Brodsky Yeah that sucks. Were you around for the Arduino 1.0 release debacle? This stuff pinStatus & pinMOde API stuff hurts but nearly as bad as when they chose to intentionally break 100% of all the existing third party code & libraries when moving from the release candidate to the final release.

I'd "just say no" and drop support for the megaAVR platform until they fix a few things. And don't forget that installing that platform can break other AVR platforms which can make the problem even worse. In the earlier days of the megaAVR platform, I told my library users not only that that the library didn't support it but not to install it as it caused other issues.

While this github issue was about the PinStatus & pinMode stuff, some of the issues I'm seeing in this github issue are do to some of the other things that have been done, or not done, in the megaavr platform beyond the PinStatus and pinMode types.

I'm still not sure of the value of using stricter types like enums in the API at this point in time. Yeah it would have made sense 10+ years ago, but at this point in time it seems to have much more downside than upside, given the desire/need for a certain level of backward compatibility for existing code. There are now function overloads to get around having to use the new types so what value do the new types really bring to the table even for the long term?

IMO, the only way out of these types of issues would have been to plan for API migration and make these types of significant changes with a new Arduino version like say 2.0 And then have an option in the IDE GUI to set a compile time define to enable pre 2.0 API compatibility to allow existing code to continue to work for some period of time - which might be forever.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[bxparks]

For what it's worth, I decided to explicitly blacklist the ArduinoCore-API for most the 15-20 Arduino libraries that I publish publicly, using something like the following at the top of the main header file:

#if defined(ARDUINO_API_VERSION)
#error Platforms using ArduinoCore-API not supported
#endif

One, it gives a short, understandable error message to the end-user, instead of hundreds of lines of inscrutable compiler errors.

Two, it completely stopped the repeated bug tickets against my libraries, saying "Your libraries does not work with [MegaAVR | Nano Every | RaspberryPi Pico | MKR | etc...]".

Three, it saved me dozens (maybe hundreds?) of hours of development and debugging time that I don't have to spend porting my libraries to the new Arduino API, and hundreds of dollars for buying the various microcontrollers that I no longer have to test and validate.

[PaulStoffregen]

FWIW, I'm keeping an eye on this with concern about how this is likely to affect the Arduino ecosystem, and eventually what I'll need to support on Teensy boards.

[Michael-Brodsky]

Hi guys,

For what it's worth, after hunting down all the relevant megaavr bugs and refactoring my code to get it working, I compiled a list of resolved and persistent issues. Just providing these in case they're useful to someone.

This is a list of issues encountered using the megaavr architecture and my current fixes:

Problem 1: Bad or missing C Standard Library header Solution: Wrote my own that includes about a dozen of the functions that my libs/apps depend on (might be ANSI/ISO compliant, maybe, kinda, sorta).

Problem 2: Macro definitions of "pin states" (LOW,HIGH,FALLING,RISING,etc.) redefined as an enumerated type PinStatus' in ..\arduino\hardware\megaavr\1.8.7\cores\arduino\api\Common.h Solution: Conditionals to detect architecture,using PinStatus = int;' for non-megaavr architectures, and lots of `static_cast(...)'.

Problem 3: avr WiFi api generates errors/doesn't compile when used according to Arduino example sketches. Solution: Dependencies are sensitive to the order in which they're included.

Problem 4: Signature of WiFi::begin(...) function changed ssid parameter from type char to const char. Solution: Conditional typedefs.

For both problems 3 & 4, I wrote a header to take care of the dependency order and typedefs:

include

if defined ARDUINO_ARCH_MEGAAVR // megaavr uses WiFiNINA:

include

include

using ssid_t = const char; // WiFiNINA ssid param is type const char.

else // everything else uses whatever these are,

include // and they must be included in this order:

include

include

include

using ssid_t = char; // everything else ssid param is type char.

endif // defined ARDUINO_ARCH_MEGAAVR

The following is a list of persistent issues on both avr and megaavr architectures. They're compiler/uploader warnings that seem to all be related to the WiFi and Ethernet api's.

AVR and MEGAAVR: In file included from C:\Program Files (x86)\arduino-1.8.16\libraries\Ethernet\src\Dns.cpp:8:0: C:\Program Files (x86)\arduino-1.8.16\libraries\Ethernet\src\Dns.cpp: In member function 'uint16_t DNSClient::BuildRequest(const char*)': C:\Program Files (x86)\arduino-1.8.16\libraries\Ethernet\src\utility/w5100.h:457:25: warning: result of '(256 << 8)' requires 18 bits to represent, but 'int' only has 16 bits [-Wshift-overflow=]

define htons(x) ( (((x)<<8)&0xFF00) | (((x)>>8)&0xFF) )

                  ~~~^~~

C:\Program Files (x86)\arduino-1.8.16\libraries\Ethernet\src\Dns.cpp:164:18: note: in expansion of macro 'htons' twoByteBuffer = htons(QUERY_FLAG | OPCODE_STANDARD_QUERY | RECURSION_DESIRED_FLAG); ^~~~~

AVR only: In file included from C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src/WiFi.h:26:0: C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src/utility/wl_definitions.h:42:0: warning: "MAX_SOCK_NUM" redefined

define MAX_SOCK_NUM 4

C:\Program Files (x86)\arduino-1.8.16\libraries\Ethernet\src/Ethernet.h:39:0: note: this is the location of the previous definition

define MAX_SOCK_NUM 8

C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src\utility\wifi_drv.cpp: In static member function 'static uint8_t WiFiDrv::getEncTypeNetowrks(uint8_t)': C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src\utility\wifi_drv.cpp:451:10: warning: converting to non-pointer type 'uint8_t {aka unsigned char}' from NULL [-Wconversion-null] return NULL; ^~~~

C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src\utility\wifi_drv.cpp: In static member function 'static int32_t WiFiDrv::getRSSINetoworks(uint8_t)': C:\Program Files (x86)\arduino-1.8.16\libraries\WiFi\src\utility\wifi_drv.cpp:476:10: warning: converting to non-pointer type 'int32_t {aka long int}' from NULL [-Wconversion-null] return NULL; ^~~~

MEGAAVR only Uploader generates the following warning:

avrdude: WARNING: invalid value for unused bits in fuse "fuse5", should be set to 1 according to datasheet This behaviour is deprecated and will result in an error in future version You probably want to use 0xcd instead of 0xc9 (double check with your datasheet first).

Cheers!

Michael B.

On Thu, 28 Apr 2022 at 15:33, Paul Stoffregen @.***> wrote:

FWIW, I'm keeping an eye on this with concern about how this is likely to affect the Arduino ecosystem, and eventually what I'll need to support on Teensy boards.

— Reply to this email directly, view it on GitHub https://github.com/arduino/ArduinoCore-API/issues/25#issuecomment-1112673914, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSCUI4XHYEOD7DRLDM5DILVHL72NANCNFSM4GM5J24Q . You are receiving this because you were mentioned.Message ID: @.***>

[Michael-Brodsky]

Here's some suggestions to fix the `bit shift overflow' and 'duplicate definition' warnings:

// TYPE INFERENCE

template auto bitShiftThingy(T x) -> decltype(((x << 8) & 0xFF00) | ((x >> 8) & 0xFF)) { return ((x << 8) & 0xFF00) | ((x >> 8) & 0xFF); }

// ENCAPSULATED COMPILE-TIME CONSTANTS

class EtherMonster { public: static constexpr uint8_t MaxSockNum = 8;

};

class WiFiGuy { public: static constexpr uint8_t MaxSockNum = 4;

};

This compiles without errors or warnings (it does however require 64-bit type support for the println() function if you want to print intmax_t. I use one in my libs.)

[image: Arduino IDE.png]

On Wed, 20 Apr 2022 at 10:57, bperrybap @.***> wrote:

You're absolutely right, the changes to the megaavr api broke tons of my code and I'm now having to sift through everything and put in conditionals. Nice of them to tell us, and there's no docs to speak of. Right on!!

While I do agree that the documentation could be better, most code that breaks is code that abused the digital i/o function API by making assumptions on the values of HIGH and LOW and the underlying implementation and used the digital pin i/o API in a way that was not assured/guaranteed by the existing minimal documentation. i.e. the code did things in a way that was not supported by the existing documentation so it was not portable and got burned by abusing the API rather than use the API the way it was documented.

I do feel for the users that get caught in the middle.

The only thing that tripped up my library was the lack of a macro define for INPUT_PULLUP. I got caught & burned by using something that was not documented. i.e. checking for the existence of macro INPUT_PULLUP I only did it in one diagnostic test example so it didn't affect the overall library or any other of its examples.

Since INPUT_PULLUP became part of Arduino in 1.0.1 ( the next release after 1.0) I now no longer support Arduino 1.0 and will just let the one example ungracefully fail should INPUT_PULLUP not exist This allows the code to run on all platforms that have the INPUT_PULLUP symbol regardless of how it is declared. So far, I haven't run into a platform that doesn't have this symbol.

A much bigger concern for me that is related to this is that the Arduino IDE allows the tools in one platform to override the tools in another platform if the tools are newer and are for the same architecture.

This new behavior caused the installation of the megaavr platform to break other unrelated AVR platforms. IMO, one platform should not be able to override something in another platform. i.e. the installations of one platform should not be allowed to modify, override, or even break things in another installed platform. Each platform should be separate and independent of the any other installed platform, even if they are the same architecture.

--- bill

— Reply to this email directly, view it on GitHub https://github.com/arduino/ArduinoCore-API/issues/25#issuecomment-1104181950, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSCUI3CZOJXGYI4FDL6ZVLVGAZOTANCNFSM4GM5J24Q . You are receiving this because you commented.Message ID: @.***>