Removal of modular list of sensors and modem options in latest examples is confusing

[fisherba]

As of the merge #212 from develop, I don't know how to use the sketches anymore, or to migrate old ones to the new versions. Can we please include the options for sensors and modems as before? I keep getting errors when trying to add them back in.

We have a deployment right after new year's and we were hoping to have the stations tested and running before the holiday and this is really slowing down testing.

[SRGDamia1]

I'm sorry. The intent of removing all of the extra code from the examples was to make them less confusing and easier for me to maintain. It seemed like the examples had become so long that there was no chance anyone would actually go through them and customize them properly.

I put the chunks of code for each sensor and modem in the wiki pages so you should be able to paste them right back. Where are you having trouble?

[fisherba]

Hi @SRGDamia1, I understand the ease of maintenance is a concern, and I fully understand the time it takes to put together documentation. I really appreciate all of the work you have done. The Modular Sensors wrapper is truly the only means I have has an end-user to deploy complicated monitoring stations that log to the internet and preserve battery, etc. Thank you.

I have looked through the documentation (the readme, wiki, and the new tutorial by SWRC) and after some effort I found the list of code snippets for different modems and sensors.

I am in this world as an end-user and I have "taught" dozens of people how to use Modular Sensors (not arduino, or monitoring, but Modular Sensors specifically). I really liked the approach of having a list of modem and sensor options to choose from and simply comment out what users do not need. My experience in teaching people was that the long list of sensors was not a source of confusion. They were able to understand it as a "menu" or a list of options to choose from. While I understand that a shorter list might be useful all around (we can't add every possible sensor out there), having multiple sensor and modem options in the example sketches is very helpful.

I find a list of multiple example sensors and modems helpful because having them in the example automatically has them located in the right part of the sketch. It shows users examples of where to add additional items. The documentation for each sensor has a long set of instructions for where to add things and there are 3-4 locations in the code to add items, depending on the sensor (include statement, variables, the paste-able code snippet, serial begins). This greatly increases the opportunity for new users to make errors and put things in the wrong places. Having to go search for each sensors slows down and complicates the getting started process. In the current version users just need to comment out the code snippet for the sensors that they are not using, which I find to be drastically more user-friendly.

I also am concerned from the standpoint of maintenance and updates that having the documentation and examples separate from the latest code updates has a greater chance of decoupling the updates from the documentation (we all know that documentation is extremely important, but even with the best intent it's hard to keep at the top of the list of things to update).

The changes in the new examples are abundant and in my attempt to take an older code and update it to utilize the latest functionality, I was completely unable to produce a code that would compile. In the interest of time, I needed to go back to an old commit of Modular Sensors and deploy without the latest functionality. So as an end-user who has potentially more experience deploying from Modular Sensors than most other users, my experience was that the updates were much less user-friendly.

[SRGDamia1]

I'll add all of the sensors back to most examples. The compile test code also has everything in it.

[SRGDamia1]

Ok, so I've made a new example called "menu_a_la_carte" which is more complete than any of the previous examples were. Let me know what you think of it.

[aufdenkampe]

@SRGDamia1, I just looked it over quickly and it looks really complete. I like it. Thanks for taking into account the concerns from @fisherba.

It also looks like somethings are simpler than they used to be. I'll probably give you a ring to have you describe to me some of the bigger architectural changes & refactoring. Thanks for all this hard work!

[SRGDamia1]

Is the new example ok? Can I close this?

[aufdenkampe]

I'm not sure that @fisherba has taken a look yet. It would be good to get her input. Also, I wouldn't mind actually programing a board from the new menu_a_la_carte.ino example, to just test it out a bit.

[SRGDamia1]

If you think the menu example is an ok solution when you're programming in the next few days, please close this.

Also, see #238 and #239

[aufdenkampe]

Will do!

[fisherba]

After a lot of sorting through the new examples, I am finally swimming again. They are definitely confusing and harder to teach to folks than the previous modular examples.

I created two new sketches in the examples, both add the DS18 and BME280 sensors to simple_logging.ino and menu_a_la_carte.ino, with all of the necessary pieces to make these ready to run. This allows new users to see see exactly where to add (or remove) sensors (respectively) from the sketches, and using split-dif, this exercise will hopefully facilitate teaching/learning. These are the sensors that we are using in the LearnEnviroDIY tutorial because they are inexpensive and useful (https://envirodiy.github.io/LearnEnviroDIY/index.html).

There are still more things to include if you have a sonar or a modbus sensor, and that will need more documentation for new users, but hopefully this will help people along.

I hope that it isn't too much trouble to maintain these two sketches in the examples. Without any sensors in simple_logging.ino it's really tricky for new users to see where they need to put the necessary pieces to run the sketch. Let me know if it's a problem.

[SRGDamia1]

It did have a sensor - the temperature sensor in the RTC. The set-up in the Maxim DS3231 block (after #include <sensors/MaximDS3231.h>) is only for using the temperature sensor portion of the clock. There's no need for that section at all if you're not using not using the DS3231 as a sensor and only using it as a clock.

[SRGDamia1]

You'd said earlier that the easiest way was to give a list of all the options and then comment out what you didn't need.. so the menu example had almost all the options.

[fisherba]

Yes, I should have written "external sensors*". My apologies.

I also really like that the new simple_logging sketch is now indeed very simple, and can be run as is. It is now a perfect intro for a new user to run their first modular sensors sketch in a way that puts the datalogger to sleep, etc. However, the very next step menu_a_la_carte is a really big step for new users. If you are asking users to learn to copy and paste from a menu, we would like a stepping stone example that demonstrates how so they can see the pattern of where the pieces fit. So, having more than one sensor option is how they can see a pattern.

I also really like how the menu_a_la_carte sketch gave all the current possible examples of possible code snippets, including calculated variables, but it also also added hundreds of code lines of modem and logger options, which is helpful for those who are testing new combinations, but quite confusing to the new user. Expanding all of the modem options is great to enable higher level users to tackle development and trouble-shooting, but it makes it harder for new users to choose their hardware and get up and running.

menu_a_la_carte sketch also added the option of logging to a couple of different portals, which again is helpful for people who are trying different combinations. I'll take it, but it was a lot of sift through and understand. It was a complete paradigm shift from the examples that were available for the last two years.

The bottom line is that grabbing code snippets from one sketch and pasting them into another (from menu_a_la_carte to simple_logging), in 2-3 places in the sketch for each sensor, is harder for the end user to tackle than just commenting out code. The user has to make sure the snippets are complete and in the right places, so there's a lot of room for error.

If you want to hear from an end-user about what makes the system accessible, I'll keep chiming in, but if it's not helpful, I'll back away and just do my best to keep swimming. I know that I don't understand what a monumental task it is to develop the library that you've made. I just know that it's not easy to use for a non-coder, and I've been on the front end of teaching it to people like me for more than a year, so I know what people can manage and what overwhelms them. I really appreciate the library because it's frankly the only way I can accomplish all of the tasks of environmental monitoring.

[SRGDamia1]

It's helpful. If people can't use the library; it's worthless.

It's the modems that are the nightmare. If they we not all there it would be so much less scary. The extra serial libraries are a little upsetting, but not nearly as bad. I've got to figure out a better way to deal with the modems. It was the modems that you specifically flagged as being the hardest to paste in - so they're all there.

[SRGDamia1]

Would it be better to put your examples in the LearnEnviroDIY instead of here?

Also, tag #238

[fisherba]

I was trying to avoid that because they are more likely to get missed (by me) when you make updates. Also, the goal of the tutorial is to point people to Modular Sensors.

We've each spent a full day restructuring the tutorial to learn and then accommodate the new example format, so just trying to figure out what is the most sustainable.

I guess I could re-do again to not depend on the twosensor examples staying in the repo. Let me know.

[SRGDamia1]

They can stay. Most updates are a directory-wide find-and-replace for me so they shouldn't be too much trouble to keep up. Can they get better names though? Maybe LearnDIY_xxx like I have the DRWI_xxx examples.

Kind-of ironic, actually, mentioning the DRWI examples. I've been maintaining them since the beginning and I don't know if they've ever been used.

[fisherba]

Naming them differently is fine with me.

[fisherba]

I created a couple of sketches that mimic simple_logging and menu_a_la_carte examples (you are welcome to rename them), that log two external sensors. For the one that sends to Monitor My Watershed, I am wondering if you can teach us (whether by just editing it or by instructing us, whichever is most efficient for you) how to remove extra stuff (or what extra stuff we can remove) to perform the functions of logging just those two external sensors to Monitor My Watershed.

In this version it would be really ok to not expose all of the code but still retain the list of modem choices (like below). If it's not possible to hide the modem code and still retain the list of modems, that's fine. People can handle sections of "you don't have to change anything here," but it's hard when they get really long. I also recognize that we need the modem pins and networking information sections, so of course those would need to remain exposed.

// ==========================================================================
//    Wifi/Cellular Modem Main Chip Selection
// ==========================================================================

// Select your modem chip - this determines the exact commands sent to it
// #define TINY_GSM_MODEM_SIM800  // Select for a SIMCOM SIM800, SIM900, or variant thereof
// #define SIM800_GPRSBEE_R6  // Select with SIM800 for - these use atypical sleep and wake fxns
// #define TINY_GSM_MODEM_SIM808  // Select for a SIMCOM SIM808 or SIM868, or variant thereof
// #define TINY_GSM_MODEM_UBLOX  // Select for most u-blox cellular modems
// #define USE_UBLOX_R410M  // Select with UBLOX for a non-XBee SARA R4 or N4 model
// #define USE_XBEE_BYPASS  // Select with UBLOX for a Digi 3G or LTE-M XBee in bypass mode
// #define TINY_GSM_MODEM_ESP8266  // Select for an ESP8266 using the DEFAULT AT COMMAND FIRMWARE
#define TINY_GSM_MODEM_XBEE  // Select for Digi brand WiFi or Cellular XBee's
// #define USE_XBEE_WIFI  // Select with XBEE for an S6B wifi XBee
// #define TINY_GSM_MODEM_M590  // Select for a Neoway M590
// #define TINY_GSM_MODEM_A6  // Select for an AI-Thinker A6, A6C, A7, A20
// #define TINY_GSM_MODEM_M95  // Select for a Quectel M95
// #define TINY_GSM_MODEM_BG96  // Select for a Quectel BG96
// #define TINY_GSM_MODEM_MC60  // Select for a Quectel MC60 or MC60E

[aufdenkampe]

I minimized the logging_to_MMW.ino example, providing a stepping stone from simple_logging.ino to menu_a_la_carte.ino, but kept all the radio functionality. This now provides something akin to logging_to_ThingSpeak.ino.

[SRGDamia1]

Do all the recent changes solve this?

[aufdenkampe]

Yes!

• The menu_a_la_carte.ino example was introduced with https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.19.6
• The logging_to_MMW.ino example was built to use with LearnEnviroDIY with https://github.com/EnviroDIY/ModularSensors/releases/tag/v0.21.3