Wait until the modem is ready before calling transmitText

[K4KDR]

While everything works perfectly when testing uplink transmissions via an Arduino Due connected as 'Serial1' while I have a USB/Serial connection to my computer (to observe things via the Serial Monitor), I find that uplink messages are NOT being queued when I use the M138 as it will eventually be deployed -- running on battery alone with NO computer connected.

Using code based on the example Example14_transmitText.ino, I have commented out the following:

//  while (!Serial)
//    ; // Wait for the user to open the Serial console

... thinking that everything else will function normally. But, after waiting 24 hours or so for uplink messages to appear, there are none. Upon reconnecting the USB/Serial connection to the computer, the Serial Monitor displays the following:

15:06:31.681 -> Swarm Satellite example
15:06:31.681 -> 
15:06:31.780 -> getDateTime returned: 2022/6/8 15:6:31
15:06:31.780 -> 
15:06:31.780 -> 
15:06:31.780 -> sensorValue = : 556
15:06:31.780 -> Volts-IN = : 8.97
15:06:31.813 -> 
15:06:31.813 -> 
15:06:32.409 -> There are 0 unsent messages in the TX queue
15:06:32.409 -> 
15:06:32.409 -> getDateTime returned: 2022/6/8 15:6:31
15:06:32.409 -> 
15:06:32.409 -> 
15:06:32.409 -> TX Payload: Voltage:  8.97 @ 2022/6/8 15:6
15:06:32.409 -> 
15:06:44.789 -> sensorValue = : 562
15:06:44.789 -> Volts-IN = : 9.06
15:06:44.789 -> 
15:06:44.789 -> 
15:06:45.384 -> There is 1 unsent message in the TX queue
15:06:45.384 -> 
15:06:55.413 -> sensorValue = : 562
15:06:55.413 -> Volts-IN = : 9.06
15:06:55.413 -> 
15:06:55.413 -> 
15:06:56.008 -> There is 1 unsent message in the TX queue

... so, clearly no messages are being queued for uplink until there is a USB/Serial connection present.

I have looked & looked but am unable to identify why no messages queue absent a computer USB/Serial connection. Could I please trouble you for guidance? I have to think that others might run into this as well since the whole purpose of the SWARM system is to allow for stand-alone communications from areas otherwise not connected to any infrastructure.

Thanks very much!

-Scott, K4KDR

[PaulZC]

Hi Scott (@K4KDR ),

I don't know how the Due behaves if your code is attempting to use "Serial" but the USB is not actually connected. The code may be stalling at the first Serial.print?

My advice would be to delete all mention of Serial from the example and either flash out some Morse code diagnostics via the LED_BUILTIN or select a different UART (Serial2 perhaps?) and use that for your 'console' / debug messages. We have a number of Serial Breakout boards which you could use to connect Serial2 back to a terminal emulator on your PC to help with debugging.

I suspect there's nothing I can change in this library to help with this, so please close this issue as soon as you're ~happy.

Best wishes, Paul

[K4KDR]

Thanks very much for the reply & link - I'll certainly check those items out!

Sorry; I should have been clear that I wasn't requesting any changes, but rather was shamelessly hunting for pointers since you're obviously more familiar with the communications to/from the M138 & Arduino.

Excellent idea to sideline all references to the USB/Serial link to the computer. That will be a great way to narrow my search.

Thanks again!

[PaulZC]

Hi Scott,

"shamelessly hunting for pointers" is how we all get started - with Arduino especially! ;-)

I absolutely don't mind helping out but: my time is finite; and we need to make sure an audience can benefit from our conversations. In this case, it might have been better to ask in the SparkX Forum. It might attract a bigger audience and/or you may get good suggestions from others.

I don't have a Due to test but, looking at the schematic, the main USB port D+ and D- signals go straight to the ATSAMD3X8 processor. The hardware 'UART' for USB "Serial" is built-in. So the processor itself will know if the USB cable is disconnected. That may cause calls to Serial to misbehave or stall when you're not connected.

You can test this with something like:

setup()
{
  Serial.begin(115200);
  pinMode(LED_BUILTIN, OUTPUT);
}

loop()
{
  Serial.println(F("Hello")); // Print something on Serial
  digitalWrite(LED_BUILTIN, !digitalRead(LED_BUILTIN)); // Blink the LED (the ! inverts the digitalRead)
  delay(1000); // Once per second
}

I haven't tested this but it should send a "Hello" and toggle the LED each second. If the LED stops blinking when you disconnect the Serial USB, you have your answer. Please give it a try and post the answer here - then others can find it too.

Best wishes, Paul

[K4KDR]

You're very kind to offer so much assistance - much appreciated!

After trying endless coding alternatives related to serial comms throughout this entire evening, I finally identified the root cause. Hopefully if anyone finds this discussion it might help them.

In short, there is one very important difference between working with devices via USB/Serial & powering them stand-alone on with battery power. When using USB/Serial to a computer, the devices have normally been powered on for a good while and are fully booted up. When powering them on cold with a battery, both devices are booting from scratch.

Background: in the stand-alone (battery) configuration, my only indication of normal operation was the status of the LED_BUILTIN... I coded it to illuminate when the TX Queue contained a message.

  // show status of uplink TX message queue
  uint16_t count;
  mySwarm.getUnsentMessageCount(&count); // Get the number of untransmitted messages
    if (count == 0)                      // LED off if no messages in TX queue
    {
      txQueueEmpty = true;
      digitalWrite(LED_BUILTIN, LOW);   // turn the LED off

    }
  if (count != 0)                        // blink & leave LED on if there ARE messages in TX queue
    {
      txQueueEmpty = false;
      digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on
      delay(100); 
      digitalWrite(LED_BUILTIN, LOW);    // turn the LED off
      delay(100);
      digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on 
      delay(100);
      digitalWrite(LED_BUILTIN, LOW);    // turn the LED off
      delay(100);
      digitalWrite(LED_BUILTIN, HIGH);   // turn the LED on 

Where that became my downfall was, of course, later in my code. I don't want to uplink 'old' data, so before a message is submitted to the TX queue on the M138, I empty the queue. However, when the M138 is first powered on, there is not a valid GPS fix and uplink messages are not accepted!

So, when powered on cold, the TX queue was immediately emptied but because the M138 was not accepting messages yet, the LED_BUILTIN remained dark. I interpreted that to mean 'nothing' was working. (In contrast, when using the USB/Serial connection to a PC, the M138 has typically been powered on long enough to have a valid GPS fix and uplink messages are accepted immediately!). AND, with a 1-hour delay between the submission of uplink messages, it would be a LONG time before there would even be an ATTEMPT to submit another uplink message.

My solution, then was to BYPASS the 1-hour check 'if' the TX queue was empty. In short, the code would attempt to submit an uplink message on EVERY LOOP until the M138 had a GPS fix & accepted it. Only then would subsequent loops abide by the 1-hour delay:

static unsigned long lastTransmit;

//  skip 1-hour TX check if this is first run through loop OR if TX Queue is empty
  if (First_Loop || txQueueEmpty) {
      lastTransmit = -3600000;
  }

if (millis() > (lastTransmit + 3600000))    // Transmit once per hour
{
  lastTransmit = millis(); // Update lastTransmit

... of course I'm not finished - since each legitimate message uplink ALSO empties the queue, I need to accommodate that and keep my 1-hour spread in effect. But I'm close and wanted to document my findings as well as thank you for the support.

Best regards,

Scott, K4KDR

[PaulZC]

Hi Scott,

Aha! Nice detective work! Nothing to do with Due Serial (USB) then...

Some comments / suggestions:

@adamgarbo noticed that $DT @ date time requests time out while the modem is getting time from GPS (see the bottom of #22 ). It seems likely that the transmitText is also timing out - or returning an error - while the modem sorts itself out. You could check the return value from transmitText: Swarm_M138_Error_e err = mySwarm.transmitText((const char *)myMessage, &id); If err is SWARM_M138_ERROR_SUCCESS then you know the message was accepted and added to the TX queue. SWARM_M138_ERROR_TIMEOUT or SWARM_M138_ERROR_ERR will indicate the modem isn't yet ready to accept the message.

You could set up a callback to check for the $M138 BOOT,RUNNING and/or $M138 DATETIME messages. setModemStatusCallback sets up the callback. status == SWARM_M138_MODEM_STATUS_BOOT_RUNNING indicates the boot is complete. status == SWARM_M138_MODEM_STATUS_DATETIME indicates the modem has GPS time and will be ready to accept TX messages. Example10 is the best place to start.

Have fun! Paul

[K4KDR]

That's so helpful - thanks! I don't fully understand what a 'callback' is, but the internet can come to my rescue for that. It will be extremely useful to be able to take action based on the feedback that an uplink message was accepted into the queue. Appreciate that very much!!

[adamgarbo]

Hey Scott (@K4KDR),

Great to see your interest in the Swarm modem. Given that we're both tinkering with the modem, I was thinking we should connect over email! Do you have an address I could reach you at?

Cheers, Adam

[K4KDR]

Absolutely, Adam. I'm at:

--- email deleted ---

[K4KDR]

While I'm sure any real programmer will react in horror, I wanted to close the loop on this by showing how I was able to use the Check if the message was queued successfully testing from Example-14 (THANK YOU!!!) to set a flag txMsgSubmitted = true;so that my loop would not submit any additional messages for at least 1 hour:

static unsigned long lastTransmit;

//  skip 1-hour TX check if this is first run through loop OR if TX Queue is empty
//    but NOT if an uplink message has been queued successfully in the past hour

  if ((First_Loop || txQueueEmpty) && !txMsgSubmitted) {
      lastTransmit = -3600000;
  }

if (millis() > (lastTransmit + 3600000))    // Transmit once per hour
{
  lastTransmit = millis(); // Update lastTransmit
  uint64_t id;
  char myMessage[192];
  char myVolts[5];
  dtostrf(realVolts, 5, 2, myVolts);
  getTIME();

  strcpy(myMessage, "Voltage: ");
  strcat(myMessage, myVolts);
  strcat(myMessage, " @ ");
  strcat(myMessage, myTimeStamp);

  // Send sensor value as TEXT

  Serial.print(F("TX Payload: "));
  Serial.println(myMessage);
  Serial.print('\n');

  mySwarm.deleteAllTxMessages();

  delay(1000);

  Swarm_M138_Error_e err = mySwarm.transmitText(myMessage, &id);

 // Check if the message was queued successfully
  if (err == SWARM_M138_SUCCESS)
  {
    Serial.print(F("The message has been added to the transmit queue. The message ID is "));
    serialPrintUint64_t(id);
    Serial.println();
    Serial.println();
    txMsgSubmitted = true;
  }
  else
  {

... and then thanks to Node Red leveraging the API abilities that SWARM lists on the dashbord, the resulting TEXT message can be displayed on my local Grafana Dashboard in its entirety (along with timestamps for queue submission + the time received @ SWARM's server). I'm also using Node Red to parse out the actual voltage value & convert it to a number so that it can it can be displayed & graphed separately to see trending. (Yes, I could of course transmit that value as a number natively, but I'm not concerned about byte count, etc., as I would be using public LoRaWAN facilities such as 'The Things Stack', so TEXT is fine for now). The result:

[PaulZC]

Very nice Scott - thank you!

Have a great weekend, Paul