New Rx interrupt Buffer causes wdt with network usage

[sticilface]

This bug has taken my a long time to track down. One of my usages of the ESP is to take data streamed over serial and pipe it to WS2812 LEDs. This is my adalight implementation, and one of the first things that I wrote. It has worked fine, at baud rates up to 2,000,000 for over a year, until the introduction of https://github.com/esp8266/Arduino/commit/f8a8a2a3595751c39ec8604ab21fdfbd7daccad6.

My main sketch is very complex with asyncwebserver, asyncmqtt, some UDP libs of my own, neopixelbus... but i have now excluded all of these things.

To reproduce the bug, use the sketch below with the python script. This sketch contains the shell of my ada light sketch but only containing the serial receive components and the wifi/OTA stuff. use master, or any commit after https://github.com/esp8266/Arduino/commit/f8a8a2a3595751c39ec8604ab21fdfbd7daccad6. configure sketch to join wifi, then run the python sketch making sure to configure the right serial port. you should see serial output from the esp... Ada..Ada... now you know that the ESP will be receiving the serial data... Now perform an OTA... you should get a wdt fairly quickly.. Revert back to stable 2.3.0. and the OTA will work fine during serial data being streamed.

For me this error is very sporadic (unless using this example), and probably to do with interrupts and wifi, something similar to the neopixelbus and biting methods. Performing and OTA gives enough network traffic to generate the wdt. I stress that this occurs with normal usage in a sketch that has been working for a long time without problems..

sketch

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <ArduinoOTA.h>

const char* ssid = "xx";
const char* password = "xx";
const char * hostName = "esp-async";

#define pixelCount 64 

const int SerialSpeed = 115200; 

   uint8_t prefix[] = {'A', 'd', 'a'}, hi, lo, chk, i; 
   uint16_t effectbuf_position = 0;
   enum mode { MODE_INITIALISE = 0, MODE_HEADER, MODE_CHECKSUM, MODE_DATA, MODE_SHOW, MODE_FINISH};
   mode state = MODE_INITIALISE;
   int effect_timeout = 0;
   uint8_t prefixcount = 0;
   unsigned long ada_sent = 0; 
   unsigned long pixellatchtime = 0; 
   const unsigned long serialTimeout = 15000; 
   long update_strip_time = 0; 

void  Adalight () {   

  switch (state) {

    case MODE_INITIALISE:
      Serial.println("Begining of Adalight");
      state = MODE_HEADER;

    case MODE_HEADER:

      effectbuf_position = 0; // reset the buffer position for DATA collection...

          if(Serial.available()) { // if there is serial available... process it... could be 1  could be 100....

            for (int i = 0; i < Serial.available(); i++) {  // go through every character in serial buffer looking for prefix...

              if (Serial.read() == prefix[prefixcount]) { // if character is found... then look for next...
                  prefixcount++;
              } else prefixcount = 0;  //  otherwise reset....  ////

            if (prefixcount == 3) {
              effect_timeout = millis(); // generates START TIME.....
              state = MODE_CHECKSUM; // Move on to next state
              prefixcount = 0; // keeps track of prefixes found... might not be the best way to do this. 
              break; 
            } // end of if prefix == 3

            } // end of for loop going through serial....
            } else if (!Serial.available() && (ada_sent + 5000) < millis()) {
                  Serial.print("Ada\n"); // Send "Magic Word" string to host every 5 seconds... 
                  ada_sent = millis(); 
            }

    break;

    case MODE_CHECKSUM:

        if (Serial.available() >= 3) {
          hi  = Serial.read();
          lo  = Serial.read();
          chk = Serial.read();
          if(chk == (hi ^ lo ^ 0x55)) {
            state = MODE_DATA;
          } else {
            state = MODE_HEADER; // ELSE RESET.......
          }
        }

      if ((effect_timeout + 1000) < millis()) state = MODE_HEADER; // RESET IF BUFFER NOT FILLED WITHIN 1 SEC.

      break;

    case MODE_DATA:

        while (Serial.available() && effectbuf_position < 3 * pixelCount) {   
          //pixelsPOINT[effectbuf_position++] = Serial.read();  
          Serial.read();
        }

      if (effectbuf_position >= 3*pixelCount) { // goto show when buffer has recieved enough data...
        state = MODE_SHOW;
        break;
      } 

        if ((effect_timeout + 1000) < millis()) state = MODE_HEADER; // RESUM HEADER SEARCH IF BUFFER NOT FILLED WITHIN 1 SEC.

      break;

    case MODE_SHOW:

      //strip.Dirty(); // MUST USE if you're using the direct buffer version... 
      pixellatchtime = millis();
      state = MODE_HEADER;
      break;

    case MODE_FINISH:

    // nothing in here...

    break; 
    }

}

void setup()
{
    Serial.begin(SerialSpeed); 

  Serial.setDebugOutput(true);
  WiFi.hostname(hostName);
  WiFi.mode(WIFI_AP_STA);
  WiFi.begin(ssid, password);
  if (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.printf("STA: Failed!\n");
    WiFi.disconnect(false);
    delay(1000);
    WiFi.begin(ssid, password);
  }

  //Send OTA events to the browser
  ArduinoOTA.onStart([]() { Serial.print("Update Start"); });
  ArduinoOTA.onEnd([]() { Serial.print("Update End"); });

  ArduinoOTA.setHostname(hostName);
  ArduinoOTA.begin();

}

void loop()
{

  Adalight();
  ArduinoOTA.handle();

}

python script

from __future__ import print_function # needs to be first statement in file

import serial
import sys
import time
import random

print('running')

# configure the serial connections (the parameters differs on the device you are connecting to)
ser = serial.Serial("/dev/cu.usbserial")
ser.baudrate=115200
# ser.timeout = 0 
ser.writeTimeout = 0

# this is the Ada and CRC to get it working... 
initstring = bytearray.fromhex("41 64 61 00 79 2C")

try:
    while True:
        if ser.out_waiting == 0:    
            ser.write(initstring)
            for x in xrange(0, 64 * 3):
                val = bytearray([random.randint(1, 60)])
                ser.write(val)
            time.sleep(0.05)  # 20Hrz rate... same as hyperion
            bytesToRead = ser.inWaiting()
            if bytesToRead > 0:
                data = ser.read(bytesToRead)
                print (data,end='')
        else:
            print ("Buffer RESET")
            ser.reset_output_buffer()

except KeyboardInterrupt:
    print('interrupted!')

I'd like to suggest that the RX buffer be made optional

[sticilface]

I've found that if i completely disable the RX buffer then things are a lot more stable. Seeing as I do not need to RX in this sketch, my short term solution is to enable serial like this when I'm not using my Adalight implementation.

Serial.begin(115200, SERIAL_8N1, SERIAL_TX_ONLY, 1); 

I'm suspecting that as my computer sends data constantly is causes problems when the UART buffer is not cleared by calling available() or flush().

[d-a-v]

OTA in IDE has received a small fix, could you try again without your TX-only fix ?

[devyte]

Closing via #4328 .