await data from SerialPort

[JacTech]

Im fairly new to dart and Flutter so maybe this has already be implemented and i just cand find it/get it to work. Basically this is the only way i was able to receive data from my ESP8266:

String data;
 port.readBytesOnListen(16, (value) {
 data = String.fromCharCodes(value);
 print(data);
});

and as far as i understand it i can't "await" this in an async function, which i would have to do cause im waiting for a feedback from the device. Other ways of reading data like print(await port.readBytesUntil(Uint8List.fromList("\n".codeUnits))); didnt work for me i just wouldnt get any output in the console.

[FengChendian]

What is your data content? port.readBytesUntil(Uint8List.fromList("\n".codeUnits)) just return data until '\n' character. If readBytesOnListen works fine, readBytesUntil should work as long as you set true terminator.

PS: If you use ESP UART or printf, data may not end with '\n'. Even when you use c++ string, str does not end with '\0'.

By the way, in the previous version, there is read(n) function to directly read n bytes from buffer. But it will cause UI freeze if you set wrong bytes size (await will block dart thread until bytes fitted size or timeout). And it raises many I/O issues. So I use listen function readBytesOnListen and async/await readBytesUntil instead.

[JacTech]

I had the ide to use Serial.println() in the ESP as this should definitly include a "\n". Doing this and using print(port.readBytesUntil(Uint8List.fromList("\n".codeUnits))); to read the data i get flutter: Instance of 'Future<Uint8List>' in the console every time the ESP sends something. However adding an await before the port.readBytesUntil gives me no output at all.

could you maybe show me how you would read serial data from an ESP (with async/await)

[FengChendian]

I had the ide to use Serial.println() in the ESP as this should definitly include a "\n". Doing this and using print(port.readBytesUntil(Uint8List.fromList("\n".codeUnits))); to read the data i get flutter: Instance of 'Future<Uint8List>' in the console every time the ESP sends something. However adding an await before the port.readBytesUntil gives me no output at all.

could you maybe show me how you would read serial data from an ESP (with async/await)

I used await port.readBytesUntil(Uint8List.fromList("\n".codeUnits)) in example app folder to receive data. It's same as your code. But I don't use println function of esp-idf. Please tell me your ESP-IDF version. I will test it.

However, I guess your data may be

\n
message\r\n # println()

So print is empty.

Do you try code like await port.readBytesUntil(Uint8List.fromList("other alphabets".codeUnits))? (for example, your data is hello word, you read until 'd')

Or do you have serial debugging assistant like putty? It's helpful to know all ascii codes sent from your device.

[JacTech]

so after a lot of tinkering i finally got it working, its a bit sketchy tho... This is the code running on the ESP:

if (Serial.available())
  {
    incomingString = Serial.readString();
  }
  if (incomingString == "device_name")
  {
    Serial.println("test");
    for (int i = 0; i < 5; i++) //continously write to Serial port, so flutter picks up "test"
    {
      Serial.println("___");
      delay(200);
    }
    incomingString = "___";     //Clear incomingString and wait for a nother "device_name"
  }

and this is the flutter code:

  void establishConnection() async {
    if (!port.isOpened) {
      port.openWithSettings(BaudRate: 115200);
    }
    while (!port.isOpened) {}

    print(port.writeBytesFromString("device_name"));

    Uint8List rawInput = await port.readBytesUntil(Uint8List.fromList('\n'.codeUnits));
    List<int> cutASCIIInput = rawInput.sublist(0, rawInput.length - 2); // Ignore last 2 elements
    String serialInput = String.fromCharCodes(cutASCIIInput);
    print(serialInput);
  }

with this im able to receive the test message from the ESP. wierdly when i remove the for loop, that just spamms some text 5 times im not able to receive/print the test message in flutter. I also tried just adding a various delays between receiving device_name and sending test but to no effect. Also just sending some text once after test doesnt work, it has to be sent multiple times. There also is a big (2-4sec) delay between sending and receiving in Flutter. For me this is fine but a nicer way would be cool :) Thanks for the quick help anyways

[FengChendian]

I test your code on ESP32-S3-WROOM-1 with arduino. This bug may be due to arduino Serial timeout. I found that if you send data to ESP32, it will not response before many data sent. Maybe you need set proper timeout.

some docs

Serial.readString() reads characters from the serial buffer into a String. The function terminates if it times out (see setTimeout()).

Serial.setTimeout() sets the maximum milliseconds to wait for serial data. It defaults to 1000 milliseconds.

And you code delays 1000 ms. Just like a coincidence ?

  for (int i = 0; i < 5; i++) //continously write to Serial port, so flutter picks up "test"
  {
    Serial.println("___");
    delay(200);
  }

images

@JacTech Please check my image for debugging this. I send 1 to device. Device only response a few times. Because timeout is 1s and string is from all data in 1s.

[JacTech]

Ok but receiving the data on the ESP isnt the problem. Receiving at least some of the data in Flutter is the problem. I also tested the same sketch as yours and when giving enough time (1000ms) the ESP responds correctly every time. I also added some code, so the onboard LED would blink up when receiving a "1" and tested the whole thing in flutter again. Sending a one => LED blinks => nothing received in Flutter. When adding the for loop (which indeed is coincidentaly 1000ms) and i receive data in flutter again. I receive the "www" but when writing "1" again i receive whatever was in the for loop 5 times before receiving "www" again. This would make sense cause its still in the pipe, but i dont know why "www" alone wont be read. Fortunally this can be solved by closing the port after receiving "www" so i am fine with this, but if you would like i could send you the full project and arduino code, to try around some more

[FengChendian]

Ok but receiving the data on the ESP isnt the problem. Receiving at least some of the data in Flutter is the problem. I also tested the same sketch as yours and when giving enough time (1000ms) the ESP responds correctly every time. I also added some code, so the onboard LED would blink up when receiving a "1" and tested the whole thing in flutter again. Sending a one => LED blinks => nothing received in Flutter. When adding the for loop (which indeed is coincidentaly 1000ms) and i receive data in flutter again. I receive the "www" but when writing "1" again i receive whatever was in the for loop 5 times before receiving "www" again. This would make sense cause its still in the pipe, but i dont know why "www" alone wont be read. Fortunally this can be solved by closing the port after receiving "www" so i am fine with this, but if you would like i could send you the full project and arduino code, to try around some more

Thanks for your response and tests. Please send me project and arduino code. I will dig in this problem.

By the way, I also found serial uses async transmission in serial write doc. Maybe it's helpful for this issue (add Serial.flush() in the code).

Serial transmission is asynchronous. If there is enough empty space in the transmit buffer, Serial.write() will return before any characters are transmitted over serial. If the transmit buffer is full then Serial.write() will block until there is enough space in the buffer. To avoid blocking calls to Serial.write(), you can first check the amount of free space in the transmit buffer using availableForWrite().

[JacTech]

Hey. Here is the link to my Flutter project, be warned its a big mess (my first flutter project) :) https://drive.google.com/file/d/1_4wbjMS6vqk7y9qmNlSSj48vqG9v_GN3/view?usp=drive_link I will have to permision you to see the file but ill try to do it as soon as i get the request. The Serial communication part is in page_new_device.dart line 33. Ill also try out the Serial.flush() stuff you recomended, as soon as ive got time. Thanks a lot for the efforts.

[FengChendian]

Hey. Here is the link to my Flutter project, be warned its a big mess (my first flutter project) :) https://drive.google.com/file/d/1_4wbjMS6vqk7y9qmNlSSj48vqG9v_GN3/view?usp=drive_link I will have to permision you to see the file but ill try to do it as soon as i get the request. The Serial communication part is in page_new_device.dart line 33. Ill also try out the Serial.flush() stuff you recomended, as soon as ive got time. Thanks a lot for the efforts.

I have requested access permission. And I will test it. Thanks for your help!

[FengChendian]

@JacTech I think it's due to long ERROR_IO_PENDING timeout on ESP8266. In version 2.1.7, I use ClearCommError like pyserial to get bytesSize immediately without waitForSingleObject. Maybe the bug is fixed. I'm not sure. Please try it.

• And I have removed old readOnListen and give 3 new API for more precise control.

print(await port.readBytesUntil(Uint8List.fromList("T".codeUnits))); /// '\0' is not included
/// or
var read = port.readBytes(18, timeout: Duration(milliseconds: 10))..then((onValue) => print(onValue));
var result = await read;
print(result);
/// or
var fixedBytesRead = port.readFixedSizeBytes(2)..then((onValue) => print(onValue));
await fixedBytesRead;
/// see more in small example