Fragmentation of received data

[OlexKov]

The data is not received entirely, but in parts. The transmitted 100 bytes are received not at one time, but three times, 32 bytes each and once 4 bytes. This code:

reader.stream.listen((data) { String str = utf8.decode(data); print('received: $data'); print(data.length);}

...when transferring 100 bytes, it gives the following result :

received: [49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52] length: 32 received: [52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55] length: 32 received: [55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 48, 48, 48, 48, 48, 48] length: 32 received: [48, 48, 48, 48] length: 4

What is the reason for this behavior and is it possible to make the data received at one time?

[OlexKov]

Nobody knows the solution to the problem or is my question just being ignored?

[jpnurmi]

The purpose of SerialPortReader is to provide an asynchronous stream of data, which is emitted in chunks whenever there's data available. What it does under the hood, is to run a read/wait loop in a separate isolate to avoid blocking the current isolate. This makes it possible to read data in a Flutter UI service class without blocking the UI.

You can use SerialPort.read() with a timeout if you want to block until a certain amount of bytes have been received. Just be aware since it's a blocking call, there's no way to execute other code in the same isolate meanwhile, nor cancel the call until the requested bytes have been received or the specified timeout is hit. This may be fine in a command-line application, but you wouldn't want to do that from the main isolate of a Flutter UI app. In order to avoid blocking the UI, you'd have to wait for and read the data in a separate isolate, which is what SerialPortReader does for you.

[OlexKov]

Thank you very much for your reply. I cannot use SerialPort.read () because this function requires a value for the amount of data and I don’t know their amount because it is different every time ... As far as I understand, the only way out for me is to add the start and end point of the data to the data being transmitted and buffer them when receiving. Or is it still possible to increase the chunk size, which, as I understand it, is limited to 32 bytes?

[elgansayer]

I actually found another plugin that works flawless. I use a similar method to stream the data and it misses nothing.

Yes the data is chunked but it means creating a second loop collecting the chunks and a new stream. Meaning the listener from the plugin is kinda useless because you then need to duplicate this and yet sometimes the data seems malformed.

Something somewhere somehow is returning some bad data.

In a few days I can show barebones examples

On Sat, 23 Oct 2021, 20:45 Sashok9203, @.***> wrote:

Thank you very much for your reply. I cannot use SerialPort.read () because this function requires a value for the amount of data and I don’t know their amount because it is different every time ... As far as I understand, the only way out for me is to add the start and end point of the data to the data being transmitted and buffer them when receiving. Or is it still possible to increase the chunk size, which, as I understand it, is limited to 32 bytes?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jpnurmi/libserialport.dart/issues/45#issuecomment-950203361, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPNVNGRP47IIWREPTJ3A33UIMGFZANCNFSM5GRGJ25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[OlexKov]

I will be very grateful to you if you post an example of your code.

[OlexKov]

I actually found another plugin that works flawless. I use a similar method to stream the data and it misses nothing. Yes the data is chunked but it means creating a second loop collecting the chunks and a new stream. Meaning the listener from the plugin is kinda useless because you then need to duplicate this and yet sometimes the data seems malformed. Something somewhere somehow is returning some bad data. In a few days I can show barebones examples … On Sat, 23 Oct 2021, 20:45 Sashok9203, @.***> wrote: Thank you very much for your reply. I cannot use SerialPort.read () because this function requires a value for the amount of data and I don’t know their amount because it is different every time ... As far as I understand, the only way out for me is to add the start and end point of the data to the data being transmitted and buffer them when receiving. Or is it still possible to increase the chunk size, which, as I understand it, is limited to 32 bytes? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#45 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPNVNGRP47IIWREPTJ3A33UIMGFZANCNFSM5GRGJ25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Did not you forget? Wee said that you will post the code.

[elgansayer]

Errm I'm busy moving house. I don't have my office right now. Don't be a dick.

On Tue, 26 Oct 2021, 17:57 Sashok9203, @.***> wrote:

I actually found another plugin that works flawless. I use a similar method to stream the data and it misses nothing. Yes the data is chunked but it means creating a second loop collecting the chunks and a new stream. Meaning the listener from the plugin is kinda useless because you then need to duplicate this and yet sometimes the data seems malformed. Something somewhere somehow is returning some bad data. In a few days I can show barebones examples … <#m3678139501564279519> On Sat, 23 Oct 2021, 20:45 Sashok9203, @.***> wrote: Thank you very much for your reply. I cannot use SerialPort.read () because this function requires a value for the amount of data and I don’t know their amount because it is different every time ... As far as I understand, the only way out for me is to add the start and end point of the data to the data being transmitted and buffer them when receiving. Or is it still possible to increase the chunk size, which, as I understand it, is limited to 32 bytes? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#45 (comment) https://github.com/jpnurmi/libserialport.dart/issues/45#issuecomment-950203361>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPNVNGRP47IIWREPTJ3A33UIMGFZANCNFSM5GRGJ25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

Did not you forget? Wee said that you will post the code.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jpnurmi/libserialport.dart/issues/45#issuecomment-952131413, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPNVNEQJEHFXPGCYUSB2QDUI3TXJANCNFSM5GRGJ25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[OlexKov]

To be rude is not obligatory, I just reminded.

[elgansayer]

I see that you're not English and the phrasing reads harshly and it's just a culture clash. So never mind. I'll get the code when I can. It's frustrating waiting for code.

I still don't have an office. I will post as soon as I can.

For the other plugin the code is on my GitHub but for this plugin I need to remake the sample.

I think this plugin would work much better with w non blocking read and w stream that streams the raw data and does not do any loops. It would make the user have to handle the data with their own loops but the user has to do that anyway.

Using my mobile to type this out and I'm a lazy writer. Sorry if it makes no sense 🤣

On Wed, 27 Oct 2021, 10:31 Sashok9203, @.***> wrote:

To be rude is not obligatory, I just reminded.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jpnurmi/libserialport.dart/issues/45#issuecomment-952726071, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPNVNABLDVMZ6JVTASCMKTUI7IJ5ANCNFSM5GRGJ25Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[OlexKov]

Promst me, I use Google translate. If I was rude then I didn’t want it all Google. I hope this phrase he did not make offensive.:) Sorry one more time

[jiangweihu]

Hello Did you post an example, I didn't see it on you github

[jiangweihu]

The data is not received entirely, but in parts. The transmitted 100 bytes are received not at one time, but three times, 32 bytes each and once 4 bytes. This code:

reader.stream.listen((data) { String str = utf8.decode(data); print('received: $data'); print(data.length);}

...when transferring 100 bytes, it gives the following result :

received: [49, 49, 49, 49, 49, 49, 49, 49, 49, 49, 50, 50, 50, 50, 50, 50, 50, 50, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51, 51, 51, 52, 52] length: 32 received: [52, 52, 52, 52, 52, 52, 52, 52, 53, 53, 53, 53, 53, 53, 53, 53, 53, 53, 54, 54, 54, 54, 54, 54, 54, 54, 54, 54, 55, 55, 55, 55] length: 32 received: [55, 55, 55, 55, 55, 55, 56, 56, 56, 56, 56, 56, 56, 56, 56, 56, 57, 57, 57, 57, 57, 57, 57, 57, 57, 57, 48, 48, 48, 48, 48, 48] length: 32 received: [48, 48, 48, 48] length: 4

What is the reason for this behavior and is it possible to make the data received at one time?

Hello Did you solve this problem please?

[OlexKov]

Hello Did you solve this problem please?

No. I think it will not work as I need and that the bi program did not freeze when receiving / transmitting data

[nilotpalkapri]

So, I found a workaround by adding a delimitator at the end of the data sending from arduino or other device, and in the application concatenating data till the delimitator received & splitting using the same.