Missing bytes in the socket

[renanqts]

Hello,

I'm facing missing bytes in the socket created by the library

Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_gpio_aux: transition from IDLE to RX state
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_input_disable
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_uart: received 8 bytes for a total of 8 bytes from uart
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_uart: received 8 bytes for a total of 16 bytes from uart
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_gpio_aux: transition from RX to IDLE state
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_gpio_aux: received 0 bytes for a total of 16 bytes from uart
Nov 19 09:52:50 raspberrypi e32[745]: e32_write_output: sending 16 bytes to socket /home/pi/e32.rx.data
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_input_enable
Nov 19 09:52:50 raspberrypi e32[745]: e32_poll_uart: received 1 bytes for a total of 17 bytes from uart

As we can see above in the logs, 17 bytes were received but only 16 were sent to the socket.

[renanqts]

I was debugging it, this is an issue introduced in https://github.com/lloydroc/ebyte-sx1276/pull/13/commits/37bf0ef8a01ee3c293a9cdbf482e20f8a01f69b8 as the line removed in this commit says, it requires to sleep a bit

    /* we need to sleep and read from the uart again as remaining
      * bytes are not ready until AFTER the AUX pin transitions from
      * low to high. If we don't do this we will leave bytes in the
      * buffer
      */
    usleep(54000);

[lloydroc]

@renanqts sorry to hear you're having this issue. I've seen an issue similar to this on one ONLY on my Raspberry Pi B+ and not on a Pi Zero or Pi 4. The problem as far as I remember is it should be buffering 1 byte and not 8 bytes at a time. So it leaves the remainder modulus 8 out. I will try to confirm it but I was only able to reproduce it on a B+. Give me a couple days and I'll see if I can reproduce it. Just to confirm adding that sleep back has resolved your issue?

[renanqts]

Hey @lloydroc

Yes, it solved the issue including the lines back. I'm Pi 3 B+, probably that is why. Sure, take your time. No rush here :D

[lloydroc]

For example here are the logs in my Pi 4, it will not buffer the 8 bytes and you can see it is 1 byte. That delay is not present. Not sure why that delay would fix it.

pi@raspberrypi:/var/log $ journalctl -u e32
-- Logs begin at Tue 2021-08-17 21:30:19 MDT, end at Fri 2021-11-19 17:35:52 MST. --
Aug 17 21:30:19 raspberrypi e32[19855]: e32_poll_gpio_aux: transition from RX to IDLE state
Aug 17 21:30:19 raspberrypi e32[19855]: e32_poll_gpio_aux: received 0 bytes for a total of 10 bytes from uart
Aug 17 21:30:19 raspberrypi e32[19855]: e32_poll_input_enable
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_gpio_aux: transition from IDLE to RX state
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_input_disable
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 1 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 2 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 3 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 4 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 5 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 6 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 7 bytes from uart
Aug 17 21:30:29 raspberrypi e32[19855]: e32_poll_uart: received 1 bytes for a total of 8 bytes from uart

[renanqts]

As far as I understood, the data is not there when the line is executed, meaning that the delay waits a bit for it. I just didn't get where these 8 bytes come from. Is this internally in the library? At least I didn't see it there.

[lloydroc]

Good find! When I send 18 bytes I actually do see 8+8+2 coming in. Before I thought the 2 additional bytes wouldn't come in until there were 6 more to make it 8.

Nov 20 07:57:27 pi1 e32[20667]: e32_poll_gpio_aux: transition from IDLE to RX state
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_input_disable
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_uart: received 8 bytes for a total of 8 bytes from uart
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_uart: received 8 bytes for a total of 16 bytes from uart
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_gpio_aux: transition from RX to IDLE state
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_gpio_aux: received 0 bytes for a total of 16 bytes from uart
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_input_enable
Nov 20 07:57:27 pi1 e32[20667]: e32_poll_uart: received 2 bytes for a total of 18 bytes from uart

Will need to add a fix for this. It only seems to happen this way in the B+. Not sure yet if the sleep should be added always (for all Raspberry Pi Models) or just when we are buffering >1 bytes. I really don't like adding in unnecessary sleeps when we don't have to.

[renanqts]

I would just add it when finding multiples of 8

[renanqts]

@lloydroc should I send this PR?

[lloydroc]

Is it a PR that just adds the delay or adds the delay when we read > 1 byte at a time?

On Tue, Nov 23, 2021, 2:04 PM Renan Rodrigues @.***> wrote:

should I send this PR?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lloydroc/ebyte-sx1276/issues/15#issuecomment-977167717, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASHO6UJIEWZKZYDNHXBGLTUNP6XBANCNFSM5ILXWAUA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[renanqts]

yes, I thought this should the one

[lloydroc]

Appears we can pass the struct options into the e32_poll_uart function. If we read more than 1 byte at a time we can set a new option that we can conditionally add that delay.

On Wed, Nov 24, 2021, 1:23 AM Renan Rodrigues @.***> wrote:

yes, I thought this should the one

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lloydroc/ebyte-sx1276/issues/15#issuecomment-977640004, or unsubscribe https://github.com/notifications/unsubscribe-auth/AASHO6WYYONBKBQEVXRSSBDUNSOJFANCNFSM5ILXWAUA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[lloydroc]

I created a branch and will send a PR shortly need to do some testing with the RPI models I have.

[lloydroc]

https://github.com/lloydroc/ebyte-sx1276/pull/16

[lloydroc]

The PR seems to be working for me. Here is an example of the worst case scenario:

Nov 24 15:23:42 pi1 e32[27060]: e32_poll_gpio_aux: transition from IDLE to RX state
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_input_disable
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_uart: received 8 bytes for a total of 8 bytes from uart
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_uart: received 8 bytes for a total of 16 bytes from uart
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_uart: received 8 bytes for a total of 24 bytes from uart
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_gpio_aux: transition from RX to IDLE state
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_gpio_aux: additional sleep for uart buffered data
Nov 24 15:23:42 pi1 e32[27060]: e32_poll_gpio_aux: received 7 bytes for a total of 31 bytes from uart

[renanqts]

I will give a try

[lloydroc]

PR 16 above addresses the issue.