Refresh lags after a few seconds

[MatthewMWR]

Moving discussion from Play store to GitHub...

When the app first starts getting data from the Arduino everything is fine for a few seconds, but then the app becomes substantially laggy in refreshing screen contents.

For example, when using the BlueDisplayExample sketch with a fast blink (say 100ms), the on-screen blinker loses sync with the Arduino LED, and refreshes at less frequent and random seeming intervals thereafter. The problem goes away briefly if you interact with anything on the BlueDisplay screen; for example, tapping the + button in BlueDisplayExample. After a few more seconds the problem returns. This is happening on 2 out of 3 devices I have tried. It is happening on a Kindle Fire HD 8 running current Fire OS (6.3.1.5), and it is happening on a Nexus 6P (don't have access to see the Android version right now, probably 8.x). It is not happening on a Pixel 4 XL running Android 10.

Any thoughts? Seen this before? I suppose it could be the OS trying to throttle app CPU consumption whenever there is no interaction. If it matters, I am using this BT module: https://smile.amazon.com/gp/product/B07XVM8Y27 and 115200 baud.

[ArminJo]

Hi Matthew, I know these effects, on a few devices the DSO example lags more and more until it stops. It seems to correlate with the CPU power. But the stop is caused by getting no data from the Bluetooth driver. So I assume, that a buggy Bluetooth driver implementation for Serial is the main cause for that. I put a few days in searching for delays and reducing garbage collections, but I really have no clue what the real cause is. On my Nexus7 Tablet with 6.0.1 it lags ans stops, on my Note3 with 5.0 it runs perfectly. Maybe I start a new research, the bug is really annoying.

[MatthewMWR]

Thanks. I have been doing some experimenting with reducing data volumes and draw frequency to see if that is a good workaround. So far results are inconclusive. I'll let you know anything else I find.

[MatthewMWR]

I've been working on creating a minimal repro in case that sparks any ideas. Below is what I'm down to so far. The LED and the BlueDisplay app screen stay in sync for about 5 flashes before the screen starts refreshing slowly/erratically while the LED pulses steadily. Whenever I touch the screen (but only in the draw/canvas area) it recovers for a few seconds before losing sync again. If I touch the screen outside the draw/canvas area (just the black background BlueDisplay has otherwise) the recovery phenomenon does not happen. I can make a video if that would be interesting.

Minimal repro so far: `#include

include "BlueDisplay.h"

ifndef BLUETOOTH_BAUD_RATE

define BLUETOOTH_BAUD_RATE BAUD_115200

endif

define DISPLAY_WIDTH DISPLAY_HALF_VGA_WIDTH // 320

define DISPLAY_HEIGHT DISPLAY_HALF_VGA_HEIGHT // 240

const char pulseChar = 'x';

void initDisplay(void); void pulseBlueDisplay(void);

void setup() { pinMode(LED_BUILTIN, OUTPUT); initSerial(BLUETOOTH_BAUD_RATE); BlueDisplay1.initCommunication(&initDisplay, &pulseBlueDisplay); initDisplay(); pulseBlueDisplay(); }

void loop() { pulseLED(); pulseBlueDisplay(); delay(1000); }

void initDisplay(void) { BlueDisplay1.setFlagsAndSize(BD_FLAG_FIRST_RESET_ALL, DISPLAY_WIDTH, DISPLAY_HEIGHT); }

void pulseBlueDisplay(void) { BlueDisplay1.drawChar(50, 50, pulseChar, 20, COLOR_BLACK, COLOR_WHITE); delay(80); BlueDisplay1.clearDisplay(COLOR_BLUE); }

void pulseLED(){ digitalWrite(LED_BUILTIN, HIGH); delay(10); digitalWrite(LED_BUILTIN, LOW); } `

[ArminJo]

Thanks a lot, maybe I can test your code this week and profile/debug the app. Regards Armin

[SuchaiNop]

I also got the same situation on my Lenovo K3 Note with 6.0

[ArminJo]

What I found.

1. If I connect the Arduino with a USB cable, the effect vanishes.
2. After some random amount of loops, the smartphone makes a "short break" and then resumes, but does not skip something. I changed the example to:

   
   void setup() {
   pinMode(LED_BUILTIN, OUTPUT);
   initSerial(BLUETOOTH_BAUD_RATE);
   BlueDisplay1.initCommunication(&initDisplay, NULL);
   }

void loop() { static int sXPos = 0; static int sYPos = 10; digitalWrite(LED_BUILTIN, HIGH); BlueDisplay1.drawChar(sXPos, sYPos, 'x', 16, COLOR_BLACK, COLOR_WHITE); delay(50); BlueDisplay1.drawChar(sXPos, sYPos, ' ', 16, COLOR_BLUE, COLOR_WHITE); digitalWrite(LED_BUILTIN, LOW);

sXPos += TEXT_SIZE_16_WIDTH;
if(sXPos >= DISPLAY_HALF_VGA_WIDTH) {
    sXPos = 0;
    sYPos += TEXT_SIZE_16;
}
delay(250);

}

void initDisplay(void) { BlueDisplay1.setFlagsAndSize(BD_FLAG_FIRST_RESET_ALL | BD_FLAG_USE_MAX_SIZE, DISPLAY_WIDTH, DISPLAY_HEIGHT); BlueDisplay1.clearDisplay(COLOR_BLUE); }

I will research further.

[ArminJo]

I added a chapter in the readme.md

Known problems

Depending on the device you use, you can observe some "breaks" in the timing of the display refresh. The "breaks" does not occur if you use a USB connection instead of the Bluetooth one.
The reason is, that the Bluetooth driver does not return for a longer time. This driver is usually delivered by the hardware vendor, so it may depend on the chips used in your smartphone.
Known devices are: Lenovo K3 Note with 6.0, Nexus7 with 6.0.1, Nexus 6P with ?8.x?, Kindle Fire HD 8 with 6.3.1.5.

Can you confirm, that it is a "break" also for your device? Regards Armin

[MatthewMWR]

It sounds to me like our experiences with that latest repro might be different. Here's a video, which shows my experience better than I could describe: https://photos.app.goo.gl/k76Sk4gn47uYzP3h6

If you watch the whole thing to see the behavior come and go with screen touch.

[MatthewMWR]

On second thought this video is probably more useful, it starts with a clean background so you can see the rendering pattern better, and it shows that where you touch makes a difference.

https://photos.app.goo.gl/HD5ZoD8oEudVkYon9

You'll have to forgive my dog trying to get our attention. :)

[ArminJo]

Hi Matthew, Thanks for your Video. Here you see the behavior of the Bluetooth SPP profile. Even with an ESP32 it is not realtime. I have no clue how to improve the timing.

[ArminJo]

Hi Matthew, I think, the Android scheduler on your tablet behaves different when touched on the screen. But this is something I can not influence 😟 . Do you have an USB-Host adapter to check if the delays vanish if using USB-connection? I documented the random delays on the homepage. If you have any idea,what I can try, you are welcome, otherwise I will close this issue.

[MatthewMWR]

Thanks for looking into it. I agree the touching phenomenon feels like some kind of thread priority shift. Weird though that it depends on where in the app UI one touches. Unfortunately, I am using a Mega (which uses USB type B connector), so I don't have the cabling or adapters I would need to directly connect the tablet.

I'm thinking of trying a whole new approach where the Arduino isn't trying to drive the tablet in real-time with layout and everything, but instead is just sending sensor data points. I'm thinking the tablet may be rendering a (local?) web page (either just in the browser or in an app which is a web view). The page would respond to incoming data from the Arduino. Not sure yet how I will connect the two.

I might have to write an Android app (never tried before!) which takes data from Bluetooth and posts writes it to a file or posts it to a URL or....

Maybe this could be a future operating mode in BlueDisplay or a sister project. From Arduino library you just send data, and then in the Android app there are predefined options for how to render the data (like Blynk, but without their cloud service), or where to send the data (CSV, HTTP POST, etc.).