transfer chunks of data

[MaBecker]

Hi,

have you thought about transfer chunks of data instead of bytes for a full screen update ?

https://github.com/noopkat/oled-js/blob/master/oled.js#L398

What do you think ?

[MaBecker]

I expect this to be much faster as updating dirty bytes

[MaBecker]

https://github.com/MaBecker/oled-i2c-bus/commit/c099f6145024ea9b7c190d16a936eba45232e2a8

[noopkat]

@MaBecker hi! Thanks for the suggestion. Would you mind explaining this technique in a bit more detail, and how it differs to this here, which finds the smallest chunk area to update (so that the 7 byte header is only sent once overall)?

I'm sure it's faster as you're telling me it is, I'm just not quite grokking the concept from the code alone. Thanks! 😁

[noopkat]

@MaBecker it looks like perhaps it saves on the repetitive firmata I2C calls? That's my best understanding, it's been a while since I wrote this library hehe

[MaBecker]

Hi @noopkat,

sure - let me try ;-)

so first of all I use all call with sync = false to avoid this._updateDirtyBytes(this.dirtyBytes);

because this does a byte by byte update of all the changes in buffer.

The big thing is to send the complete buffer with 8 calls of 128 bytes to the oled and not with 1024 calls of 1 byte.

So a total update is even fast as a update of dirty bytes.

please let me know if you need more information.

[normen]

Hi, I did this in my fork of a fork of this library and it's indeed MUCH faster. I was using the C++ library to access the display before and was a bit shocked by how slow the JS library was so I investigated.

https://github.com/normen/oled_ssd1306_i2c

Specifically this commit: https://github.com/normen/oled_ssd1306_i2c/commit/bd46541b9056e18789ca9e570f624fb0907fc941

[noopkat]

Hi @normen

Thanks for showing me your code! The reason why my library is so slow is because some Arduino boards emulate their USB connections. This means you cannot send large chunks of bytes to the device in this way. I would encourage you to give this a try yourself using any Arduino board that uses a 32U4 chip, as it is a noticeably bad experience to have this library not work properly due to overwhelming the device. I have had some success by sending up to 10 bytes at a time, but it didn't feel stable enough to publish officially in this library.

What I do like about your sample though, is that you're allowing the chunk size to be specified by the library user. This is something I can definitely implement and document, while leaving the default chunk size alone so that it retains the maximum compatibility with all boards. This will probably be the happy path going forward.

[normen]

I see. I am using the OLED with an Arduino Micro, so that's a 32U4 as well. While it's true that the performance is slightly worse than with the C library on RasPi (my only other comparison) in my experience thats mostly due to the Arduinos speed and not the displays.

The JS library (at least the fork I used - I didn't try your current version) performed so much worse that it prompted me to look for the issue.

Are you sure that the issue isn't rather the specific OLEDs you're using or your cable connection? I am running these displays on a 600kHz i2c bus and they seldom drop any data? I can animate single bitmaps like falling stars no problem with the C library on RasPi.

[noopkat]

that's so interesting! I have always tested this library with an Arduino Micro also 🤔 What is the largest chunk size you're able to successfully send using my version of this library rather than any forks (if you just quickly change the chunk size in my source)?

I guess I'm interested in if you're writing this C library to use the Firmata Protocol specifically as well. Firmata running on an Arduino takes up a lot of memory as far as the sketch size goes. So my theory is that it can't maintain large buffers of data before being able to free it up after sending it via i2c.

[normen]

Sorry, the fork I am using is running on RasPi completely, using the i2c library to access the displays via the RasPis GPIOs so I can't test your code. However, the C library I am using for Arduino and RasPi seems to send 16 bytes per package: https://github.com/normen/ArduiPi_OLED/blob/4119148b051a6975816de2319eb6d11f6769499d/ArduiPi_OLED.cpp#L943

With my modified version for your JS library on RasPi-I2C I can send 32 bytes per package at 600kHz without issues. With the C library I do get the occasional hiccup when writing a lot at 600kHz - it does seem to push the data yet a bit faster for some reason. But with that I already get pretty much fluent animations at 100kHz.

[noopkat]

ah! I see. Yes this is definitely not an apples to apples comparison. This library was originally designed for integration with Johnny-Five specifically which uses Firmata protocol, over a USB cabled connection. Firmata running on the board is probably the main bottleneck when using my library (it's almost the full 32kb in size), and combined with the emulated USB connection there's going to be issues. This is why I was super conservative after noticing problems with chunk size initially 😄

[normen]

I was just wondering because you seem to have a reason to pin the issue to the data transfer to the OLED - and I saw these little things handle more data that I expected. In my experience data transfer issues are hardware issues more often than not so maybe a few caps around the data transfer lines could make the appropriate bug fix? :)

[noopkat]

Perhaps I am completely misunderstanding you, so let me know if the below makes sense 😄

We're perhaps talking about two different transfers and not the same? I am specifically targeting the USB side of things and the buffering the board is doing.

I'm not pinning it to the OLED screen's capabilities at all. I know that i2c devices can handle a lot of data. I'm pinning the issue to the buffering of the bytes chunks which firmata likely holds onto in memory while it's transferring it via i2c to the screen. Firmata takes up almost the full program memory on most Arduino boards, so the buffer space is extremely limited. Sending smaller chunks most likely allows those chunks to be freed up faster for the next chunks coming through the serial connection.

[normen]

Alright, I see. My impression was mainly coming from how the JS code for the OLED is laid out - the error check and always sending one byte at a time. That looked like there was issues being solved in the past :) As said I am using a fork of your code and have really no idea of this johnny-five related setup. So my perspective is pretty much solely the JS code and while I was looking into getting this change in as far upstream as possible I found this issue.

So yeah, broadening my view to all of that I suppose your problem could be anything ;P

[noopkat]

thanks for getting in touch about it all the same. I'll consider implementing the dynamic chunk size option in the API but as far as my tests go not many will be able to make use of this feature given the nature of how this is integrated.