Waveshare 4.2" Display - Greyscale Driver

[surdouski]

What's your view on a V2 greyscale driver? Do you want to do this?

I'm interested in giving it a shot. I'm a bit unsure how to test the greyscale though. How did you test you other gs driver?

Edit: This does look quite a bit more difficult than the other driver and I may just outright fail at this attempt, but I'll give it a go anyways.

[peterhinch]

I've pushed a revised DRIVERS.md detailing your driver. You might be interested in my current consumption measurements. These confirm my view that the only reason to issue .sleep is prior to a shutdown - there are no power savings to be had. I supplied but never documented the .display_on method as I can't see a use case for it.

Re the 2-bit driver, this paragraph indicates a simple test. I only tested with nano-gui because on the V1 display partial updates and 2-bit colors were mutually exclusive (micro-gui is unusable without good partial updates). I will be interested to see if the options can coexist on the V2 version.

On nano-gui it's best to pass color values in the range 1 (light grey)..3 (black) as per the demo script.

[surdouski]

• 92μA while inactive.
• 92μA after running .sleep. Conclusion: there is no reason to call .sleep other than in preparation for a shutdown.

That's surprising!

[surdouski]

I'm a tad bit confused by the final table in this image. I'm unsure how the values for the rows "0x10 Real Transmission" and "0x13 Real Transmission" are obtained. Is the table supposed to be split into 7 and 9 bits, or is that just a visual error in the table they are presenting?

_source: https://www.waveshare.com/wiki/4.2inch_e-Paper_Module_Manual#Introduction_

[surdouski]

Just came back for another look at this and I think they just made some typo's on the table. The idea itself made sense before I tried to line it up with the 1s and 0s in the rows for 0x10Real Bit and 0x13Real bit.

Update: I went through your gs driver as well as some supplied code from waveshare and have had very little success. I'm probably a little bit out of my depth on this.

[peterhinch]

Note that EPD's have a white background while other display types have black. The sense of the colors in my drivers is reversed relative to those tables: 0 is white (background) with 3 being black. This mapping enables that demo code written for TFT displays renders sensibly on EPD's.

I'm probably a little bit out of my depth on this.

My view of the 2-bit driver is that it's not urgent. If you want to carry on experimenting and learning, take your time. If you reach a point where you decide to abandon it, please let me have what code you have and I will try to knock it into shape.

I'm very pleased with the 1-bit driver: I wanted to retain a currently available EPD that works with micro-gui. You did a great job figuring out and matching the API in the V1 driver. Thanks for doing this work.

[surdouski]

My view of the 2-bit driver is that it's not urgent. If you want to carry on experimenting and learning, take your time. If you reach a point where you decide to abandon it, please let me have what code you have and I will try to knock it into shape.

In that case, I'll continue to poke at it for a bit. I'll be sure to let you know if I give up on it.

[surdouski]

Made an initial draft, seems to be working correctly for the example you mentioned.

[peterhinch]

Excellent.

I'd very much like to know whether the hardware allows for a greyscale driver capable of partial updates. Use with micro-gui depends on partial updates, and micro-gui compatibility affects the driver API. A greyscale driver for micro-gui would be a big plus!

From my brief look at the Waveshare code I suspect that partial and greyscale are mutually exclusive, as in the V1 display but I'm keen to hear your view/experience.

[surdouski]

Have you seen any greyscale epaper displays with partial updates? I did try messing around a little bit and have so far been unsuccessful.

---

After doing some more research, I think it's probably doable, but would take a decent amount of trial and error. While I am interested in these things, I have a feeling a lot of this knowledge will be inapplicable to other domains and thus maybe too low-value for me to do (depending on how long trial and error takes). But since I am interested for the time being, I'll let you know if and when I abandon that particular questline entirely.

---

Somewhat tangent to that -- I am now thinking it makes sense for epaper-display drivers to additionally have a setting relating to the number of partial refreshes that can be done subsequently before forcing a full refresh (due to possibility of burn-in). I think it's easily enough supported as either an __init__ kwarg with a default of something like 60, which would protect anyone who was not aware. Do you have any thoughts on this?

[peterhinch]

I don't think a trial-and-error approach is a good idea. There would always be a doubt as to whether any long term damage was being done to the display. I think we need to assume that Waveshare understand their own hardware.

My rather brief study of Waveshare's driver indicates that they don't support partial updates of greyscale. If I'm right in this I don't think we should attempt to improve on this. The best we can do is to copy Waveshare's code in terms of the specific data sent to the panel.

Assuming the greyscale driver does not support partial updates, then we only need to provide the nano-gui API. See the existing V1 driver.

I have contacted Waveshare to request a datasheet for the chip.

Do you have a reference to this burn-in issue? I've never heard of this - and I've never experienced a full refresh that failed to clear an ePaper display. If this is real I'll have to think about whether this should be handled by the driver, the GUI, or the application.

[surdouski]

I don't think a trial-and-error approach is a good idea. There would always be a doubt as to whether any long term damage was being done to the display. I think we need to assume that Waveshare understand their own hardware.

My rather brief study of Waveshare's driver indicates that they don't support partial updates of greyscale. If I'm right in this I don't think we should attempt to improve on this. The best we can do is to copy Waveshare's code in terms of the specific data sent to the panel.

Fair enough, I will put a halt to any tinkering on that -- they do not appear to support partial refresh for greyscale.

---

Assuming the greyscale driver does not support partial updates, then we only need to provide the nano-gui API. See the existing V1 driver.

I'll take a look and reconcile any discrepancies tomorrow.

---

Do you have a reference to this burn-in issue? I've never heard of this - and I've never experienced a full refresh that failed to clear an ePaper display. If this is real I'll have to think about whether this should be handled by the driver, the GUI, or the application.

I've seen it mentioned in quite a few places as it relates to epaper. I think you can probably look for youtube videos of the image being burned in, particularly this video talks about it (this is timestamped). That video is actually about custom waveforms, however, but it is apparently something that can happen if you continually partially refresh and never do full refreshes.

Waveshare mentions it a few times on the info page here for the 4.2" waveshare driver. Taking a few quotes from the page...

For e-Paper displays that support partial refresh, please note that you cannot refresh them with the partial refresh mode all the time. After refreshing partially several times, you need to fully refresh EPD once. Otherwise, the display effect will be abnormal, which cannot be repaired!

Partial refresh: The e-paper screen has no flickering effect during the refresh process. Users who use the partial brushing function note that after refreshing several times, a full refresh operation should be performed to remove the residual image, otherwise the residual image problem will become more and more serious, or even damage the screen (currently only some black and white e-paper screens support partial refreshing, please refer to product page description).

I'm pretty sure this is a real thing, however, I'm not truly certain how many times a partial refresh needs to be done in order to start getting this "burn-in". Also, in that video I linked, the person talking actually mentioned a fix for the burn-in, which is to constantly flash the screen black/white and occasionally checkered pattern for about a weeks worth of time to get the pixels to get "unstuck."

[peterhinch]

That Waveshare doc is really extreme - I hadn't seen that.

Those recommendations imply that running with micro-gui is not feasible. A typical micro-gui application will run 24/7, with the GUI calling the driver's do_refresh method continuously with partial updates. It would be easy to suggest that users create a task to regularly perform a full refresh, but the Waveshare doc recommends against continuous uptime.

Their recommendation against storing the devices with an image on screen is bizarre. I have had ePaper devices delivered with an image on screen, and have stored them in that state for years without any sign of deterioration. I also have an application which has been running for at least eight years so my impression of ePaper is that its long term reliability is good. These observations are on an early model of ePaper display - I do wonder if they have really got worse.

I'll start a long running test.

I'm not keen on the driver forcing a full refresh on the user. I think GUI users should have control of this process. If an application wants to do a full refresh every N partial ones, the API provides the means to do this. Depending on the outcome of my long running test I'll consider having micro-gui run a task to do this - or I might simply document code for users to implement or adapt.

[surdouski]

I've waded through some other content related to epaper and my understanding is that their suggestions are a bit too severe, but there is a hint of truth to the possible consequence.

I'll start a long running test.

I'm curious to any results you find.

[peterhinch]

So far I've run up about 24 hours of running with no ill effects. I can't yet do 24/7 running because I have only one display which I need to use for testing, but whenever I'm not actively using it, it's running the micro-gui demo. I'll order another unit.

That is the good news. Unfortunately there is also some bad news. I wanted to set up a second Pico so that it could run 24/7 with the new display when it arrives. After several hours work I haven't managed to achieve this. Here are the steps I went through.

1. Got a brand new Pico, installed a build of firmware with micro-gui and the drivers frozen as bytecode. Wrote a hardware_setup.py with pin numbers for the built-in socket. Tried to run the epaper demo: the Pico crashed. No traceback, a hard crash. Very carefully checked pin numbers - all OK. Studied Waveshare schematic (although it's clearly for a V1).
2. Changed the pin numbers to match my test jig, plugged the Pico into test jig, connected the display by the Waveshare cable. Still crashed. This despite using a different SPI bus and different pins compared to the built-in socket.
3. Thinking frozen bytecode might be at fault (rather a long shot) I installed the exact same build of firmware as on the working board. Copied micro-gui and drivers to Flash. Plugged into test jig. Crash. Concluded that the Pico must be at fault (despite the fact that firmware and code installed correctly, and a REPL appeared).
4. Repeated with another brand new Pico and repeated (3). Crash.

This is a somewhat condensed version of the testing I did. I removed the overclock from the setup to no effect. I proved that the working Pico also works in the built-in socket.

I have one Pico that is 100% reliable and two that score 0%.

I'm seriously puzzled by this, not least because I can't see how the driver can crash the Pico. You have to do something radical to crash a MP host like directly accessing host RAM or registers or run assembler code. There is nothing in the driver, GUI or demo that does this. Testing with the built in socket removes any question over the wiring: in this mode there is no wiring!

I'm clearly going to have to get serious about finding what's causing the crash and break out the logic analyser and scope.

Ideas welcome. Have you tried more than one Pico?

[surdouski]

I do not have any ideas, but I have tried more than one pico. I'll try out a few more other picos in the morning and report back after I do.

[peterhinch]

I think I've solved it.

[EDIT] Removed incorrect conclusion re hardware.

For some reason freezing the driver causes the crash. It's possible to freeze all of micro-gui including the demo script, so long as the driver is not frozen.

The very puzzling thing is that if the driver is precomplied as bytecode and put in the filesystem replacing the .py file, it works. Yet the same bytecode, when frozen, fails. This makes no sense and suggests some kind of MicroPython bug, but I can imagine trying to produce a repro could be challenging.

Apologies for the confusion, I'll let you get on with your efforts.

Re long running test: this continues with no apparent problems.

[surdouski]

The very puzzling thing is that if the driver is precomplied as bytecode and put in the filesystem replacing the .py file, it works. Yet the same bytecode, when frozen, fails. This makes no sense and suggests some kind of MicroPython bug, but I can imagine trying to produce a repro could be challenging.

That sounds like quite the challenge to debug, I haven't the slightest idea.

[surdouski]

I believe it now matches the API of the previous, of which I'm assuming you meant the "pico_epaper_42_gs" driver. I don't believe I saw any mismatch of methods, but I did shuffle them around to match the ordering to make it a bit easier to see any discrepancies.

However, I don't believe the do_refresh is needed? I do see it in the previous driver, though.

[peterhinch]

Interesting point. It begs the question of whether to provide async support. If we get rid of do_refresh then _as_show becomes redundant. On the other hand if we keep it, I should document it.

I think we should keep it. It enables an async application to control and limit blocking time, and keeps the API consistent across the four drivers. It also means nothing will break if someone runs micro-gu (although the continuous full refreshes make it unusable in practice0.

Let's make the call signature

    async def do_refresh(self, split=0):  # For micro-gui

to save users from having to specify an unused arg. (micro-gui supplies this arg).

[surdouski]

I think we should keep it.

Sounds good to me. Updated the call signature and rebased all the commits into a single commit for tidyness.

---

Also updated to do the check for async from the previous update so that just sb() can be called in synchronous applications.

[peterhinch]

I've looked at the code and it seems good. Except one thing: _LUT is implemented as a list of strings. (I know Waveshare do this but it's horribly inefficient.) In your earlier implementation you implemented this as a bytes object. See also the LUT's in the V1 GS driver.

I would recommend reverting to bytes and using a memoryview to implement allocation-free slicing. If you want any assistance in implementing this I'm glad to help.

[peterhinch]

Or, if you prefer, I'll merge this as-is and fix up the LUT.

[surdouski]

I'll get around to in the coming days, just been a bit busy lately is all.

[surdouski]

Ok, managed to find some time. Did I do this correctly? It is working on my device, but I'm not 100% sure that this is what you were asking for.

[peterhinch]

Thank you!

[peterhinch]

@surdouski To keep you posted, my long running test on this display has completed 1000 hours. This shows continuously changing data by partial updates only using this script. No full updates. While there is visible ghosting this hasn't got any worse over the duration of the test.

[surdouski]

I have to admit, I did not expect this result; however, I am delightfully surprised.

[peterhinch]

I've never before seen a case of a manufacturer dissing their own product to such an extent. I do wonder if the warnings in that that wiki doc were written for another product. It's odd that there are two wiki posts, one without the warnings. I have mixed views about Waveshare... But this display is very good.

I'll leave the test running for at least another 1000 hours.

[surdouski]

It is indeed odd.

Could you try doing a full refresh after those 1000 hours to see if any of the ghosting is starting to linger thru that?

[peterhinch]

As far as I can see the ghosting does not get worse with time. Take the meter in the demo script. Once the bar has reached the top, and then returned to zero, slight ghosting is visible. 1000 hours later, it looks exactly the same (to my eyes). Adjacent to it is my other display. It is unpowered and shows the outcome of the greyscale test using your driver. The color of the background of both displays looks identical - I had wondered if it might darken in the running unit - it doesn't.

Given that there seems to be no worsening with time I think I'll give it another 1,000 hours before refreshing.

[peterhinch]

I've now ended the test at 2000 hours. I can see no evidence of long term change. It looks the same at 2K hours. I think these displays are excellent.

[surdouski]

I have recently begun using mine with partial updates only -- so that's great news!