Dithering support

[CptPotato]

From what I can tell, Plato doesn't seem to utilize dithering when drawing images compared to the standard document viewer of my Forma. I'm using the CBZ format so this might be isolated to MuPDF only (I haven't checked if other formats like ePUB have the same behaviour, yet).

[baskerville]

As of 2a67bc470e64a301639016d8b29d2eca8f8652e2, Make Bitonal (in the main menu) will use dithering on mark 7 devices.

[CptPotato]

Thanks for the reply. I tried the bitonal setting but it switches to 1 bit colors which is not really what I was looking for (also the screen updates seem to be broken for me when using it :/ ).

What I actually meant is grayscale dithering using all the shades that the device can display (I think it's 8 or 16 on mine):

Within Plato you can clearly see the single steps of gradients as in the top half of the image. From what I can tell it outputs the content without modifications which is then clipped to the closest shade that the device can handle (which results in the visible steps). When I use the standard viewer instead, it dithers the output beforehand and gradients seem much smoother as in the bottom half.

I don't really know the ins and outs of this display technology so this might be more complicated than I initially thought.

[baskerville]

(also the screen updates seem to be broken for me when using it :/ ).

Could you elaborate?

Could you try the version before the aforementioned commit and see if the screen updates are also broken?

What I actually meant is grayscale dithering using all the shades that the device can display

I see what you mean now.

(I think it's 8 or 16 on mine):

It's 16 on all the devices.

I don't really know the ins and outs of this display technology so this might be more complicated than I initially thought.

Could you remove the following line, recompile and try the bitonal setting once again?

@NiLuJe: could you strace Nickel and see if it's automatically using dithering (on top of which waveform?) when reading CBZ/CBR files?

[NiLuJe]

GLR16 + ORDERED @ q7 for partials. GC16 + ORDERED @ q7 for fulls. (Which basically translates to: business as usual, with HW dithering on top ;)).

The RMSDK also does a (useless, on these devices) software ordered dithering pass. ACCESS can do it too, but I don't rightly recall if it stacks the two. It definitely uses the HW one, though.

[NiLuJe]

(also the screen updates seem to be broken for me when using it :/ ).

Could you elaborate?

That probably trips the "usual" Mk. 7 bugs where stacking EPDC flags has a rather strong tendency to cause the hardware to ignore all of them ;p.

(The v2 dithering flags have usually been safe, in my experience. As long as you don't stack INVERT on top of that. And while you can stack FORCE_MONOCHROME, you shouldn't, because it's applied before dithering, which is obviously counter productive ;))

[baskerville]

REAGL + ORDERED @ q7, IIRC. Or GC16 + ORDERED @ q7.

Thanks, is it using dithering only for CBZ/CBR?

In the fbink comments, I see that you recommend Y1 instead of ORDERED?

[CptPotato]

(also the screen updates seem to be broken for me when using it :/ ).

Could you elaborate?

A lot of the screen's content (or basically all of it) doesnt get cleared; for example when menus disappear or new pages are displayed. On screenshots everything is fine so here's a photo after flipping through a few pages:

photo

Could you try the version before the aforementioned commit and see if the screen updates are also broken? ... Could you remove the following line, recompile and try the bitonal setting once again?

I could give these a try.

@NiLuJe: could you strace Nickel and see if it's automatically using dithering (on top of which waveform?) when reading CBZ/CBR files?

Not sure if it's helpful but I've observed that dithering is only performed when drawing the whole page. If the menu is visible while the page refreshes it is disabled.

[NiLuJe]

Thanks, is it using dithering only for CBZ/CBR?

Everywhere outside of the UI. Thumbnails may or may not be software dithered, depends on what generated them. But ACCESS will switch to HW dithering when encountering images in KePubs, for instance. (On older devices, and modern Nickel versions, those will be software dithered, too, FWIW).

In the fbink comments, I see that you recommend Y1 instead of ORDERED?

Only instead of q1 dithering (e.g., @ A2/DU), which is admittedly not very useful to begin with.

[baskerville]

(also the screen updates seem to be broken for me when using it :/ ).

Could you elaborate?

That probably trips the "usual" Mk. 7 bugs where stacking EPDC flags has a rather strong tendency to cause the hardware to ignore all of them ;p.

Unless @CptPotato also had Invert Colors set, the only flag present would have been Y1.

The v2 dithering flags have usually been safe, in my experience. As long as you don't stack INVERT on top of that.

What happens if you do?

[NiLuJe]

The v2 dithering flags have usually been safe, in my experience. As long as you don't stack INVERT on top of that.

What happens if you do?

Risks triggering the bug where all the flags are ignored until a couple of actual unflagged updates are sent ;).

[CptPotato]

I just realized the commit you've mentioned (2a67bc4) was just from earlier today, so actually I've been running the latest release version. Sorry for the confusion. Though it seems that building on windows is not a thing as of now, so I'm not able to build and test current master right now.

Also to clear things up, Invert Colors was not enabled, I was only using the bitonal setting in the photo above.

[NiLuJe]

Yeah, A2 would be a ghost factory on Mk. 7, and suffer from myriad weird refresh glitches if the actual framebuffer content isn't already bitonal. (Which should be FORCE_MONOCHROME's job, but, eh, there you have it). Even then, it's potentially glitchy as all hell beyond simple "highlight a button for me" use-cases.

[baskerville]

A2 would be a ghost factory on Mk. 7, and suffer from myriad weird refresh glitches if the actual framebuffer content isn't already bitonal.

Can the same be said about DU?

[NiLuJe]

It's vaguely less buggy, yeah. But a DU + FORCE_MONOCHROME + INVERT stack will still trip the flags bug.

FWIW, just double-checked on 0.9.12, and yeah, make bitonal is utterly unusable on Mk. 7. (As in, I had to do a classic refresh via FBInk over SSH to even see the menu to disable it ;p).

DU + Y1 usually works, but I've never really used it outside of one-offs for fun, so, who knows. If the intended use-case of Bitonal is speed, that's potentially a no-go, since the dithering is done purely in C by the EPDC driver in kernel.

DU + q1 ORDERED "works", but looks like crap, and reveals the PxP's tiled processing patterns (outside of rotation 0, where it's slightly less awful).

[baskerville]

It's vaguely less buggy, yeah. But a DU + FORCE_MONOCHROME + INVERT stack will still trip the flags bug.

But does DU require the framebuffer to already be bitonal?

DU + Y1 usually works, but I've never really used it outside of one-offs for fun, so, who knows. If the intended use-case of Bitonal is speed, that's potentially a no-go, since the dithering is done purely in C by the EPDC driver in kernel.

I see, I thought the Mk. 7 devices only did hardware dithering.

[NiLuJe]

It's vaguely less buggy, yeah. But a DU + FORCE_MONOCHROME + INVERT stack will still trip the flags bug.

But does DU require the framebuffer to already be bitonal?

Less markedly, because you could get away with the extra 2 shades of gray in its palette, but, for all intents and purposes, yes, it won't work outside of small and controlled regions.

Because the fun doesn't end here, FORCE_MONOCHROME works properly, and as such fixes the refresh issues, but exposes you to the flags bug ;). Which makes things extra broken when it trips ^^.

[NiLuJe]

I see, I thought the Mk. 7 devices only did hardware dithering.

They kept the standard set of software algo (Y1/Y4), but they dropped the custom D8 introduced on Mk. 6. Which makes some kind of sense. Y4 has a niche use case that no-one generally cares about, but may or may not behave better than the q3/q4 HW variant. D8 made no sense given that the q7 HW variants works perfectly well, and you get it essentially for free. And Y1 looks much, much better than its q1 HW counterpart.

[CptPotato]

So... you lost me a little on the dithering modes and hardware specifics there :) But from what I understand a one-fits-all solution is out of question at this point since this is handled differently across devices (unless it's done in software, probably).

Still, let me know when there's something I can help with.

[NiLuJe]

For your actual original query? No, that's easy, just stack ORDERED @ q7 on image content, and you're done. c.f., KOReader, where this is automatic in the UI & ePubs; and a per-document choice for PDFs & co. And, well, Nickel ;).

The issue lies in Plato's Bitonal switch. If the intent was speed, then you can forget about it and just drop it on Mk. 7, too many bugs to make it predictable in all instances (with the one aggravating factor being nightmode, though). (You don't really need things to go any faster than GLR16 anyway). If the intent is instead the wow factor, then swap it to DU + Y1 (at least on Mk. 7, possibly everywhere ;p). Or DU + ORDERED @ q1, knowing that it'll look like crap.

EDIT: Forgot that Y1 & Y4 are Mk.6+, anyway.

[baskerville]

@NiLuJe: To answer your comment regarding EPDC_FLAG_USE_DITHERING_NTX_D8: I've tried it on the Aura ONE and it looks like crap: the light gray background of the directories bar, for example, is rendered with a much darker gray covered with evenly spaced black dots…

[NiLuJe]

Ah, good to know, thanks ;).

[baskerville]

Did you know that GC16 can do partial updates? And can do so when UPDATE_MODE_FULL is passed?

---

Looking at the mapping from waveform to quant_bit, I'm guessing that it has to do with the number of colors that the waveform can display. But why are A2 and DU both @ q1? I thought DU could display 4 colors. How many colors can GC4 display?

[baskerville]

Here's how EPDC_FLAG_USE_DITHERING_NTX_D8 renders your test image on the Glo HD:

[NiLuJe]

Did you know that GC16 can do partial updates? And can do so when UPDATE_MODE_FULL is passed?

Yep, the fact that we use it with FULL is just a convention given the intended use-cases, but it's perfectly usable without a flash. (Some kernels may enforce a flash regardless, though).

---

Looking at the mapping from waveform to quant_bit, I'm guessing that it has to do with the number of colors that the waveform can display. But why are A2 and DU both @ q1? I thought DU could display 4 colors. How many colors can GC4 display?

As best as I can tell, GC4/DU4 is severely broken/missing on NTX boards. It's noticeably less useless on Kindle kernels, for example. And they do add two colors to the palette (0x55 & 0xCC, if memory serves).

I don't rightly recall the details, because it's generally crap, but I probably settled on q3 for GC4 because it yielded the least terrible results.

[NiLuJe]

As for DU, it's as bitonal as A2, the only difference is that it handles switching from grays, instead of only from B & W. (So q1 makes perfect sense for DU).

[NiLuJe]

Hence the recommendation of bracketing A2 refreshes between a white screen, to ensure the "from B & W only" A2 restriction is met.

(And that works on Mk. 7, FWIW).

I suspect older kernels don't actually expose A2, and instead use DU in its place. Or simply switch to DU as-needed if the requirements aren't met during the histogram analysis.

[NiLuJe]

But that's on paper ;).

It's vaguely less buggy, yeah. But a DU + FORCE_MONOCHROME + INVERT stack will still trip the flags bug.

But does DU require the framebuffer to already be bitonal?

Less markedly, because you could get away with the extra 2 shades of gray in its palette, but, for all intents and purposes, yes, it won't work outside of small and controlled regions.

In practice, this observation makes me think that DU may have a better time handling things if the "from" pixel isn't any grayscale, as it's supposed to be able to handle, but only the GC4/DU4 palette (0x00, 0x55, 0xCC, 0xFF).

This is slightly trickier to test, because I tend to do those tests with images, which DU is really not made for.

And I keep remembering Kindles, where DU may magically inherit the capabilities of DU4. But I just re-checked, DU is definitely bitonal on Mk. 7, while GC4 can display 0x55 & 0xCC.

(The intended use-cases for DU4 would have been stuff like AA text in menus. In practice, that's bearable but not great on Kindles, where we actually have a DU4 that works mostly as advertised. On Kobo, we don't have a DU4, we have a GC4, and it looks like absolute crap in that use-case ^^).

[NiLuJe]

In practice, this observation makes me think that DU may have a better time handling things if the "from" pixel isn't any grayscale, as it's supposed to be able to handle, but only the GC4/DU4 palette (0x00, 0x55, 0xCC, 0xFF).

That, or it's just an artifact given what happens to pixels aren't B & W in the fb: they're left untouched on-screen, which can yield weird overlay effects. (And makes the fb content differs severely from what's actually visible on screen).

And that may or may not be device specific (e.g., I seem to recall Kindles whiting out those pixels instead), but I may be wrong.

[NiLuJe]

In fact, thinking about the results of DU + Y1/q1/FORCE_MONOCHROME, it all makes perfect sense:

DU is from anything to B & W only: pixels that aren't B/W in the "new" content are left untouched on screen (hence the potentially confusing results, with new content overlaid on the previous content. Meaning the fb and screen content may differ significantly).

(i.g., the sole difference with A2 is that A2 is to and from B&W only).

If what's to be displayed isn't already B&W, you can enforce that via FORCE_MONOCHROME, or Y1, or ORDERED @ q1.

Things become dicey because of the flags bug, because if it trips, that means pixels will be unexpectedly left as-is.

On that note:

• I've definitely tripped the flags bug with DU + FORCE_MONOCHROME + INVERT. I think I've also tripped it with only DU + FORCE_MONOCHROME.
• I've never tripped it with any wfm + ORDERED @ qany n. Possibly not even with INVERT, although I'm not an INVERT user (it looks like crap). As mentioned above, using FORCE_MONOCHROME in these cases makes no sense, as it's at the head of the pipeline, effectively rendering dithering useless.
• You won't trip it with Y1/Y4/D8, as that's implemented outside the PxP, in plain C in the kernel.

[NiLuJe]

Okay, I just went over the FBInk docs again to describe each waveform mode's behavior properly.

And I also went over the Mk. 7 quirks again.

Turns out the weirdness with A2 can be pretty squarely laid at the feet of the flags bug, because otherwise, FORCE_MONOCHROME is enough to satisfy the to/from B&W constraints.

(That doesn't change my recommendations for Plato's "Bitonal" button on Mk. 7: either drop it or keep the latest patch that swapped it to DU + Y1).

[baskerville]

Okay, I just went over the FBInk docs again to describe each waveform mode's behavior properly.

And I also went over the Mk. 7 quirks again.

This is really great!

That doesn't change my recommendations for Plato's "Bitonal" button on Mk. 7: either drop it or keep the latest patch that swapped it to DU + Y1.

I've chosen to remove the Make Bitonal menu entry on Mk. 7. The monochrome concept is still present though, because I need it for Sketch.

[NiLuJe]

I don't recall any issues with Sketch on the Forma, FWIW (in fact, I seem to remember it working much better than on my H2O ;)).

[CptPotato]

Seems to work perfectly for me. No more need to switch back and forth between Nickel and Plato, thanks a bunch!