Try to add sub-pixel rendering

[kjk]

People occasionally ask for sub-pixel rendering.

I tried to add sub-pixel rendering by:

• defining FT_CONFIG_OPTION_SUBPIXEL_RENDERING in sumatrapdf_ftoption.h
• using FT_RENDER_MODE_LCD in mupdf calls to freetype's font rendering

However, on my test document there either was no change in text rendering or the change was too subtle for me to notice.

So this is just documenting my test for the future.

pdfium uses it (https://pdfium.googlesource.com/pdfium/+/06b087a3306a521db0d9d0a3e28c3ad0040c7055%5E%21/#F0) so maybe there's code to look at to figure out how they do it.

This is what I tried:

kjk@win7pro32 /cygdrive/c/kjk/src/sumatrapdf
$ git diff
diff --git a/ext/freetype2/config/sumatrapdf_ftoption.h b/ext/freetype2/config/sumatrapdf_ftoption.h
index 2ac2327..b2de079 100644
--- a/ext/freetype2/config/sumatrapdf_ftoption.h
+++ b/ext/freetype2/config/sumatrapdf_ftoption.h
@@ -21,6 +21,9 @@ FT_BEGIN_HEADER

 #define CFF_CONFIG_OPTION_OLD_ENGINE

+#undef FT_CONFIG_OPTION_SUBPIXEL_RENDERING
+#define FT_CONFIG_OPTION_SUBPIXEL_RENDERING
+
 #ifdef _DEBUG
 #define FT_DEBUG_LEVEL_ERROR
 #endif
diff --git a/mupdf/source/fitz/font.c b/mupdf/source/fitz/font.c
index bcb12ce..f1944d3 100755
--- a/mupdf/source/fitz/font.c
+++ b/mupdf/source/fitz/font.c
@@ -603,7 +603,7 @@ retry_unhinted:
                FT_Outline_Translate(&face->glyph->outline, -strength * 32, -strength * 32);
        }

-       fterr = FT_Render_Glyph(face->glyph, fz_aa_level(ctx) > 0 ? FT_RENDER_MODE_NORMAL : FT_RENDER_MODE_MONO);
+       fterr = FT_Render_Glyph(face->glyph, FT_RENDER_MODE_LCD);
        if (fterr)
        {
                fz_warn(ctx, "freetype render glyph (gid %d): %s", gid, ft_error_string(fterr));
@@ -756,7 +756,7 @@ do_render_ft_stroked_glyph(fz_context *ctx, fz_font *font, int gid, const fz_mat

        FT_Stroker_Done(stroker);

-       fterr = FT_Glyph_To_Bitmap(&glyph, fz_aa_level(ctx) > 0 ? FT_RENDER_MODE_NORMAL : FT_RENDER_MODE_MONO, 0, 1);
+       fterr = FT_Glyph_To_Bitmap(&glyph, FT_RENDER_MODE_LCD, 0, 1);
        if (fterr)
        {
                fz_warn(ctx, "FT_Glyph_To_Bitmap: %s", ft_error_string(fterr));

[debjan]

Hi,

any news on this?

I was going to ask about subpixel rendering of DjVu images, but as there seems to be a years long problem for subpixeling fonts, I'm guessing I wont see this feature for DjVu?

I wanted to ask because the "blurness" of grey antialias is more obvious in DjVu then in font rendered PDF.

[kjk]

Nothing has changed.

[tico-tico]

@kjk I think you didn't notice any change because you had forgotten to uncomment FT_USE_MODULE( FT_Renderer_Class, ft_smooth_lcd_renderer_class ) in sumatrapdf_ftmodule.h . I tried to uncomment but unfortunately it broke rendering completely :confused:.

[kekkc]

The difference with enabled sub-pixel rendering should be quite obvious: https://upload.wikimedia.org/wikipedia/commons/0/0d/Subpixel_demonstration_%28Quartz%29.png

Besides pdfium it seems that currently only Acrobat and Foxit are supporting sub-pixel rendering (by conincidence I compared exactly those 2 with Sumatra and wondered why the text appeared darker with a better contrast and sharper until I found this issue here ;))

Guess font rendering and readability is one of the most important points for viewers. Sumatra is blazingly fast, but this issue holds me back from using it constantly.

@tico-tico @kjk Was it possible to do some proof of concept, e.g. rendering a single letter or anything?

[char101]

I think freetype subpixel rendering is ignored by mupdf since mupdf does its own gray subpixel rendering. A possible aproach is to rescale the image after it has been rendered like what windjview-subpix or this patch does.

[mzso]

@kekkc commented on 2015. dec. 19. 13:22 CET:

The difference with enabled sub-pixel rendering should be quite obvious:

Only if you magnify a screenshot. Otherwise depending on circumstances and configuration it can be subtle.

Besides pdfium it seems that currently only Acrobat and Foxit are supporting sub-pixel rendering (by conincidence I compared exactly those 2 with Sumatra and wondered why the text appeared darker with a better contrast and sharper until I found this issue here ;))

Also There's PDF-Xchange Editor. Though unfortunately rendering can't be configured with either. With Firefox you can fine-tune rendering: https://addons.mozilla.org/firefox/addon/anti-aliasing-tuner/

Guess font rendering and readability is one of the most important points for viewers. Sumatra is blazingly fast, but this issue holds me back from using it constantly.

Because of these reasons I only use it as a quick preview tool. For actual reading I use one with sub-pixel AA. It's least strenuous to read stuff this way.

[avih]

Only if you magnify a screenshot. Otherwise depending on circumstances and configuration it can be subtle.

It is subtle, but it's visible also without magnification. Or else you wouldn't:

Because of these reasons I only use it as a quick preview tool. For actual reading I use one with sub-pixel AA. It's least strenuous to read stuff this way.

So for those who care - it's visible. Whether or not it's preferable over gray AA is a subjective thing, but IMO many find subpixel AA preferable, and I think the main OSs out there also default to subpixel AA (Windows, OS X and Ubuntu - which cares about font rendering).

[calexHG]

Of course it is most definitely noticeable. Unfortunately, despite growing OS support for it, HiDPI displays aren't really a thing yet on desktop Windows machines (at least not in my world). So this WILL be obvious when trying to view documents (text, not images that one commenter highlighted) at more natural (close to real life page sizes) zoom levels rather than blown-up-way-too-big-for-usability-but-finally-readable-since-it's-no-longer-suffering-from-poor-aliasing cropped zooms.

Everything @kekkc says is spot on about my experience and likely everyone else's (unless they are on, say, one of the finally available high DPI Windows tablets). Non-experimental and/or non-rich desktop users are out of luck in this way when using this otherwise great software.

Thanks for the list of similarly lean readers that do support sub-pixel AA, whoever that was. I haven't been reading much on the desktop lately but just tried recently (hence my subscription to this topic); I'll try switching to one of those readers.

On Wed, Apr 13, 2016, 10:00 AM avih notifications@github.com wrote:

Only if you magnify a screenshot. Otherwise depending on circumstances and configuration it can be subtle.

It is subtle, but it's visible also without magnification. Or else you wouldn't:

Because of these reasons I only use it as a quick preview tool. For actual reading I use one with sub-pixel AA. It's least strenuous to read stuff this way.

So for those who care - it's visible. Whether or not it's preferable over gray AA is a subjective thing, but IMO many find subpixel AA preferable, and I think the main OSs out there also default to subpixel AA (Windows, OS X and Ubuntu - which cares about font rendering).

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/sumatrapdfreader/sumatrapdf/issues/51#issuecomment-209456264

[jimcarst]

Are there plans to add sub-pixel rendering anytime soon? It would, as earlier, mentioned greatly improve legibility of text.

[mzso]

@jimcarst commented on 2021. jún. 11. 19:49 CEST:

Are there plans to add sub-pixel rendering anytime soon? It would, as earlier, mentioned greatly improve legibility of text.

Since it wasn't implemented all these years, I wouldn't expect it.

[kjk]

I don't plan to make extensive changes to mupdf so if mupdf does it, Sumatra will inherit it.

[char101]

This is my experiment to try rendering the font using lcd rendering in mupdf (in sumatrapdf source code). IMO the result doesn't seem to be any different with the gray antialiasing. Maybe because the rgb subpixel is not blended with the text color.

The first text is from gray antialiasing, the 2nd lcd antialiasing using the patch below, the 3rd text is from pdf-xchange subpixel antialiasing.

diff --git a/ext/freetype/include/freetype/config/ftoption.h b/ext/freetype/include/freetype/config/ftoption.h
index 097f19b8..e5de62e3 100644
--- a/ext/freetype/include/freetype/config/ftoption.h
+++ b/ext/freetype/include/freetype/config/ftoption.h
@@ -124,7 +124,7 @@ FT_BEGIN_HEADER
    * When this macro is not defined, FreeType offers alternative LCD
    * rendering technology that produces excellent output.
    */
-/* #define FT_CONFIG_OPTION_SUBPIXEL_RENDERING */
+#define FT_CONFIG_OPTION_SUBPIXEL_RENDERING

   /**************************************************************************
diff --git a/mupdf/scripts/freetype/slimftoptions.h b/mupdf/scripts/freetype/slimftoptions.h
index e0c67586..9e1166af 100644
--- a/mupdf/scripts/freetype/slimftoptions.h
+++ b/mupdf/scripts/freetype/slimftoptions.h
@@ -9,7 +9,7 @@
 #undef FT_CONFIG_OPTION_USE_HARFBUZZ
 #undef FT_CONFIG_OPTION_MAC_FONTS
 #undef FT_CONFIG_OPTION_INCREMENTAL
-#undef FT_CONFIG_OPTION_SUBPIXEL_RENDERING
+/* #undef FT_CONFIG_OPTION_SUBPIXEL_RENDERING */

 #undef TT_CONFIG_OPTION_EMBEDDED_BITMAPS
 #undef TT_CONFIG_OPTION_SFNT_NAMES
diff --git a/mupdf/source/fitz/font.c b/mupdf/source/fitz/font.c
index 002cd3d3..3116f002 100755
--- a/mupdf/source/fitz/font.c
+++ b/mupdf/source/fitz/font.c
@@ -874,6 +874,8 @@ glyph_from_ft_bitmap(fz_context *ctx, int left, int top, FT_Bitmap *bitmap)
    (void)Memento_label(bitmap->buffer, "ft_bitmap");
    if (bitmap->pixel_mode == FT_PIXEL_MODE_MONO)
        return fz_new_glyph_from_1bpp_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
+       else if (bitmap->pixel_mode == FT_PIXEL_MODE_LCD)
+       return fz_new_glyph_from_lcd_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
    else
        return fz_new_glyph_from_8bpp_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
 }
@@ -884,6 +886,8 @@ pixmap_from_ft_bitmap(fz_context *ctx, int left, int top, FT_Bitmap *bitmap)
    (void)Memento_label(bitmap->buffer, "ft_bitmap");
    if (bitmap->pixel_mode == FT_PIXEL_MODE_MONO)
        return fz_new_pixmap_from_1bpp_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
+       else if (bitmap->pixel_mode == FT_PIXEL_MODE_LCD)
+       return fz_new_pixmap_from_lcd_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
    else
        return fz_new_pixmap_from_8bpp_data(ctx, left, top - bitmap->rows, bitmap->width, bitmap->rows, bitmap->buffer + (bitmap->rows-1)*bitmap->pitch, -bitmap->pitch);
 }
@@ -969,7 +973,7 @@ retry_unhinted:
        FT_Outline_Translate(&face->glyph->outline, -strength * 32, -strength * 32);
    }

-   fterr = FT_Render_Glyph(face->glyph, aa > 0 ? FT_RENDER_MODE_NORMAL : FT_RENDER_MODE_MONO);
+   fterr = FT_Render_Glyph(face->glyph, aa > 0 ? FT_RENDER_MODE_LCD : FT_RENDER_MODE_MONO);
    if (fterr)
    {
        if (aa > 0)
@@ -1122,7 +1126,7 @@ do_render_ft_stroked_glyph(fz_context *ctx, fz_font *font, int gid, fz_matrix tr

    FT_Stroker_Done(stroker);

-   fterr = FT_Glyph_To_Bitmap(&glyph, aa > 0 ? FT_RENDER_MODE_NORMAL : FT_RENDER_MODE_MONO, 0, 1);
+   fterr = FT_Glyph_To_Bitmap(&glyph, aa > 0 ? FT_RENDER_MODE_LCD : FT_RENDER_MODE_MONO, 0, 1);
    if (fterr)
    {
        fz_warn(ctx, "FT_Glyph_To_Bitmap(): %s", ft_error_string(fterr));
diff --git a/mupdf/source/fitz/glyph-imp.h b/mupdf/source/fitz/glyph-imp.h
index cae85e50..e0af597b 100644
--- a/mupdf/source/fitz/glyph-imp.h
+++ b/mupdf/source/fitz/glyph-imp.h
@@ -62,6 +62,7 @@ fz_glyph *fz_new_glyph_from_pixmap(fz_context *ctx, fz_pixmap *pix);
    to allocate.
 */
 fz_glyph *fz_new_glyph_from_8bpp_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span);
+fz_glyph *fz_new_glyph_from_lcd_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span);

 /*
    Create a new glyph from 1bpp data
diff --git a/mupdf/source/fitz/glyph.c b/mupdf/source/fitz/glyph.c
index 4d1a31e1..e2915e7e 100644
--- a/mupdf/source/fitz/glyph.c
+++ b/mupdf/source/fitz/glyph.c
@@ -313,6 +313,38 @@ try_pixmap:
    return glyph;
 }

+fz_glyph *
+fz_new_glyph_from_lcd_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span)
+{
+   fz_glyph *glyph = NULL;
+   fz_pixmap *pix = NULL;
+   unsigned char *orig_sp = sp;
+
+   fz_var(glyph);
+   fz_var(pix);
+
+   fz_try(ctx)
+   {
+       glyph = Memento_label(fz_realloc(ctx, glyph, sizeof(fz_glyph)), "fz_glyph(8r)");
+       FZ_INIT_STORABLE(glyph, 1, fz_drop_glyph_imp);
+       pix = fz_new_pixmap_from_lcd_data(ctx, x, y, w, h, orig_sp, span);
+       glyph->x = pix->x;
+       glyph->y = pix->y;
+       glyph->w = pix->w;
+       glyph->h = pix->h;
+       glyph->size = fz_pixmap_size(ctx, pix);
+       glyph->pixmap = pix;
+   }
+   fz_catch(ctx)
+   {
+       fz_drop_pixmap(ctx, pix);
+       fz_free(ctx, glyph);
+       fz_rethrow(ctx);
+   }
+
+   return glyph;
+}
+
 fz_glyph *
 fz_new_glyph_from_1bpp_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span)
 {
diff --git a/mupdf/source/fitz/pixmap-imp.h b/mupdf/source/fitz/pixmap-imp.h
index ff6ba405..a8fbceb5 100644
--- a/mupdf/source/fitz/pixmap-imp.h
+++ b/mupdf/source/fitz/pixmap-imp.h
@@ -24,6 +24,7 @@ void fz_decode_indexed_tile(fz_context *ctx, fz_pixmap *pix, const float *decode
 void fz_unpack_tile(fz_context *ctx, fz_pixmap *dst, unsigned char *src, int n, int depth, size_t stride, int scale);

 fz_pixmap *fz_new_pixmap_from_8bpp_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span);
+fz_pixmap *fz_new_pixmap_from_lcd_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span);
 fz_pixmap *fz_new_pixmap_from_1bpp_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span);

 #ifdef HAVE_VALGRIND
diff --git a/mupdf/source/fitz/pixmap.c b/mupdf/source/fitz/pixmap.c
index ba14b68d..d87aa4eb 100644
--- a/mupdf/source/fitz/pixmap.c
+++ b/mupdf/source/fitz/pixmap.c
@@ -1045,6 +1045,27 @@ fz_new_pixmap_from_8bpp_data(fz_context *ctx, int x, int y, int w, int h, unsign
    return pixmap;
 }

+fz_pixmap *
+fz_new_pixmap_from_lcd_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span)
+{
+   fz_pixmap *pixmap = fz_new_pixmap(ctx, ctx->colorspace->rgb, w / 3, h, NULL, 0);
+   int stride = pixmap->stride;
+   unsigned char *s = pixmap->samples;
+   pixmap->x = x;
+   pixmap->y = y;
+
+   for (y = 0; y < h; y++)
+   {
+       memcpy(s, sp + y * span, w);
+       s += stride;
+   }
+
+   fz_invert_pixmap(ctx, pixmap);
+
+   return pixmap;
+}
+
 fz_pixmap *
 fz_new_pixmap_from_1bpp_data(fz_context *ctx, int x, int y, int w, int h, unsigned char *sp, int span)
 {

[GitHubRulesOK]

@char101 thanks for trying to test font sub pixel, but I would be interested in comparing how MuPDF allows users setting 10 AA values and yet another for vector boundary AA but SumatraPDF uses fixed values. Does that setting make any significant difference as one appears to be the cause of some filled boundary issues where 2 user values could close them out. In some of my raw tests with mutool/mupdf-gl.exe I saw counter benefit but better with other inputs, so accept there are edge cases that don't work well with a median setting. You cant please all ....

[mzso]

@char101 commented on 2021. jún. 13. 11:30 CEST:

This is my experiment to try rendering the font using lcd rendering in mupdf (in sumatrapdf source code). IMO the result doesn't seem to be any different with the gray antialiasing.

What? How is that not different? It's massively different, even if not in a good way.

[char101]

Adding some note, freetype FT_PIXEL_MODE_LCD bitmap contains alpha in the three channels (r, g, b) while mupdf pixmap seems to contain one alpha channel max, so if we want font color to be workking, it seems that we need to blend the color ourselves. Unfortunately I can't find where the color blending happens.

Also mupdf implements pixmap compression for large glyph in the grayscale mode, something which I can't implement for the rgb subpixel version.

[GitHubRulesOK]

Very old comparison from 2010 placed SumatraPDF 2nd with Acrobat naturally first but then I am guessing they optimise Subpixel hinting and Cleartype to the nth and also have multiple settings for tweaking dissimilar objects. Slightly newer SuperUser subjective comparison agreed Team Acrobat solution was marginally better with Foxit SubPixel getting mixed reviews. My own tests suggested for artwork that SumatraPDF was up there with Acrobat when its using the same settings. But Graphics system and scaling can also have a bearing.

[VsevolodGolovanov]

I don't plan to make extensive changes to mupdf so if mupdf does it, Sumatra will inherit it.

MuPDF's latest stance as far as I can see it in https://bugs.ghostscript.com/show_bug.cgi?id=696819 is:

In order to support subpixel rendering, we need to know the layout of the sub-pixels on the screen.

Most computer monitors are RGBRGBRGB, though some are BGRBGRBGR.

Many devices have more complex layouts (with certain subpixels shared between pixels). For handheld devices this is even harder to get right due to the need to support different rotations.

We therefore take the view that if you want subpixel rendering, you should update the transform on input to be 3 times as wide, render to a wider pixmap, and then filter it down according to your own specific devices screen.

The core is entirely capable of doing this with no extra changes required on our part.

So they consider the rendering itself sufficiently supported already. I think the implication here is that detecting device subpixel layout is considered out of scope for MuPDF, that's why the issue is closed as WONTFIX. If this detection is a blocker for Sumatra too then perhaps the feature could be implemented as an optional feature at first, turned off by default, with a manual selection of the subpixel layout required to turn it on? That would probably be good enough for some people.