add support for building zlib and libpng libraries in setup-build-freetype.py

[mammo0]

Because I renamed some branches my old PR #133 was closed by GitHub. So I reopen it here with the correct branch names.

To continue the discussion with @HinTak from christmas 2020:

If you bundle very common libraries like zlib and libpng you might need to look into the symbol export systems (not sure about it under cmake etc, but it is part of autoconf'ed build) not to re-export 3rd-party symbols, to avoid the Linux version of dll hell - where software using freetype and libpng get confused by the bundled libpng symbols within your build of freetype.

I checked again the sources to find out what symbols get exported. If I get it right the only file that creates the ftexport.sym file with the symbols is the Jamfile in the root directory of the freetype library: https://github.com/freetype/freetype/blob/132f19b779828b194b3fede187cee719785db4d8/Jamfile#L161-L193

And there you can see it only exports symbols from the header files in the include directory of the freetype library itself (https://github.com/freetype/freetype/tree/132f19b779828b194b3fede187cee719785db4d8/include). And this directory does not contain any external header files. So I think that wrongly exported symbols are not a problem here.

But you can put me right if I'm wrong here :)

[rougier]

I'm no sure to fully understand this PR but if @anthrotype approve, it's fine for me. @HinTak Can you have a quick look?

[mammo0]

@rougier: I ran into the problem which is described in #123. I wanted to use an emoji font with freetype. But to get this working libpng and zlib is required. So I extended the setup-build-freetype.py to build and embed those libraries.

[HinTak]

@mammo0 Hmm, I think building with Jam has been dropped years ago. For unix and mingw builds, the export symbol list is generated by this sub-makefile (if you build via ./configure , etc).

./builds/exports.mk

As for your problem /motivation for this pull - I actually do understand why you want to do this. Some emoji (I think you are talking about Apple Emoji specifically, actually!) fonts use png as the embedded bitmap format, so freetype needs to be linked with libpng to support that. As for libz, it is one of the specific compression schemes that libpng uses, so it comes as an indirect dependency at least because of needing libpng.

There are 4 kinds of emoji fonts, I think - one with png embedded (Apple's), svg (Adobe/Mozilla folks), CPAL (Microsoft), CBDT (Google...).

I think providing an option to statically build libpng/libz info freetype is a good idea. Hopefully it is done correctly.

[HinTak]

I am not familar with cmake enough to tell - but it is probably simple enough to just analyse the outcome - if it loads the apple emoji fonts, and does not show the png/zlib symbols under nm (on linux/mac) PE export table (for windows) , that would be fine.

[mammo0]

@mammo0 Hmm, I think building with Jam has been dropped years ago. For unix and mingw builds, the export symbol list is generated by this sub-makefile (if you build via ./configure , etc).

./builds/exports.mk

Ok, back in 2020, when I made the original PR, you used freetype version 2.10.2. That version still used the Jamfile. Now you've switched to version 2.11.0 (current master). That version uses the exports.mk file. I will update this PR to the current master and post the output of nm here.

As for your problem /motivation for this pull - I actually do understand why you want to do this. Some emoji (I think you are talking about Apple Emoji specifically, actually!) fonts use png as the embedded bitmap format, so freetype needs to be linked with libpng to support that. As for libz, it is one of the specific compression schemes that libpng uses, so it comes as an indirect dependency at least because of needing libpng.

Yes, you're absolutely right about this.

I think providing an option to statically build libpng/libz info freetype is a good idea. Hopefully it is done correctly.

Thanks for supporting my idea. As I mentioned above, I will rework my PR and post a list of the exported symbols.

[mammo0]

Ok, now I've rebased this PR to the current master branch.

@HinTak regarding the problem with the re-export of symbols from included libraries: I found out that this problem was already there with the Harfbuzz library. If I execute nm -g libfreetype.so on the library that was built from the current master (without any changes of this PR), then all Harfbuzz symbols were re-exported.

But I think I found a solution to not re-export the symbols of the embedded libraries. On Linux and Windows the linker flag --exclude-libs and on OSX -hidden-lx can be used. Please have a look at the last commit in this PR that uses those linker flags. Afterwards the output of nm -g libfreetype.so is:

Click to expand

``` U abort@GLIBC_2.2.5 000000000008b8b0 T adler32 000000000008b8c0 T adler32_combine 000000000008b9b0 T adler32_combine64 000000000008b440 T adler32_z U __assert_fail@GLIBC_2.2.5 U calloc@GLIBC_2.2.5 U close@GLIBC_2.2.5 000000000008bf80 T crc32 000000000008bf90 T crc32_combine 000000000008bfa0 T crc32_combine64 000000000008bcb0 T crc32_z U __cxa_atexit@GLIBC_2.2.5 w __cxa_finalize@GLIBC_2.2.5 00000000000afbe0 T deflate 00000000000afad0 T deflateBound 00000000000b12c0 T deflateCopy 00000000001626c0 R deflate_copyright 00000000000b0ee0 T deflateEnd 00000000000af6a0 T deflateGetDictionary 00000000000b1290 T deflateInit_ 00000000000b0f90 T deflateInit2_ 00000000000b0d10 T deflateParams 00000000000af950 T deflatePending 00000000000af9a0 T deflatePrime 00000000000af830 T deflateReset 00000000000af730 T deflateResetKeep 00000000000af470 T deflateSetDictionary 00000000000af910 T deflateSetHeader 00000000000afa70 T deflateTune 0000000000162920 R _dist_code U __errno_location@GLIBC_2.2.5 U fclose@GLIBC_2.2.5 U fcntl@GLIBC_2.2.5 U feof@GLIBC_2.2.5 U ferror@GLIBC_2.2.5 U fflush@GLIBC_2.2.5 U fopen@GLIBC_2.2.5 U fprintf@GLIBC_2.2.5 U fputc@GLIBC_2.2.5 U fread@GLIBC_2.2.5 U free@GLIBC_2.2.5 U frexp@GLIBC_2.2.5 U fstat@GLIBC_2.33 000000000001c770 T FT_Activate_Size 0000000000027c70 T FT_Add_Default_Modules 0000000000024cc0 T FT_Add_Module 000000000001ffc0 T FT_Angle_Diff 000000000001f950 T FT_Atan2 00000000000202e0 T FT_Attach_File 00000000000201e0 T FT_Attach_Stream 0000000000026720 T FT_Bitmap_Blend 0000000000025a90 T FT_Bitmap_Convert 0000000000025890 T FT_Bitmap_Copy 0000000000026030 T FT_Bitmap_Done 0000000000026080 T FT_Bitmap_Embolden 0000000000025850 T FT_Bitmap_Init 0000000000025870 T FT_Bitmap_New 0000000000033190 T FTC_CMapCache_Lookup 0000000000032da0 T FTC_CMapCache_New 0000000000019cb0 T FT_CeilFix 0000000000031ea0 T FTC_ImageCache_Lookup 00000000000321b0 T FTC_ImageCache_LookupScaler 0000000000031e90 T FTC_ImageCache_New 0000000000027c50 T FT_ClassicKern_Free 0000000000027bd0 T FT_ClassicKern_Validate 00000000000336d0 T FTC_Manager_Done 0000000000033040 T FTC_Manager_LookupFace 0000000000032db0 T FTC_Manager_LookupSize 0000000000033520 T FTC_Manager_New 00000000000338d0 T FTC_Manager_RemoveFaceID 00000000000337f0 T FTC_Manager_Reset 0000000000033b20 T FTC_Node_Unref 000000000001fa50 T FT_Cos 0000000000032590 T FTC_SBitCache_Lookup 0000000000032940 T FTC_SBitCache_LookupScaler 0000000000032580 T FTC_SBitCache_New 0000000000019e40 T FT_DivFix 0000000000023190 T FT_Done_Face 0000000000027ed0 T FT_Done_FreeType 0000000000027640 T FT_Done_Glyph 0000000000024fc0 T FT_Done_Library 0000000000028260 T FT_Done_MM_Var 0000000000022d90 T FT_Done_Size 000000000001a750 T FT_Error_String 0000000000028b10 T FT_Face_CheckTrueTypePatents 000000000001c5a0 T FT_Face_GetCharsOfVariant 000000000001c6f0 T FT_Face_GetCharVariantIndex 000000000001c6a0 T FT_Face_GetCharVariantIsDefault 000000000001c660 T FT_Face_GetVariantSelectors 000000000001c600 T FT_Face_GetVariantsOfChar 000000000001bed0 T FT_Face_Properties 0000000000028b20 T FT_Face_SetUnpatentedHinting 0000000000019cc0 T FT_FloorFix 0000000000021380 T FT_Get_Advance 00000000000211a0 T FT_Get_Advances 0000000000025700 T FT_Get_BDF_Charset_ID 00000000000257d0 T FT_Get_BDF_Property 000000000001bd60 T FT_Get_Char_Index 000000000001bd10 T FT_Get_Charmap_Index 0000000000026ff0 T FT_Get_CID_From_Glyph_Index 0000000000026f60 T FT_Get_CID_Is_Internally_CID_Keyed 0000000000026e80 T FT_Get_CID_Registry_Ordering_Supplement 000000000001c420 T FT_Get_CMap_Format 000000000001c3a0 T FT_Get_CMap_Language_ID 000000000001cbf0 T FT_Get_Color_Glyph_Layer 000000000001cc30 T FT_Get_Color_Glyph_Paint 000000000001cce0 T FT_Get_Colorline_Stops 000000000001be40 T FT_Get_First_Char 000000000001a760 T FT_Get_Font_Format 0000000000027090 T FT_Get_FSType_Flags 0000000000027150 T FT_Get_Gasp 0000000000027780 T FT_Get_Glyph 000000000001c060 T FT_Get_Glyph_Name 000000000001bab0 T FT_Get_Kerning 0000000000028790 T FT_Get_MM_Blend_Coordinates 00000000000281c0 T FT_Get_MM_Var 00000000000283f0 T FT_Get_MM_WeightVector 000000000001c850 T FT_Get_Module 0000000000028120 T FT_Get_Multi_Master 000000000001bfa0 T FT_Get_Name_Index 000000000001bd90 T FT_Get_Next_Char 000000000001ccb0 T FT_Get_Paint 000000000001cc70 T FT_Get_Paint_Layers 0000000000028d20 T FT_Get_PFR_Advance 0000000000028c80 T FT_Get_PFR_Kerning 0000000000028bc0 T FT_Get_PFR_Metrics 000000000001c160 T FT_Get_Postscript_Name 000000000002be30 T FT_Get_PS_Font_Info 000000000002bef0 T FT_Get_PS_Font_Private 000000000002bf60 T FT_Get_PS_Font_Value 000000000001c810 T FT_Get_Renderer 0000000000022800 T FT_Get_Sfnt_LangTag 0000000000022650 T FT_Get_Sfnt_Name 000000000001e6f0 T FT_Get_Sfnt_Name_Count 000000000001c200 T FT_Get_Sfnt_Table 000000000001cb60 T FT_Get_SubGlyph_Info 000000000001bbf0 T FT_Get_Track_Kerning 000000000001b220 T FT_Get_Transform 000000000001cb00 T FT_Get_TrueType_Engine_Type 00000000000288f0 T FT_Get_Var_Axis_Flags 0000000000028840 T FT_Get_Var_Blend_Coordinates 00000000000285d0 T FT_Get_Var_Design_Coordinates 000000000002bfe0 T FT_Get_WinFNT_Header 000000000001a790 T FT_Get_X11_Font_Format 0000000000027690 T FT_Glyph_Copy 0000000000027580 T FT_Glyph_Get_CBox 000000000002bcd0 T FT_GlyphSlot_Embolden 000000000002bc50 T FT_GlyphSlot_Oblique 0000000000025f80 T FT_GlyphSlot_Own_Bitmap 000000000002b970 T FT_Glyph_Stroke 000000000002bac0 T FT_Glyph_StrokeBorder 0000000000027880 T FT_Glyph_To_Bitmap 0000000000027520 T FT_Glyph_Transform 000000000003dd20 T FT_Gzip_Uncompress 000000000002bea0 T FT_Has_PS_Glyph_Names 0000000000027e70 T FT_Init_FreeType 000000000001ade0 T FT_Library_SetLcdFilter 000000000001add0 T FT_Library_SetLcdFilterWeights 000000000001adf0 T FT_Library_SetLcdGeometry 000000000001ca90 T FT_Library_Version 0000000000022b20 T FT_List_Add 0000000000022ff0 T FT_List_Finalize 0000000000022af0 T FT_List_Find 0000000000022d20 T FT_List_Insert 0000000000022f90 T FT_List_Iterate 0000000000022d50 T FT_List_Remove 0000000000022e80 T FT_List_Up 00000000000214b0 T FT_Load_Char 0000000000020a80 T FT_Load_Glyph 000000000001c270 T FT_Load_Sfnt_Table 0000000000019f80 T FT_Matrix_Invert 0000000000019e90 T FT_Matrix_Multiply 0000000000019cd0 T FT_MulDiv 0000000000019d90 T FT_MulFix 0000000000023bc0 T FT_New_Face 0000000000027420 T FT_New_Glyph 000000000001c9f0 T FT_New_Library 0000000000024a70 T FT_New_Memory_Face 0000000000022b60 T FT_New_Size 0000000000024ae0 T FT_Open_Face 0000000000028af0 T FT_OpenType_Free 0000000000028a10 T FT_OpenType_Validate 000000000001d230 T FT_Outline_Check 000000000001d2a0 T FT_Outline_Copy 000000000001cd20 T FT_Outline_Decompose 000000000001d370 T FT_Outline_Done 000000000001e2d0 T FT_Outline_Embolden 000000000001de00 T FT_Outline_EmboldenXY 00000000000254e0 T FT_Outline_Get_BBox 000000000001daa0 T FT_Outline_Get_Bitmap 000000000001d440 T FT_Outline_Get_CBox 000000000002ab10 T FT_Outline_GetInsideBorder 000000000001dbf0 T FT_Outline_Get_Orientation 000000000002ab30 T FT_Outline_GetOutsideBorder 0000000000021f30 T FT_Outline_New 000000000001d930 T FT_Outline_Render 000000000001d860 T FT_Outline_Reverse 000000000001db90 T FT_Outline_Transform 000000000001d830 T FT_Outline_Translate 000000000001a630 T FT_Palette_Data_Get 000000000001a6b0 T FT_Palette_Select 000000000001a720 T FT_Palette_Set_Foreground_Color 000000000001c9a0 T FT_Property_Get 000000000001c990 T FT_Property_Set 000000000001b250 T FT_Reference_Face 000000000001c9d0 T FT_Reference_Library 0000000000024af0 T FT_Remove_Module 0000000000020a40 T FT_Render_Glyph 000000000001b860 T FT_Request_Size 0000000000019c90 T FT_RoundFix 000000000001bc80 T FT_Select_Charmap 000000000001b7f0 T FT_Select_Size 000000000001c4a0 T FT_Set_Charmap 000000000001b970 T FT_Set_Char_Size 000000000001cad0 T FT_Set_Debug_Hook 0000000000087d30 T FT_Set_Default_Log_Handler 0000000000027cb0 T FT_Set_Default_Properties 0000000000087d20 T FT_Set_Log_Handler 0000000000028680 T FT_Set_MM_Blend_Coordinates 0000000000028290 T FT_Set_MM_Design_Coordinates 0000000000028340 T FT_Set_MM_WeightVector 0000000000028920 T FT_Set_Named_Instance 000000000001ba10 T FT_Set_Pixel_Sizes 0000000000022ed0 T FT_Set_Renderer 000000000001b170 T FT_Set_Transform 0000000000028780 T FT_Set_Var_Blend_Coordinates 00000000000284d0 T FT_Set_Var_Design_Coordinates 000000000001c300 T FT_Sfnt_Table_Info 000000000001fa90 T FT_Sin 00000000000306b0 T FT_Stream_OpenBzip2 000000000003d9a0 T FT_Stream_OpenGzip 000000000003e1a0 T FT_Stream_OpenLZW 000000000002afb0 T FT_Stroker_BeginSubPath 000000000002af50 T FT_Stroker_ConicTo 000000000002af80 T FT_Stroker_CubicTo 000000000002ace0 T FT_Stroker_Done 000000000002b010 T FT_Stroker_EndSubPath 000000000002b530 T FT_Stroker_Export 000000000002b400 T FT_Stroker_ExportBorder 000000000002b2b0 T FT_Stroker_GetBorderCounts 000000000002b350 T FT_Stroker_GetCounts 000000000002add0 T FT_Stroker_LineTo 000000000002ab50 T FT_Stroker_New 000000000002b560 T FT_Stroker_ParseOutline 000000000002ac60 T FT_Stroker_Rewind 000000000002aca0 T FT_Stroker_Set 000000000001f8f0 T FT_Tan 0000000000087d10 T FT_Trace_Set_Default_Level 0000000000087d00 T FT_Trace_Set_Level 0000000000027bb0 T FT_TrueTypeGX_Free 0000000000027b10 T FT_TrueTypeGX_Validate 000000000001ff90 T FT_Vector_From_Polar 000000000001fca0 T FT_Vector_Length 000000000001fe40 T FT_Vector_Polarize 000000000001fad0 T FT_Vector_Rotate 000000000001db10 T FT_Vector_Transform 000000000001fa00 T FT_Vector_Unit U fwrite@GLIBC_2.2.5 000000000008bca0 T get_crc_table U getenv@GLIBC_2.2.5 w __gmon_start__ U gmtime@GLIBC_2.2.5 0000000000088250 T inflate 000000000008a8a0 T inflateCodesUsed 000000000008a4b0 T inflateCopy 000000000015c380 R inflate_copyright 0000000000089f90 T inflateEnd 000000000008ae30 T inflate_fast 0000000000089ff0 T inflateGetDictionary 000000000008a150 T inflateGetHeader 00000000000881a0 T inflateInit_ 00000000000880b0 T inflateInit2_ 000000000008a810 T inflateMark 00000000000881b0 T inflatePrime 0000000000087fb0 T inflateReset 0000000000088010 T inflateReset2 0000000000087ec0 T inflateResetKeep 000000000008a0a0 T inflateSetDictionary 000000000008a1c0 T inflateSync 000000000008a450 T inflateSyncPoint 000000000008a910 T inflate_table 000000000008a740 T inflateUndermine 000000000008a7a0 T inflateValidate w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable 0000000000162820 R _length_code U longjmp@GLIBC_2.2.5 U malloc@GLIBC_2.2.5 U memchr@GLIBC_2.2.5 U memcmp@GLIBC_2.2.5 U memcpy@GLIBC_2.14 U memmove@GLIBC_2.2.5 U memset@GLIBC_2.2.5 U mmap@GLIBC_2.2.5 U modf@GLIBC_2.2.5 U mprotect@GLIBC_2.2.5 U munmap@GLIBC_2.2.5 U open@GLIBC_2.2.5 U pow@GLIBC_2.29 U qsort@GLIBC_2.2.5 U read@GLIBC_2.2.5 U realloc@GLIBC_2.2.5 U remove@GLIBC_2.2.5 U roundf@GLIBC_2.2.5 U _setjmp@GLIBC_2.2.5 U setlocale@GLIBC_2.2.5 U snprintf@GLIBC_2.2.5 U sprintf@GLIBC_2.2.5 U __stack_chk_fail@GLIBC_2.4 U stderr@GLIBC_2.2.5 U stpcpy@GLIBC_2.2.5 U strchr@GLIBC_2.2.5 U strcmp@GLIBC_2.2.5 U strcpy@GLIBC_2.2.5 U strerror@GLIBC_2.2.5 U strlen@GLIBC_2.2.5 U strncmp@GLIBC_2.2.5 U strncpy@GLIBC_2.2.5 U strrchr@GLIBC_2.2.5 U strstr@GLIBC_2.2.5 U strtod@GLIBC_2.2.5 U strtol@GLIBC_2.2.5 U strtoul@GLIBC_2.2.5 U sysconf@GLIBC_2.2.5 00000000000b2940 T _tr_align 00000000000b28b0 T _tr_flush_bits 00000000000b2a90 T _tr_flush_block 00000000000b26a0 T _tr_init 00000000000b27a0 T _tr_stored_block 00000000000b3160 T _tr_tally 000000000007bf40 T TT_New_Context 0000000000078d20 T TT_RunIns U vsnprintf@GLIBC_2.2.5 000000000008b420 T zcalloc 000000000008b430 T zcfree 00000000001b7740 D z_errmsg 000000000008b400 T zError 000000000008b3f0 T zlibCompileFlags 000000000008b3e0 T zlibVersion ```

There are no references to harfbuzz, libpng or zlib.

[HinTak]

adler32, deflate and zlib* are zlib (libz) symbols. I just remember zlib is treated slightly differently - freetype bundles its sources under src/gzip/ .

[HinTak]

I think I used this before -Wl,--exclude-libs,ALL on a different project.I am wondering about the syntax - I think you only need harfbuzz, z,png (the part without lib and .a), just like what you do with -lharfbuzz -lpng -lz in linking.

[mammo0]

Oh, you're right. Zlib is still there... Sorry about that, but it was a long debugging session yesterday.

At first I globally used -Wl,--exclude-libs,ALL, but that flat is not available on Mac OS. So I splitted it up to only exclude the bundled libraries.

I will try your suggested syntax tomorrow and report back.

Thanks for your patience.

[mammo0]

Ok, I already found some time today.

I tested your suggested syntax with e.g. -Wl,--exclude-libs,png. But this does not work. Afterwards the libpng symbols are re-exported. Only if I use -Wl,--exclude-libs,libpng it's working. I think for Linux and Windows they have to be named like the files of the corresponding library. Only without the .a or .so suffix.

The problem with the exported zlib symbols was because of a typo. The resulting library file was libz.a, but I excluded zlib. Sorry, my mistake. I corrected it and now it works:

nm -g libfreetype.so:

``` U abort@GLIBC_2.2.5 U __assert_fail@GLIBC_2.2.5 U calloc@GLIBC_2.2.5 U close@GLIBC_2.2.5 U __cxa_atexit@GLIBC_2.2.5 w __cxa_finalize@GLIBC_2.2.5 U __errno_location@GLIBC_2.2.5 U fclose@GLIBC_2.2.5 U fcntl@GLIBC_2.2.5 U feof@GLIBC_2.2.5 U ferror@GLIBC_2.2.5 U fflush@GLIBC_2.2.5 U fopen@GLIBC_2.2.5 U fprintf@GLIBC_2.2.5 U fputc@GLIBC_2.2.5 U fread@GLIBC_2.2.5 U free@GLIBC_2.2.5 U frexp@GLIBC_2.2.5 U fstat@GLIBC_2.33 000000000001b600 T FT_Activate_Size 0000000000026b00 T FT_Add_Default_Modules 0000000000023b50 T FT_Add_Module 000000000001ee50 T FT_Angle_Diff 000000000001e7e0 T FT_Atan2 000000000001f170 T FT_Attach_File 000000000001f070 T FT_Attach_Stream 00000000000255b0 T FT_Bitmap_Blend 0000000000024920 T FT_Bitmap_Convert 0000000000024720 T FT_Bitmap_Copy 0000000000024ec0 T FT_Bitmap_Done 0000000000024f10 T FT_Bitmap_Embolden 00000000000246e0 T FT_Bitmap_Init 0000000000024700 T FT_Bitmap_New 0000000000032020 T FTC_CMapCache_Lookup 0000000000031c30 T FTC_CMapCache_New 0000000000018b40 T FT_CeilFix 0000000000030d30 T FTC_ImageCache_Lookup 0000000000031040 T FTC_ImageCache_LookupScaler 0000000000030d20 T FTC_ImageCache_New 0000000000026ae0 T FT_ClassicKern_Free 0000000000026a60 T FT_ClassicKern_Validate 0000000000032560 T FTC_Manager_Done 0000000000031ed0 T FTC_Manager_LookupFace 0000000000031c40 T FTC_Manager_LookupSize 00000000000323b0 T FTC_Manager_New 0000000000032760 T FTC_Manager_RemoveFaceID 0000000000032680 T FTC_Manager_Reset 00000000000329b0 T FTC_Node_Unref 000000000001e8e0 T FT_Cos 0000000000031420 T FTC_SBitCache_Lookup 00000000000317d0 T FTC_SBitCache_LookupScaler 0000000000031410 T FTC_SBitCache_New 0000000000018cd0 T FT_DivFix 0000000000022020 T FT_Done_Face 0000000000026d60 T FT_Done_FreeType 00000000000264d0 T FT_Done_Glyph 0000000000023e50 T FT_Done_Library 00000000000270f0 T FT_Done_MM_Var 0000000000021c20 T FT_Done_Size 00000000000195e0 T FT_Error_String 00000000000279a0 T FT_Face_CheckTrueTypePatents 000000000001b430 T FT_Face_GetCharsOfVariant 000000000001b580 T FT_Face_GetCharVariantIndex 000000000001b530 T FT_Face_GetCharVariantIsDefault 000000000001b4f0 T FT_Face_GetVariantSelectors 000000000001b490 T FT_Face_GetVariantsOfChar 000000000001ad60 T FT_Face_Properties 00000000000279b0 T FT_Face_SetUnpatentedHinting 0000000000018b50 T FT_FloorFix 0000000000020210 T FT_Get_Advance 0000000000020030 T FT_Get_Advances 0000000000024590 T FT_Get_BDF_Charset_ID 0000000000024660 T FT_Get_BDF_Property 000000000001abf0 T FT_Get_Char_Index 000000000001aba0 T FT_Get_Charmap_Index 0000000000025e80 T FT_Get_CID_From_Glyph_Index 0000000000025df0 T FT_Get_CID_Is_Internally_CID_Keyed 0000000000025d10 T FT_Get_CID_Registry_Ordering_Supplement 000000000001b2b0 T FT_Get_CMap_Format 000000000001b230 T FT_Get_CMap_Language_ID 000000000001ba80 T FT_Get_Color_Glyph_Layer 000000000001bac0 T FT_Get_Color_Glyph_Paint 000000000001bb70 T FT_Get_Colorline_Stops 000000000001acd0 T FT_Get_First_Char 00000000000195f0 T FT_Get_Font_Format 0000000000025f20 T FT_Get_FSType_Flags 0000000000025fe0 T FT_Get_Gasp 0000000000026610 T FT_Get_Glyph 000000000001aef0 T FT_Get_Glyph_Name 000000000001a940 T FT_Get_Kerning 0000000000027620 T FT_Get_MM_Blend_Coordinates 0000000000027050 T FT_Get_MM_Var 0000000000027280 T FT_Get_MM_WeightVector 000000000001b6e0 T FT_Get_Module 0000000000026fb0 T FT_Get_Multi_Master 000000000001ae30 T FT_Get_Name_Index 000000000001ac20 T FT_Get_Next_Char 000000000001bb40 T FT_Get_Paint 000000000001bb00 T FT_Get_Paint_Layers 0000000000027bb0 T FT_Get_PFR_Advance 0000000000027b10 T FT_Get_PFR_Kerning 0000000000027a50 T FT_Get_PFR_Metrics 000000000001aff0 T FT_Get_Postscript_Name 000000000002acc0 T FT_Get_PS_Font_Info 000000000002ad80 T FT_Get_PS_Font_Private 000000000002adf0 T FT_Get_PS_Font_Value 000000000001b6a0 T FT_Get_Renderer 0000000000021690 T FT_Get_Sfnt_LangTag 00000000000214e0 T FT_Get_Sfnt_Name 000000000001d580 T FT_Get_Sfnt_Name_Count 000000000001b090 T FT_Get_Sfnt_Table 000000000001b9f0 T FT_Get_SubGlyph_Info 000000000001aa80 T FT_Get_Track_Kerning 000000000001a0b0 T FT_Get_Transform 000000000001b990 T FT_Get_TrueType_Engine_Type 0000000000027780 T FT_Get_Var_Axis_Flags 00000000000276d0 T FT_Get_Var_Blend_Coordinates 0000000000027460 T FT_Get_Var_Design_Coordinates 000000000002ae70 T FT_Get_WinFNT_Header 0000000000019620 T FT_Get_X11_Font_Format 0000000000026520 T FT_Glyph_Copy 0000000000026410 T FT_Glyph_Get_CBox 000000000002ab60 T FT_GlyphSlot_Embolden 000000000002aae0 T FT_GlyphSlot_Oblique 0000000000024e10 T FT_GlyphSlot_Own_Bitmap 000000000002a800 T FT_Glyph_Stroke 000000000002a950 T FT_Glyph_StrokeBorder 0000000000026710 T FT_Glyph_To_Bitmap 00000000000263b0 T FT_Glyph_Transform 000000000003cbb0 T FT_Gzip_Uncompress 000000000002ad30 T FT_Has_PS_Glyph_Names 0000000000026d00 T FT_Init_FreeType 0000000000019c70 T FT_Library_SetLcdFilter 0000000000019c60 T FT_Library_SetLcdFilterWeights 0000000000019c80 T FT_Library_SetLcdGeometry 000000000001b920 T FT_Library_Version 00000000000219b0 T FT_List_Add 0000000000021e80 T FT_List_Finalize 0000000000021980 T FT_List_Find 0000000000021bb0 T FT_List_Insert 0000000000021e20 T FT_List_Iterate 0000000000021be0 T FT_List_Remove 0000000000021d10 T FT_List_Up 0000000000020340 T FT_Load_Char 000000000001f910 T FT_Load_Glyph 000000000001b100 T FT_Load_Sfnt_Table 0000000000018e10 T FT_Matrix_Invert 0000000000018d20 T FT_Matrix_Multiply 0000000000018b60 T FT_MulDiv 0000000000018c20 T FT_MulFix 0000000000022a50 T FT_New_Face 00000000000262b0 T FT_New_Glyph 000000000001b880 T FT_New_Library 0000000000023900 T FT_New_Memory_Face 00000000000219f0 T FT_New_Size 0000000000023970 T FT_Open_Face 0000000000027980 T FT_OpenType_Free 00000000000278a0 T FT_OpenType_Validate 000000000001c0c0 T FT_Outline_Check 000000000001c130 T FT_Outline_Copy 000000000001bbb0 T FT_Outline_Decompose 000000000001c200 T FT_Outline_Done 000000000001d160 T FT_Outline_Embolden 000000000001cc90 T FT_Outline_EmboldenXY 0000000000024370 T FT_Outline_Get_BBox 000000000001c930 T FT_Outline_Get_Bitmap 000000000001c2d0 T FT_Outline_Get_CBox 00000000000299a0 T FT_Outline_GetInsideBorder 000000000001ca80 T FT_Outline_Get_Orientation 00000000000299c0 T FT_Outline_GetOutsideBorder 0000000000020dc0 T FT_Outline_New 000000000001c7c0 T FT_Outline_Render 000000000001c6f0 T FT_Outline_Reverse 000000000001ca20 T FT_Outline_Transform 000000000001c6c0 T FT_Outline_Translate 00000000000194c0 T FT_Palette_Data_Get 0000000000019540 T FT_Palette_Select 00000000000195b0 T FT_Palette_Set_Foreground_Color 000000000001b830 T FT_Property_Get 000000000001b820 T FT_Property_Set 000000000001a0e0 T FT_Reference_Face 000000000001b860 T FT_Reference_Library 0000000000023980 T FT_Remove_Module 000000000001f8d0 T FT_Render_Glyph 000000000001a6f0 T FT_Request_Size 0000000000018b20 T FT_RoundFix 000000000001ab10 T FT_Select_Charmap 000000000001a680 T FT_Select_Size 000000000001b330 T FT_Set_Charmap 000000000001a800 T FT_Set_Char_Size 000000000001b960 T FT_Set_Debug_Hook 0000000000086bc0 T FT_Set_Default_Log_Handler 0000000000026b40 T FT_Set_Default_Properties 0000000000086bb0 T FT_Set_Log_Handler 0000000000027510 T FT_Set_MM_Blend_Coordinates 0000000000027120 T FT_Set_MM_Design_Coordinates 00000000000271d0 T FT_Set_MM_WeightVector 00000000000277b0 T FT_Set_Named_Instance 000000000001a8a0 T FT_Set_Pixel_Sizes 0000000000021d60 T FT_Set_Renderer 000000000001a000 T FT_Set_Transform 0000000000027610 T FT_Set_Var_Blend_Coordinates 0000000000027360 T FT_Set_Var_Design_Coordinates 000000000001b190 T FT_Sfnt_Table_Info 000000000001e920 T FT_Sin 000000000002f540 T FT_Stream_OpenBzip2 000000000003c830 T FT_Stream_OpenGzip 000000000003d030 T FT_Stream_OpenLZW 0000000000029e40 T FT_Stroker_BeginSubPath 0000000000029de0 T FT_Stroker_ConicTo 0000000000029e10 T FT_Stroker_CubicTo 0000000000029b70 T FT_Stroker_Done 0000000000029ea0 T FT_Stroker_EndSubPath 000000000002a3c0 T FT_Stroker_Export 000000000002a290 T FT_Stroker_ExportBorder 000000000002a140 T FT_Stroker_GetBorderCounts 000000000002a1e0 T FT_Stroker_GetCounts 0000000000029c60 T FT_Stroker_LineTo 00000000000299e0 T FT_Stroker_New 000000000002a3f0 T FT_Stroker_ParseOutline 0000000000029af0 T FT_Stroker_Rewind 0000000000029b30 T FT_Stroker_Set 000000000001e780 T FT_Tan 0000000000086ba0 T FT_Trace_Set_Default_Level 0000000000086b90 T FT_Trace_Set_Level 0000000000026a40 T FT_TrueTypeGX_Free 00000000000269a0 T FT_TrueTypeGX_Validate 000000000001ee20 T FT_Vector_From_Polar 000000000001eb30 T FT_Vector_Length 000000000001ecd0 T FT_Vector_Polarize 000000000001e960 T FT_Vector_Rotate 000000000001c9a0 T FT_Vector_Transform 000000000001e890 T FT_Vector_Unit U fwrite@GLIBC_2.2.5 U getenv@GLIBC_2.2.5 w __gmon_start__ U gmtime@GLIBC_2.2.5 w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable U longjmp@GLIBC_2.2.5 U malloc@GLIBC_2.2.5 U memchr@GLIBC_2.2.5 U memcmp@GLIBC_2.2.5 U memcpy@GLIBC_2.14 U memmove@GLIBC_2.2.5 U memset@GLIBC_2.2.5 U mmap@GLIBC_2.2.5 U modf@GLIBC_2.2.5 U mprotect@GLIBC_2.2.5 U munmap@GLIBC_2.2.5 U open@GLIBC_2.2.5 U pow@GLIBC_2.29 U qsort@GLIBC_2.2.5 U read@GLIBC_2.2.5 U realloc@GLIBC_2.2.5 U remove@GLIBC_2.2.5 U roundf@GLIBC_2.2.5 U _setjmp@GLIBC_2.2.5 U setlocale@GLIBC_2.2.5 U snprintf@GLIBC_2.2.5 U sprintf@GLIBC_2.2.5 U __stack_chk_fail@GLIBC_2.4 U stderr@GLIBC_2.2.5 U stpcpy@GLIBC_2.2.5 U strchr@GLIBC_2.2.5 U strcmp@GLIBC_2.2.5 U strcpy@GLIBC_2.2.5 U strerror@GLIBC_2.2.5 U strlen@GLIBC_2.2.5 U strncmp@GLIBC_2.2.5 U strncpy@GLIBC_2.2.5 U strrchr@GLIBC_2.2.5 U strstr@GLIBC_2.2.5 U strtod@GLIBC_2.2.5 U strtol@GLIBC_2.2.5 U strtoul@GLIBC_2.2.5 U sysconf@GLIBC_2.2.5 000000000007add0 T TT_New_Context 0000000000077bb0 T TT_RunIns U vsnprintf@GLIBC_2.2.5 ```

[anthrotype]

maybe the ci should be set up to run this (at least on Linux), otherwise even if we merge this the code will get rotten pretty soon. A test loading a subset of an emoji font would also be nice to confirm it works at runtime as well. Potentially we could even include zlib and libpng by default in the distributed wheel packages.

[mammo0]

I agree, some tests are never a mistake. I can add a test. But before the CI integration should be done by a maintainer.

[rougier]

There are some conflicts but I'm not sure I can fix them.

[mammo0]

@rougier merged it ;)

[HinTak]

Just a historical note: the jam build mechanism is perhaps 20+ years old; ./configure && make slightly younger but not much. Yes, it looks like jam support was removed recently; and building with cmake and meson were added. And visual studio support perhaps 10 years old.

[madig]

I suppose this is ready to merge after someone added a test? I think we can just ship with zlib and png deps built into the bundle by default.

[anthrotype]

just ship with zlib and png deps built into the bundle by default

+1

[anthrotype]

I just tried to run this on my local macOS machine (M1 Pro MBP) and I got an error while it tries to build libpng, not sure what this is. In any case, it would be good to hook this up to the CI.

# Next, build libpng.
-- The C compiler identification is AppleClang 14.0.3.14030022
-- The ASM compiler identification is Clang with GNU-like command-line
-- Found assembler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found ZLIB: /Users/clupo/oss/freetype-py/build/local/lib/libz.a (found version "1.2.11")
-- Symbol prefix:
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_COLOR_MAKEFILE
    CMAKE_CXX_FLAGS

-- Build files have been written to: /Users/clupo/oss/freetype-py/build/libpng-1.6.37/build
[2/34] Generating scripts/symbols.out
FAILED: scripts/symbols.out /Users/clupo/oss/freetype-py/build/libpng-1.6.37/build/scripts/symbols.out
cd /Users/clupo/oss/freetype-py/build/libpng-1.6.37/build && /opt/homebrew/Cellar/cmake/3.22.2/bin/cmake -DINPUT=/Users/clupo/oss/freetype-py/build/libpng-1.6.37/scripts/symbols.c -DOUTPUT=/Users/clupo/oss/freetype-py/build/libpng-1.6.37/build/scripts/symbols.out -P /Users/clupo/oss/freetype-py/build/libpng-1.6.37/build/scripts/genout.cmake
clang: error: no such file or directory: 'arm64'
CMake Error at scripts/genout.cmake:78 (message):
  Failed to generate
  /Users/clupo/oss/freetype-py/build/libpng-1.6.37/build/scripts/symbols.out.tf1

[4/34] Generating scripts/pnglibconf.c
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/Users/clupo/oss/freetype-py/setup-build-freetype.py", line 255, in <module>
    shell("cmake --build . --config Release --target install --parallel", cwd=build_dir_lp)
  File "/Users/clupo/oss/freetype-py/setup-build-freetype.py", line 150, in shell
    subprocess.run(cmd, shell=True, check=True, cwd=cwd)
  File "/opt/homebrew/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake --build . --config Release --target install --parallel' returned non-zero exit status 1.

[mammo0]

@anthrotype I had a quick look at your build error. It seems to be a problem with the CMAKE variables that are set for the macOS platform:

if sys.platform == "darwin":
    print("# Making a 64 bit build.")
    CMAKE_GLOBAL_SWITCHES += (
        '-DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" '
        '-DCMAKE_OSX_DEPLOYMENT_TARGET="10.9" '
        '-DCMAKE_C_FLAGS="-O2" '
        '-DCMAKE_CXX_FLAGS="-O2" '
    )
    bitness = 64

    # the library path is needed for the '-hidden-lx' option to work
    CMAKE_PREVENT_REEXPORT += "-Wl,-L{} ".format(lib_dir)
    CMAKE_PREVENT_REEXPORT += "-Wl,-hidden-lharfbuzz "
    if BUILD_ZLIB or BUILD_LIBPNG:
        CMAKE_PREVENT_REEXPORT += "-Wl,-hidden-lz "
    if BUILD_LIBPNG:
        CMAKE_PREVENT_REEXPORT += "-Wl,-hidden-lpng "

https://github.com/mammo0/freetype-py/blob/82676800f2272c8fdc8c13eef629a12b5e320620/setup-build-freetype.py#L90-L106

I have only an old Intel Mac, there these variables work and libpng compiles successfully. I'm sorry, but I can't find out which variables must be set for cmake to work on an M1/2 Mac.

[HinTak]

It is probably both of these:

    '-DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" '
    '-DCMAKE_OSX_DEPLOYMENT_TARGET="10.9" '

AFAIK, arm64 needs 11.x . 10.9 definitely does not support arm64; You need a much later 10.x (10.14/10.15?) for arm64 at least. 11.x is a safer bet.

[mammo0]

AFAIK, arm64 needs 11.x . 10.9 definitely does not support arm64

I thought also about this. But then why do the global variables

CMAKE_OSX_ARCHITECTURES="x86_64;arm64"
CMAKE_OSX_DEPLOYMENT_TARGET="10.9"

work for building the freetype and harfbuzz library on an M1 Mac? At least @anthrotype didn't report a build error of these libraries. And both get definitely built before libpng with the above variables set on MacOS (see lines 207 and 220 of the setup-build-freetype.py file).

[HinTak]

Targeting old os is less of a problem than building for a "future" arch. I suspect newer libpng (1.6.37) maybe dropping support for old mac os (10+ years old, 10.9?) too.

[madig]

If you build arm64 (stand-alone or fused to x64), I think you have to target macOS 11.0 anyway.

[mammo0]

I just tried to run this on my local macOS machine (M1 Pro MBP) and I got an error while it tries to build libpng

Today I hit the same error as @anthrotype. After some research I found the following issue: https://github.com/glennrp/libpng/issues/372

So, it's a problem on the side of libpng. As a temporary work around I use the following workaround: https://github.com/mammo0/freetype-py/commit/990de1e77d1e45a825dafabd914f13626bd50f04

I can make another PR with this change, but I don't know if a universal binary is desired?

[HinTak]

That depends - python (shipped by Apple or homebrew?) itself is a universal binary, right?

[mammo0]

itself is a universal binary, right?

On my Intel Mac the Python binary from Homebrew is only x86_64. Don't know what is it on a Silicon Mac... But I reworked my workaround from yesterday: https://github.com/mammo0/freetype-py/commit/14e29c4a50a4a9327aedbdabed99bd024b62f6e7 This time a universal library gets built. Like the others.

[anthrotype]

@mammo0 what's the status of this?

[anthrotype]

I just tried to build from this branch from my M1 Pro macbook (arm64 architecture) and it seems to have worked this time, unlike the last time I tried. Maybe it's just a matter of rebasing or master and perhaps adding a small test?

[anthrotype]

I also propose that, instead of building zlib and libpng only when FREETYPEPY_WITH_ZLIB and FREETYPEPY_WITH_LIBPNG are set, we do the opposite and build them always by default, unless one requests to build it without for whatever reason. This way the prebuilt wheels that we publish on PyPI will support loading color fonts out of the box.

[HinTak]

zlib is shipped with both Windows and Mac os x as standard as far as I know. (Besides Linux). It is really libpng which is not.

[HinTak]

There is also the opposite issue, where somebody installed a bundled build, which gradually getting out dated and buggy compared to the system zlib/libpng/freetype.

[HinTak]

That's the case with skia-python until a couple of months ago - it was bundling a copy of freetype about 5 years old, and still is a problem for a few libraries inherited from the wheel-building.

[HinTak]

See https://github.com/kyamagu/skia-python/issues/175 for example.

[anthrotype]

we can take care of keeping everything up to date just like we do for the embedded freetype and harbuzz, it's no big deal

[HinTak]

Until you no longer work for Google, you mean?

[HinTak]

There is the option of somebody else, such as you @anthrotype , maintaining such additional functionality elsewhere, for those who needs/wants this.

[HinTak]

In Sharpfont (the C# binding to freetype), https://github.com/Robmaister/SharpFont , the bundled libraries are maintained in a separate repo: https://github.com/Robmaister/SharpFont.Dependencies

[mammo0]

@mammo0 what's the status of this?

Currently, I use my forked version of freetype-py to build with libpng support. I tested this on my linux PC and on my Intel MacBook.

There is also the opposite issue, where somebody installed a bundled build, which gradually getting out dated and buggy compared to the system zlib/libpng/freetype.

Good point. But this is a strategic decision that should be discussed among the maintainers. I think, I can not help here.

[anthrotype]

@HinTak sorry, the "additional functionality" you are suggesting to maintain elsewhere, is it the bitmap color font support enabled by zlib+libpng that this PR adds, or is it the ability to pip install a precompiled freetype-py from PyPI with a bundled library that doesn't rely on system installed libraries?

Because in my view the latter has had great value in enabling more users, especially on non-linux, who might not know how to compile C/C++ from source, to use freetype-py in various Python applications. I understand that you are not interested in maintaining that piece of code, but there are others like me or @madig who can take care of occasionally making sure that the published freetype-py binary wheels are up to date.

And just like we keep freetype itself or harfbuzz updated, we can do the same for zlib or libpng, it's mostly a matter of changing the version number and the SHA256 hash (see #186), and from time to time updating cibuildwheel to fix various packaging bugs or support newer python versions.

[HinTak]

As I commented elsewhere, a line has to be drawn somewhere. Adding skia-python as a dependency to support OT-SVG color fonts (and COLRv1) is such a suggestion, for example.

[anthrotype]

skia is not a direct dependency of freetype, zlib and libpng are. If one wants to render OT-SVG or COLRv1 one would need more than just freetype. freetype-py only exposes a Python interface to the freetype API, anything more is out of scope here.

[HinTak]

Skia-python builds everything (including skia) bundled.

[HinTak]

I would probably suggest that when you no longer work for Google and @madig no longer contracts for Google, some of this stuff needs to be removed . Probably should file an issue to keep track of your employment status :-(

[HinTak]

Just FYI, I think there is an understanding between me and skia-python's owner, that if I stop having an interest in skia-python and the COLRv1-related code starts to bit-rot, it will be removed.

[HinTak]

And dependency on libpng is optional because there is only one common font that needs it: Apple's Color Emoji. So it is arguable whether libpng should be bundled for windows build.

[mammo0]

@HinTak

there is only one common font that needs it: Apple's Color Emoji.

That is not quite true. I'm using an unofficial ttf build of twemoji (https://git.sr.ht/~whynothugo/twemoji.ttf) which also uses png files.

@anthrotype

If one wants to render OT-SVG or COLRv1 one would need more than just freetype. freetype-py only exposes a Python interface to the freetype API, anything more is out of scope here.

This means, that freetype is generally not capable of rendering svg fonts?

[anthrotype]

It just parses the table and provides hooks for an external SVG library to actually parse and render the glyphs

[HinTak]

There are two examples in the example directory, one uses pycairo and the other uses skia-python, to enhance freetype-py to use OT-SVG fonts.

Twomoji.ttf also have a OT-SVG table. That's an argument to add skia-python as a dependency, actually. 😀

This is the official place for twemoji: https://github.com/twitter/twemoji

[mammo0]

There are two examples in the example directory, one uses pycairo and the other uses skia-python, to enhance freetype-py to use OT-SVG fonts.

Thank you for the hint :)

This is the official place for twemoji:

The work is currently continued here: https://github.com/jdecked/twemoji (see https://github.com/jdecked/twemoji/issues/10 for more information)