Closed asinghvi17 closed 2 years ago
I guess this is basically what I wanted to do. The framework for the Cairo special case is there but commented out, since I tried to use Cairo to draw colored emojis and it didn't (even though the docs explicitly list colored emojis as being supported). We could still draw the SVGs using Rsvg there though.
I think direct rendering of emojis will also require support for variation sequences from FreeTypeAbstraction.jl
to extract the code point from the font files. Without that, you don't even get passed layout_formatted_text
.
Also: The font not working could be related to the note made on the release page of OpenMoji, but not 100% sure: https://github.com/hfg-gmuend/openmoji/tree/master/font
:warning: The OpenMoji fonts work in all operating systems, but will currently only show color glyphs in Mozilla Firefox and Adobe CC. This is not a limitation of the generated fonts, but of the operating systems and applications.
Don't quote me on this, but I think I remember reading somewhere that unicode standards can have different implementations and that Firefox's and Adobe's implementations did have something in common or so. Perhaps that is the reason ...
We could still draw the SVGs using Rsvg there though.
Why not use PNGs
instead? We already use those for GLMakie
rendering.
Don't quote me on this, but I think I remember reading somewhere that unicode standards can have different implementations and that Firefox's and Adobe's implementations did have something in common or so. Perhaps that is the reason ...
I think this is what I remembered:
SVG in Open Type is a standard by Adobe and Mozilla for color OpenType and Open Font Format fonts.
Taken from https://github.com/DeeDeeG/noto-color-emoji-font#examples
I spent some more time on trying to understand how unicode variation sequences work and how to possibly render emojis from a font file. Here is a brief summary:
FreeType
's variation sequence interface (https://freetype.org/freetype2/docs/reference/ft2-glyph_variants.html) as provided by FreeType.jl
works or at least returns no error codes when used with the Noto Emoji Font
https://github.com/googlefonts/noto-emoji (see example below). However, using it with the .ttf
files provided by OpenMoji
I could not query any variation selectors at all (FT_Face_GetVariantSelectors(face)
returns a null pointer).OpenMoji
files seem to be not working perfectly any app. I am still wondering how Firefox is able to render them, given that the ttf
files miss the variation selectors. But perhaps they can use the underlying tables directly, because they know the right glyph index already? Wild speculations here ...FreeType
supports rendering to bitmaps and SVGs, the latter requiring an extra dependency that takes care of the rendering itself. From what I read SVGs are superior to bitmaps because they allow arbitrary scaling. Not sure if possible and how much work it would be to set this up from within Julia using Rsvg
and FreeType
alone (one has to set some callbacks in FreeType
, see https://freetype.org/freetype2/docs/reference/ft2-svg_fonts.html)Here are my experiments with the FreeType
library. Mainly adding it here for my own reference. Interesting note: the germany flag below is automatically rendered, whereas in my editor I entered it through the unicode sequence \u1f1e9\u1f1ea
.
using FreeTypeAbstraction
using FreeType
# Unicode variation sequence methods imported from `FreeType.jl`.
#
# function FT_Face_GetCharVariantIndex(face, charcode, variantSelector)
# ccall((:FT_Face_GetCharVariantIndex, libfreetype), FT_UInt, (FT_Face, FT_ULong, FT_ULong), face, charcode, variantSelector)
# end
#
# function FT_Face_GetCharVariantIsDefault(face, charcode, variantSelector)
# ccall((:FT_Face_GetCharVariantIsDefault, libfreetype), FT_Int, (FT_Face, FT_ULong, FT_ULong), face, charcode, variantSelector)
# end
#
# function FT_Face_GetVariantSelectors(face)
# ccall((:FT_Face_GetVariantSelectors, libfreetype), Ptr{FT_UInt32}, (FT_Face,), face)
# end
#
# function FT_Face_GetVariantsOfChar(face, charcode)
# ccall((:FT_Face_GetVariantsOfChar, libfreetype), Ptr{FT_UInt32}, (FT_Face, FT_ULong), face, charcode)
# end
#
# function FT_Face_GetCharsOfVariant(face, variantSelector)
# ccall((:FT_Face_GetCharsOfVariant, libfreetype), Ptr{FT_UInt32}, (FT_Face, FT_ULong), face, variantSelector)
# end
function get_variant_selectors(font::FTFont)
selectors = UInt32[]
ptr_selectors = FT_Face_GetVariantSelectors(font)
ptr_selectors == Ptr{UInt32}() && return selectors
idx = 1
v = unsafe_load(ptr_selectors, idx)
while v != 0
push!(selectors, v)
idx += 1
v = unsafe_load(ptr_selectors, idx)
end
return selectors
end
function get_charcodes_of_variant(font::FTFont, variantSelector::UInt32)
charcodes = UInt32[]
ptr_charcodes = FT_Face_GetCharsOfVariant(font, variantSelector)
ptr_charcodes == Ptr{UInt32}() && return charcodes
idx = 1
v = unsafe_load(ptr_charcodes, idx)
while v != 0
push!(charcodes, v)
idx += 1
v = unsafe_load(ptr_charcodes, idx)
end
return charcodes
end
function get_variantselectors_of_charcode(font::FTFont, charcode::UInt32)
selectors = UInt32[]
ptr_selectors = FT_Face_GetVariantsOfChar(font, charcode)
ptr_selectors == Ptr{UInt32}() && return selectors
idx = 1
v = unsafe_load(ptr_selectors, idx)
while v != 0
push!(selectors, v)
idx += 1
v = unsafe_load(ptr_selectors, idx)
end
return selectors
end
function get_charcode_variantselector_default(font::FTFont, charcode::UInt32, variantSelector::UInt32)
return FT_Face_GetCharVariantIsDefault(font, charcode, variantSelector)
end
function get_charcode_variantselector_index(font::FTFont, charcode::UInt32, variantSelector::UInt32)
code = FT_Face_GetCharVariantIndex(font, charcode, variantSelector)
return code == 0 ? nothing : code
end
# fontname = "/home/florian/.fonts/openmoji/OpenMoji-Black.ttf"
# fontname= "/home/florian/.fonts/openmoji/OpenMoji-Color.ttf"
# fontname= "/tmp/OpenMoji-Color.ttf"
fontname = "/home/florian/.fonts/NotoColorEmoji.ttf"
font = FTFont(fontname)
display(font)
selectors = get_variant_selectors(font)
# println(selectors)
charsofvariant = get_charcodes_of_variant.(Ref(font), selectors)
# println(length.(charsofvariant))
# println.(charsofvariant)
# variantsofchar = get_variantselectors_of_charcode.(Ref(font), first(charsofvariant))
### returns non-sense
# s = first(selectors)
# c = first(first(get_charcodes_of_variant.(Ref(font), selectors)))
emoji_flagde = "🇩🇪"
cp_emoji_flagde = [ convert(UInt32, codepoint(emoji_flagde[i])) for i in eachindex(emoji_flagde) ]
println(cp_emoji_flagde)
# println(cp_emoji_flagde[1] in first(charsofvariant))
# println(cp_emoji_flagde)
# println(s in selectors)
# println(get_charcode_variantselector_index.(Ref(font), cp_emoji_flagde, selectors[1]))
emoji_info = '\U2139'
cu_info = convert(UInt32, emoji_info)
# println(convert(UInt32,emoji_info))
println(cu_info in first(charsofvariant))
vs_glyphindex_info = get_charcode_variantselector_index(font, cu_info, first(selectors))
ft_face = getfield(font, :ft_ptr)
err = FreeType.FT_Load_Glyph(ft_face, vs_glyphindex_info, FreeType.FT_LOAD_NO_SCALE)
display(err)
# summary:
# it appears that glyph loading itself is not implemented in FreeTypeAbstraction,
# alhtough, this sounds wrong because how would people then load all the characters for
# text rendering? Only with FT_Load_Char?
# I think the problem is that the variation sequence methods of FreeType return
# a glyph index in the end, instead of a character code (makes sense since we started
# with a VS character code), but FT_Load_Char expects a character code.
Moving forward, we should settle on using PNG
s and/or SVG
s for now.
Ideally, I would like to only use one format at all, but from what I understand right now GLMakie
prefers PNG
s and CairoMakie
SVG
s, correct?
If it is possible to use PNG
s also with CairoMakie
, then we should do so for the moment. Reasons are that
1) frankly we have more important features to implement for a release (global slide elements, performance issues, etc.) and
2) we have PNG
s already working for GLMakie
.
I think that last commit might be unnecessary - CairoMakie should be able to render this as is without any external intervention. It's only if you want SVG output in vector graphics that you need this kind of overload.
Using only
function CairoMakie.draw_plot(scene::Scene, screen::CairoMakie.CairoScreen, fmttxt::T) where T <: FormattedText
txt = fmttxt.plots[1]
CairoMakie.draw_plot(scene, screen, txt)
if length(fmttxt.plots) > 1
scttr = fmttxt.plots[2]
CairoMakie.draw_plot(scene, screen, scttr)
end
end
gives me the following error:
I thought that was the problem you mentioned here https://github.com/fatteneder/MakieSlides.jl/pull/15#issuecomment-1137875735.
But now that you say it, it might be enough to just implement the draw_marker
method for Matrix{RGBAf}
overload.
Yeah that's what would have to happen. We could implement it here for now and upstream it to CairoMakie...see the implementation of the fast path in the heatmap draw_atomic for how you need to do it.
Got it to work.
Only thing is that I now convert the Matrix{RGBAf}
pixels first to Matrix{ARGB32}
and then create a CairoSurface
with it.
I tried to do avoid this by using Cairo.move_to
and Cairo.set_source_rgba
but could not get it work. Do you know of a more efficient solution?
AFAIK that solution is optimal since Cairo wants UInt32 color. It would have been converted either way.
Overall it looks good to me. Thanks for pushing it this far! I am on vacation now but feel free to upstream the method you implemented to CairoMakie.
Basically, this pr introduces an
emojifont
font in the opening to solve any potential bbox / char shape mismatch issues. All emojis are rendered with this font. It also changes a lot of the plot! signatures to convert_arguments which allows the user to update with any supported type.This is not yet complete, but I did this in the web editor so putting it out there.