SMuFL font glyph fall-back problem

The new SMuFL code seems to be not fully implementing font overlays (meaning that if a font does not have a particular glyph, a fallback glyph from another font is used as a substitute). The problem seems to be happening at the last stage where glyphs are inserted into the SVG <defs> element. In the tests below, notice that the glyph references are being used in the Leland output, but the glyphs for clef and mensuration are not being added to the SVG <defs> element.

The Leland font should have these glyphs (at least the clef), but perhaps the verovio does not include all of Leland? Rendering the same data in Bravura and Leipzig show the clef and mensuration as demonstrated below. The font metrics for the missing glyphs seem to be known by verovio, because in the Leland example, space is left for the missing glyphs. SVG code is added after each rendering for examination.

This problem started very recently (which is why I suspect the new dynamic SMuFL font handling system that is being mentioned in the PRs).

Click to view MEI data used to generate the following renderings.

```xml </titleStmt> <pubStmt /> </fileDesc> <encodingDesc> <appInfo> <application isodate="2022-02-16T07:46:19" version="3.9.0-dev-c21182d"> <name>Verovio</name> <p>Transcoded from Humdrum</p> </application> </appInfo> </encodingDesc> <workList> <work> <title /> </work> </workList> </meiHead> <music> <body> <mdiv xml:id="mgr2iog"> <score xml:id="sedtbb"> <scoreDef xml:id="scmhk1d" midi.bpm="400.000000"> <staffGrp xml:id="sypw0qe"> <staffDef xml:id="sqvhagx" n="1" notationtype="mensural.white" lines="5"> <clef xml:id="clef-L2F1" shape="C" line="3" /> </staffDef> </staffGrp> </scoreDef> <section xml:id="section-L1F1"> <staff xml:id="sot9n88" n="1"> <layer xml:id="layer-L1F1N1" n="1"> <mensur xml:id="mensur-L4F1" prolatio="2" tempus="2" sign="C" slash="1" /> <note xml:id="note-L6F1" dur="semibrevis" oct="4" pname="c" /> <note xml:id="note-L7F1" dur="semibrevis" oct="4" pname="d" /> </layer> </staff> </section> </score> </mdiv> </body> </music> </mei> ``` </details> <pre><code class="language-bash">verovio test.mei --font Leland --adjust-page-height --xml-id-seed 12</code></pre> <img width="485" alt="Screen Shot 2022-02-16 at 7 48 13 AM" src="https://user-images.githubusercontent.com/3487289/154303125-f1032e8a-8a27-490d-9519-10b8aa7d4987.png"> <p>The SVG output contains a <code>g@class="mensur"</code> entry, but there is no glyph in <code><defs></code> for Cut-C sign. Here is the mensur code in the following SVG output using the Leland font:</p> <pre><code class="language-xml"><g id="mensur-L4F1" class="mensur"> <use xlink:href="#E915-vwnew7" x="597" y="900" height="720px" width="720px" /> <use xlink:href="#E925-vwnew7" x="757" y="900" height="720px" width="720px" /> </g></code></pre> <p>Glyphs E915 and E925 are being referenced, but notice in the defs section, these glyphs are not included:</p> <pre><code class="language-xml"> <defs> <symbol id="E93C-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M330 -9l-154 -120c-2 -2 -5 -3 -8 -3c-4 0 -7 1 -9 3l-154 120c-3 2 -5 6 -5 9c0 4 2 7 5 9l154 121c2 2 5 3 8 3c4 0 7 -1 9 -3l154 -121c4 -2 5 -6 5 -10c0 -3 -2 -6 -5 -8zM198 -80l73 57l-133 104l-74 -58z" /> </symbol> </defs></code></pre> <details> <summary>Click to view SVG output with Leland font.</summary> ```xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <svg width="2100px" height="280px" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mei="http://www.music-encoding.org/ns/mei" overflow="visible"> <desc>Engraved by Verovio 3.9.0-dev-c21182d</desc> <defs> <symbol id="E93C-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M330 -9l-154 -120c-2 -2 -5 -3 -8 -3c-4 0 -7 1 -9 3l-154 120c-3 2 -5 6 -5 9c0 4 2 7 5 9l154 121c2 2 5 3 8 3c4 0 7 -1 9 -3l154 -121c4 -2 5 -6 5 -10c0 -3 -2 -6 -5 -8zM198 -80l73 57l-133 104l-74 -58z" /> </symbol> </defs> <style type="text/css">g.page-margin{font-family:Times;} g.ending, g.fing, g.reh, g.tempo{font-weight:bold;} g.dir, g.dynam, g.mNum{font-style:italic;} g.label{font-weight:normal;}</style> <svg class="definition-scale" color="black" viewBox="0 0 21000 2800"> <g class="page-margin" transform="translate(500, 500)"> <g id="mgr2iog" class="mdiv pageMilestone" /> <g id="sedtbb" class="score pageMilestone" /> <g id="scieszd" class="system"> <g id="section-L1F1" class="section systemMilestone" /> <g id="sot9n88" class="staff"> <path d="M0 540 L3981 540" stroke="currentColor" stroke-width="13" /> <path d="M0 720 L3981 720" stroke="currentColor" stroke-width="13" /> <path d="M0 900 L3981 900" stroke="currentColor" stroke-width="13" /> <path d="M0 1080 L3981 1080" stroke="currentColor" stroke-width="13" /> <path d="M0 1260 L3981 1260" stroke="currentColor" stroke-width="13" /> <g id="cz4k6di" class="clef"> <use xlink:href="#E909-vwnew7" x="90" y="900" height="720px" width="720px" /> </g> <g id="layer-L1F1N1" class="layer"> <g id="mensur-L4F1" class="mensur"> <use xlink:href="#E915-vwnew7" x="597" y="900" height="720px" width="720px" /> <use xlink:href="#E925-vwnew7" x="757" y="900" height="720px" width="720px" /> </g> <g id="note-L6F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="1101" y="900" height="720px" width="720px" /> </g> </g> <g id="note-L7F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="2541" y="810" height="720px" width="720px" /> </g> </g> </g> </g> <g id="sk88dic" class="systemMilestoneEnd section-L1F1" /> </g> <g id="ptl0jog" class="pageMilestoneEnd sedtbb" /> <g id="pxcblsf" class="pageMilestoneEnd mgr2iog" /> <g id="pl0h2ln" class="pgHead autogenerated" /> </g> </svg> </svg> ``` </details> <hr /> <pre><code class="language-bash">verovio test.mei --font Bravura --adjust-page-height --xml-id-seed 12</code></pre> <img width="466" alt="Screen Shot 2022-02-16 at 7 48 41 AM" src="https://user-images.githubusercontent.com/3487289/154303214-cd2b1a15-7ac6-4ae0-92f2-4c8be21ceb2a.png"> <details> <summary>Click to view SVG output with Bravura font.</summary> ```xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <svg width="2100px" height="280px" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mei="http://www.music-encoding.org/ns/mei" overflow="visible"> <desc>Engraved by Verovio 3.9.0-dev-89c053c-dirty</desc> <defs> <symbol id="E909-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M0 -783v1566c0 17 6 17 13 17h8c7 0 13 0 13 -17v-474c3 -26 15 -33 25 -33h121c25 0 28 19 30 37v227c0 17 6 17 13 17h9c6 0 12 0 12 -17v-1084c0 -17 -6 -17 -12 -17h-9c-7 0 -13 0 -13 17v237c-2 18 -5 38 -30 38h-116c-24 0 -28 -20 -30 -38v-476 c0 -17 -6 -17 -13 -17h-8c-7 0 -13 0 -13 17zM34 151v-36c2 -18 6 -37 30 -37h121c17 0 25 20 25 35v38c-6 20 -17 23 -27 23h-125c-7 0 -18 0 -24 -23zM34 -110v-22c2 -18 6 -37 30 -37h121c10 0 22 7 25 33v26c-1 18 -6 37 -27 37h-127c-17 -4 -21 -21 -22 -37z" /> </symbol> <symbol id="E915-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M0 0c0 146 120 265 266 265c89 0 172 -45 221 -120c3 -4 5 -9 5 -14c0 -8 -4 -15 -11 -19c-4 -3 -8 -4 -13 -4c-8 0 -16 4 -21 11c-40 61 -108 98 -181 98c-119 0 -218 -97 -218 -217s98 -218 218 -218c75 0 145 39 184 103c4 7 12 11 20 11c13 0 24 -12 24 -25 c0 -4 -1 -8 -3 -12c-50 -78 -132 -125 -225 -125c-147 0 -266 119 -266 266z" /> </symbol> <symbol id="E925-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M0 348c0 13 11 24 24 24c14 0 24 -11 24 -24v-695c0 -13 -10 -24 -24 -24c-13 0 -24 11 -24 24v695z" /> </symbol> <symbol id="E93C-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M0 0c0 2 1 4 2 7l152 174c2 2 4 3 8 4c3 0 6 -1 8 -4l148 -174c1 -2 2 -5 2 -7c0 -3 -1 -5 -2 -7l-148 -174c-2 -2 -5 -3 -8 -3s-6 1 -8 3l-152 174c-1 2 -2 3 -2 7zM140 118l-80 -91l125 -144l75 90z" /> </symbol> </defs> <style type="text/css">g.page-margin{font-family:Times;} g.ending, g.fing, g.reh, g.tempo{font-weight:bold;} g.dir, g.dynam, g.mNum{font-style:italic;} g.label{font-weight:normal;}</style> <svg class="definition-scale" color="black" viewBox="0 0 21000 2800"> <g class="page-margin" transform="translate(500, 500)"> <g id="mgr2iog" class="mdiv pageMilestone" /> <g id="sedtbb" class="score pageMilestone" /> <g id="scieszd" class="system"> <g id="section-L1F1" class="section systemMilestone" /> <g id="sot9n88" class="staff"> <path d="M0 540 L4040 540" stroke="currentColor" stroke-width="13" /> <path d="M0 720 L4040 720" stroke="currentColor" stroke-width="13" /> <path d="M0 900 L4040 900" stroke="currentColor" stroke-width="13" /> <path d="M0 1080 L4040 1080" stroke="currentColor" stroke-width="13" /> <path d="M0 1260 L4040 1260" stroke="currentColor" stroke-width="13" /> <g id="cz4k6di" class="clef"> <use xlink:href="#E909-vwnew7" x="90" y="900" height="720px" width="720px" /> </g> <g id="layer-L1F1N1" class="layer"> <g id="mensur-L4F1" class="mensur"> <use xlink:href="#E915-vwnew7" x="625" y="900" height="720px" width="720px" /> <use xlink:href="#E925-vwnew7" x="799" y="900" height="720px" width="720px" /> </g> <g id="note-L6F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="1160" y="900" height="720px" width="720px" /> </g> </g> <g id="note-L7F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="2600" y="810" height="720px" width="720px" /> </g> </g> </g> </g> <g id="sk88dic" class="systemMilestoneEnd section-L1F1" /> </g> <g id="ptl0jog" class="pageMilestoneEnd sedtbb" /> <g id="pxcblsf" class="pageMilestoneEnd mgr2iog" /> <g id="pl0h2ln" class="pgHead autogenerated" /> </g> </svg> </svg> ``` </details> <p>Notice that the <code><defs></code> section is filled in with more than one glyph now.</p> <hr /> <pre><code class="language-bash">verovio test.mei --font Leipzig --adjust-page-height --xml-id-seed 12</code></pre> <img width="459" alt="Screen Shot 2022-02-16 at 7 48 26 AM" src="https://user-images.githubusercontent.com/3487289/154303276-e7b5bc0f-e55d-4d3d-9f05-8b67ed7aab58.png"> <details> <summary>Click to view SVG output with Leipzig font.</summary> ```xml <?xml version="1.0" encoding="UTF-8" standalone="no"?> <svg width="2100px" height="280px" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mei="http://www.music-encoding.org/ns/mei" overflow="visible"> <desc>Engraved by Verovio 3.9.0-dev-89c053c-dirty</desc> <defs> <symbol id="E909-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M2 206l2 431c0 12 7 17 14 17c8 0 17 -7 17 -17l3 -204l2 -315h79h8c11 0 30 1 35 6c8 8 8 69 8 122v27l4 222s-1 20 19 20s12 -20 12 -20v-244s-1 -130 -1 -176c-1 -8 -1 -12 -4 -19c-7 -7 -8 -7 -16 -7c-45 0 -71 3 -71 3l-69 -3v-49v-48l64 -3l82 -5s5 0 8 -3 s5 -7 5 -7l1 -262v-154s0 -20 -13 -20c-10 0 -14 5 -16 14l-7 138l-2 219s0 4 -3 7s-9 2 -9 2h-40h-70l-3 -378l-3 -132s0 -20 -15 -20c-17 0 -19 17 -19 17c-1 22 -3 446 -3 686c0 49 1 100 1 155z" /> </symbol> <symbol id="E915-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M421 96c-46 77 -101 108 -181 108c-113 0 -204 -91 -204 -204s91 -204 204 -204c78 0 129 34 177 110c13 -8 15 -10 15 -10l-3 1s4 -2 21 -13c-41 -74 -120 -124 -210 -124c-132 0 -240 108 -240 240s108 240 240 240c89 0 163 -45 209 -122c-16 -13 -9 -8 -28 -22z" /> </symbol> <symbol id="E925-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M0 408h36v-816h-36v816z" /> </symbol> <symbol id="E93C-vwnew7" viewBox="0 0 1000 1000" overflow="inherit"> <path transform="scale(1,-1)" d="M155 194l157 -194l-157 -192l-155 192zM125 111l-57 -73l122 -149l56 71z" /> </symbol> </defs> <style type="text/css">g.page-margin{font-family:Times;} g.ending, g.fing, g.reh, g.tempo{font-weight:bold;} g.dir, g.dynam, g.mNum{font-style:italic;} g.label{font-weight:normal;}</style> <svg class="definition-scale" color="black" viewBox="0 0 21000 2800"> <g class="page-margin" transform="translate(500, 500)"> <g id="mgr2iog" class="mdiv pageMilestone" /> <g id="sedtbb" class="score pageMilestone" /> <g id="scieszd" class="system"> <g id="section-L1F1" class="section systemMilestone" /> <g id="sot9n88" class="staff"> <path d="M0 540 L3981 540" stroke="currentColor" stroke-width="13" /> <path d="M0 720 L3981 720" stroke="currentColor" stroke-width="13" /> <path d="M0 900 L3981 900" stroke="currentColor" stroke-width="13" /> <path d="M0 1080 L3981 1080" stroke="currentColor" stroke-width="13" /> <path d="M0 1260 L3981 1260" stroke="currentColor" stroke-width="13" /> <g id="cz4k6di" class="clef"> <use xlink:href="#E909-vwnew7" x="90" y="900" height="720px" width="720px" /> </g> <g id="layer-L1F1N1" class="layer"> <g id="mensur-L4F1" class="mensur"> <use xlink:href="#E915-vwnew7" x="597" y="900" height="720px" width="720px" /> <use xlink:href="#E925-vwnew7" x="757" y="900" height="720px" width="720px" /> </g> <g id="note-L6F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="1101" y="900" height="720px" width="720px" /> </g> </g> <g id="note-L7F1" class="note"> <g class="notehead"> <use xlink:href="#E93C-vwnew7" x="2541" y="810" height="720px" width="720px" /> </g> </g> </g> </g> <g id="sk88dic" class="systemMilestoneEnd section-L1F1" /> </g> <g id="ptl0jog" class="pageMilestoneEnd sedtbb" /> <g id="pxcblsf" class="pageMilestoneEnd mgr2iog" /> <g id="pl0h2ln" class="pgHead autogenerated" /> </g> </svg> </svg> ``` </details> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/lpugin"><img src="https://avatars.githubusercontent.com/u/689412?v=4" />lpugin</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Used to work in 3.8.1 <img src="https://user-images.githubusercontent.com/689412/154317436-fff908a7-9589-4895-bb18-f74c33427502.png" alt="image" /></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/craigsapp"><img src="https://avatars.githubusercontent.com/u/3487289?v=4" />craigsapp</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Notice that the clef glyph in 3.8.1 does seem to be the same as the Leipzig font glyphs, so it does seem likely to be related to font substitutions not being included in the SVG <code><defs></code>.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/lpugin"><img src="https://avatars.githubusercontent.com/u/689412?v=4" />lpugin</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>@craigsapp have a coffee first, it seems to be early for you 🤣</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/benory"><img src="https://avatars.githubusercontent.com/u/74871483?v=4" />benory</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>@craigsapp and @lpugin: thank you both so much for looking in to this! In case it is helpful, I can say that this bug/regression occurred between Sunday and Monday CET.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/craigsapp"><img src="https://avatars.githubusercontent.com/u/3487289?v=4" />craigsapp</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>I suspect that it is related to this PR: <a href="https://github.com/rism-digital/verovio/pull/2626">https://github.com/rism-digital/verovio/pull/2626</a></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/craigsapp"><img src="https://avatars.githubusercontent.com/u/3487289?v=4" />craigsapp</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>This line is a manifestation of the problem:</p> <p><a href="https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/svgdevicecontext.cpp#L130">https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/svgdevicecontext.cpp#L130</a></p> <p>When there is no glyph for a given font, there will be no contents at that path name, so it will not be inserted into the SVG <code><defs></code> section.</p> <p>the variable <code>m_smuflGlyphs</code> that is being iterated through around that line has information about all of the glyphs. For the test files there are four glyphs E909, E915, E925, E93C. The Leland font only has E93C available (in the most recent devel which I have double checked was installed on the system), so that is the only glyph being inserted into the SVG <code><defs></code> by the above code.</p> <p><code>m_smuflGlyphs</code> is being filled within the function <code>SvgDeviceContext::DrawMusicText()</code>:</p> <p><a href="https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/svgdevicecontext.cpp#L888-L933">https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/svgdevicecontext.cpp#L888-L933</a></p> <p>Line 903:</p> <pre><code class="language-cpp"> Glyph *glyph = Resources::GetGlyph(c); </code></pre> <p>Is loading the glyph information that is stored in <code>m_smuflGlyphs</code>. This reads the glyph from the variable <code>s_fontGlyphTable</code> which is filled in within the function <code>Resources::LoadFont()</code>:</p> <p><a href="https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/vrv.cpp#L163-L217">https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/vrv.cpp#L163-L217</a></p> <p>This function is in turn called from <code>Resources::InitFonts()</code>:</p> <p><a href="https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/vrv.cpp#L81-L113">https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/src/vrv.cpp#L81-L113</a></p> <p>And this function is where the font overlay system is being implemented: On line 85, the Bravura font is being loaded as the base font, then on line 87, the Leipzig font is being loaded as the backup base font.</p> <p>Then in <code>tools/main.cpp</code>, a final font will be loaded on top of these two base fonts:</p> <p><a href="https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/tools/main.cpp#L416-L427">https://github.com/rism-digital/verovio/blob/83a245e414d922e5777f1a1c08560b4e51822e9d/tools/main.cpp#L416-L427</a></p> <p>Perhaps for efficiency, there should be a check for if the font is being set to Leipzig, and if so, do not load it a second time since it is already loaded in the <code>InitFont()</code> function.</p> <hr /> <p>So when writing glyphs to the SVG <code><defs></code>: if there is no glyph for a given code point in the desired font, the fallback font of Leipzig should be checked next, and if no glyph in Leipzig, then check in the Bravura font for a last resort font substitution.</p> <p>Ideally the glyphs should be loaded when the fonts are loaded in main.cpp (<code>Resources::LoadFont</code>) and/or the source font should be recorded for each glyph in a parallel table to <code>s_fontGlyphTable</code> so that the svgdevicecontext.cpp code that writes the SVG <code><defs></code> does not need to do the same font overlay process (which needs to be coordinated with the code in tools/main.cpp if it changes).</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>

rism-digital / verovio

SMuFL font glyph fall-back problem #2652