search

javafxports / openjdk-jfx

The openjfx repo has moved to:
https://github.com/openjdk/jfx
GNU General Public License v2.0
1.01k stars 145 forks source link

JavaFX and MathML. #71

Closed scientificware closed 6 years ago

scientificware commented 6 years ago

The purpose of this issue is to repair MathML display in WebView.

When I discovered that JavaFX could manage MathML, I also discovered that it was buggy. Becauseof MathML is rendered with WebKit, I compared with other WebKit based web browsers. I also noticed that since 2016 that those web browser make good progress with MaltML display. First I thought that the differences were due to outdated WebKit version used by JavaFX but last releases, with an uptodate WebKit version, show that the problem is the same.

I contacted F. Wang from Igalia who worked to port or improve MathML support in FireFox and later in WebKit. He confirms that it was a bug in JavaFX not in WebKit witch displays MathML correctly.

As he suggested me, I wrote to the JavaFX mailing list and had a short discussion about this issue on OpenJDK.Java.net :

http://mail.openjdk.java.net/pipermail/openjfx-dev/2017-December/021112.html https://bugs.openjdk.java.net/browse/JDK-8147476

So, as suggested by mainteners, I'm trying to find by myself what's wrong with MathML in javaFX HTMLEditor.

I tested all applications with Mozilla MathML Torture Test (https://developer.mozilla.org/fr/docs/Mozilla/MathML_Project/MathML_Torture_Test)

Could someone help me ?

scientificware commented 6 years ago

I think that the problem is related only with the vertical size of each element : I notice that the size of the cursor is right outside the mathematics text but when we move it through the mathematics text, the cursor size is far much lower than text size. All horizontal sizes seem good, by example a \<mtext> followed by a \<mrow> in a element : screen_javafx_mathml_2

Which part of code deals with vertical size ?

arajkumar commented 6 years ago

Problem is current JavaFX font implementation for WebKit doesn't supports OpenMathData table.

Actually WebKit has it's own MathData table parser(OpenMathData.cpp) to extract MathML rendering parameters(like numeratorGap, denominatorGap,..etc), however it expects platform implementation(javafx) to return the OpenMathData table from OpenType font, which we don't support right now.

There is a fallback logic implemented when MathData is not available from platform implementation, however it seems to be buggy.

<math xmlns='http://www.w3.org/1998/Math/MathML'>
    <mrow>
    <mfrac>
        <mn>1</mn>
        <mn>2</mn>
    </mfrac>
    </mrow>
</math>

For e.g., to render the above simple mathml tocken, RenderMathMLFraction::fractionParameters(RenderMathMLFraction.cpp) method will be called to calculated the offsets. Since we don't support MathData table, the fallback logic to calculate the denominatorGapMin as follows,

        parameters.numeratorGapMin = display ? 3 * ruleThicknessFallback() : ruleThicknessFallback();
        parameters.denominatorGapMin = parameters.numeratorGapMin;

The above logic is not correct, because of that both numerator and denominator of a fraction is overlapping. Just adding a some offset to parameters.numeratorGapMin fixes the issue.

So the appropriate solution would be to implement a logic to extract a OpenMathData table from OpenType font table.

scientificware commented 6 years ago

Thank you very much for your help, and all these informations ! Thanks given me an entry point to investigate :-)

scientificware commented 6 years ago

First working comments :

scientificware commented 6 years ago

Configure my build and run environment to debug. Mageia 6, all the required development packages

Mageia package name Description  
lib64alsa2-devel Development files for Advanced Linux Sound Architecture (ALSA)  
lib64mesagl1-devel Development files for Mesa (OpenGL compatible 3D lib)  
lib64gstreamer1.0-devel Libraries and include files for GStreamer streaming-media framework  
lib64gstreamer-plugins-base1.0-devel GStreamer Plugin Library Headers​  
lib64jpeg-devel Development tools for programs which will use the libjpeg library  
lib64png-devel Development tools for programs to manipulate PNG image format files  
lib64x11-devel Development files for libx11  
lib64xml2-devel Libraries, includes, etc. to develop XML and HTML applications  
lib64xslt-devel Libraries, includes, etc. to develop XML and HTML applications  
lib64xt-devel Development files for libxt  
lib64xxf86vm-devel Development files for libxxf86vm  
pkgconfig Pkgconfig helps make building packages easier  
x11-proto-devel Xorg X11 protocol specification headers  
lib64ffmpeg-static-devel Static library for the ffmpeg codec library  
mercurial A fast, lightweight distributed source control management system
lib64gtk+2.0-devel Development files for GTK+ (GIMP ToolKit) applications  
lib64gtk+3.0-devel Development files for GTK+ (GIMP ToolKit) applications
lib64xtst-devel Development files for libxtst  
lib64udev-devel udev library development files  
lib64ffmpeg-static-devel Static library for the ffmpeg codec library  
gcc-c++ C++ support for gcc  
.  
To build WebKit
bison A GNU general-purpose parser generator  
flex A tool for creating scanners (text pattern recognizers)  
gperf A perfect hash function generator  
ruby Object Oriented Script Language  
libstdc++-static-devel Static libraries for C++ development  

Ok

Try to run my application with a JavaFX patched WebCore :

java @run.args JavaFXMathML

Error message :

public class JavaFXMathML extends Application {
       ^
1 error
Exception in Application start method
java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:464)
        at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:363)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:941)
Caused by: java.lang.RuntimeException: Exception in Application start method
        at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
        at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
        at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.IllegalAccessError: class com.sun.javafx.scene.control.Logging (in module javafx.controls) cannot access class com.sun.javafx.logging.PlatformLogger (in module javafx.base) because module javafx.base does not export com.sun.javafx.logging to module javafx.controls
        at javafx.controls/com.sun.javafx.scene.control.Logging.getControlsLogger(Logging.java:47)
        at javafx.controls/javafx.scene.control.Control$2.invalidated(Control.java:316)
        at javafx.base/javafx.beans.property.ObjectPropertyBase.markInvalid(ObjectPropertyBase.java:112)
        at javafx.base/javafx.beans.property.ObjectPropertyBase.set(ObjectPropertyBase.java:147)
        at javafx.graphics/javafx.css.StyleableObjectProperty.set(StyleableObjectProperty.java:82)
        at javafx.controls/javafx.scene.control.Control$2.set(Control.java:250)
        at javafx.controls/javafx.scene.control.Control$2.set(Control.java:233)
        at javafx.controls/javafx.scene.control.Control.loadSkinClass(Control.java:757)
        at javafx.controls/javafx.scene.control.Control$5.invalidated(Control.java:680)
        at javafx.base/javafx.beans.property.StringPropertyBase.markInvalid(StringPropertyBase.java:110)
        at javafx.base/javafx.beans.property.StringPropertyBase.set(StringPropertyBase.java:145)
        at javafx.graphics/javafx.css.StyleableStringProperty.set(StyleableStringProperty.java:83)
        at javafx.controls/javafx.scene.control.Control$5.set(Control.java:672)
        at javafx.graphics/javafx.css.StyleableStringProperty.applyStyle(StyleableStringProperty.java:69)
        at javafx.graphics/javafx.css.StyleableStringProperty.applyStyle(StyleableStringProperty.java:45)
        at javafx.web/javafx.scene.web.HTMLEditor.<init>(HTMLEditor.java:50)
        at JavaFXMathML.start(JavaFXMathML.java:26)
        at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
        at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:451)
        at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:424)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:423)
        at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
        at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication._runLoop(Native Method)
        at javafx.graphics/com.sun.glass.ui.gtk.GtkApplication.lambda$runLoop$11(GtkApplication.java:277)
        ... 1 more
kevinrushforth commented 6 years ago

Not sure why you are using make_runargs.sh -- that script is usually meant for taking your own build of JavaFX done on one machine and copying it to another where the path to the FX modules is different (since run.args has absolute paths).

In any case, the problem seems to be that the exports from buildSrc/addExports is not being included in the generated run.args as it should be, which is a bug in make_runargs.sh. You can append the needed exports from that file to your generated run.args, or just not use it in the first place and take the build/run.args from your JavaFX build and manually adjust the paths as needed.

scientificware commented 6 years ago

Thanks a lot and sorry. You're right, I made three big mistakes :

It works well with build/run.args from my JavaFX build.

scientificware commented 6 years ago

Some working documents :

MathML Refactoring In WebKit (pdf) MathML Improvements In WebKit (pdf)

scientificware commented 6 years ago

If you want to build WebKit :

Before to run gradle, edit the file in gradle.properties.template in rt/ and save it as "rt/gradle.properties" :

# These properties give developers the chance to skip building WebKit and/or
# GStreamer. WebKit takes a fair amount of time to build (more than 50% of the
# overall full build time is taken by WebKit), so allowing a developer to
# selectively enable building of WebKit is important. To build WebKit or
# GStreamer, uncomment the appropriate lines below.

COMPILE_WEBKIT = true
scientificware commented 6 years ago
scientificware commented 6 years ago
scientificware commented 6 years ago
scientificware commented 6 years ago
scientificware commented 6 years ago

Change c++ option : -fno-rtti

Where is the best place to change this option ? …/javafx.web/src/main/native/Source/cmake/WebKitCompilerFlags.cmake : Change WEBKIT_APPEND_GLOBAL_CXX_FLAGS(-fno-rtti) to # WEBKIT_APPEND_GLOBAL_CXX_FLAGS(-fno-rtti)

arajkumar commented 6 years ago

Do you want to remove the C++ RTTI? Then you can modify the same from WebKitCompilerFlags.cmake#line105. BTW, why do you want to remove that?

To explore the chagelog of a WebKit, please refer this. Current JavaFXWebView is based on GTK WebKit 2.18 branch.

scientificware commented 6 years ago

Hi, Thank you very much,

Yes, I want to remove -fno-rtti, in order to use typeid to identify child in RenderMathMLBlock::layoutItems and understand why height is set to 1.

Have you got a specific configuration (Netbeans or others) to debug JavaFX/WebKit ?

Best regards.

Read the Wiki ! Using an IDE Of course ! This is the second time I forget that basic advice

scientificware commented 6 years ago

The child of a RenderMathMLBlock is an RenderBlockFlow.

scientificware commented 6 years ago

I think that I gone a bit farther. The problem only affects MathML elements : RenderBox, RenderBlockFlow, RenderBlock, ... work fine for all element types other than MathML elements. I'm Trying something rather simple this morning.

scientificware commented 6 years ago

I think that MathML Renders are cast in an inappropiate objects.

contains(DEFINES, ENABLE_MATHML=1) {
    SOURCES += \
        css/CSSDefaultStyleSheets.cpp \
        accessibility/AXObjectCache.cpp \
        accessibility/AccessibilityNodeObject.cpp \
        accessibility/AccessibilityObject.cpp \
        accessibility/AccessibilityRenderObject.cpp \
        dom/Document.cpp \
        mathml/MathMLInlineContainerElement.cpp \
        mathml/MathMLAnnotationElement.cpp\
        mathml/MathMLElement.cpp \
        mathml/MathMLFractionElement.cpp \
        mathml/MathMLMathElement.cpp \
        mathml/MathMLMencloseElement.cpp \
        mathml/MathMLOperatorDictionary.cpp \
        mathml/MathMLOperatorElement.cpp \
        mathml/MathMLPaddedElement.cpp \
        mathml/MathMLPresentationElement.cpp \
        mathml/MathMLRowElement.cpp \
        mathml/MathMLScriptsElement.cpp \
        mathml/MathMLSelectElement.cpp \
        mathml/MathMLSpaceElement.cpp \
        mathml/MathMLTokenElement.cpp \
        mathml/MathMLUnderOverElement.cpp \
        rendering/RenderBox.cpp \
        rendering/RenderTreeAsText.cpp \
        rendering/RenderObject.h \
        rendering/RenderTableCell.cpp \
        rendering/mathml/MathMLStyle.cpp \
        rendering/mathml/MathOperator.cpp \
        rendering/mathml/RenderMathMLBlock.cpp \
        rendering/mathml/RenderMathMLFenced.cpp \
        rendering/mathml/RenderMathMLFencedOperator.cpp
        rendering/mathml/RenderMathMLFraction.cpp \
        rendering/mathml/RenderMathMLMath.cpp \
        rendering/mathml/RenderMathMLMenclose.cpp \
        rendering/mathml/RenderMathMLOperator.cpp \
        rendering/mathml/RenderMathMLPadded.cpp
        rendering/mathml/RenderMathMLRoot.cpp \
        rendering/mathml/RenderMathMLRow.cpp \
        rendering/mathml/RenderMathMLScripts.cpp \
        rendering/mathml/RenderMathMLSpace.cpp \
        rendering/mathml/RenderMathMLToken.cpp \
        rendering/mathml/RenderMathMLUnderOver.cpp
}
scientificware commented 6 years ago

Qt [Wiki]

Qt Documentation :

scientificware commented 6 years ago

Do not add these files which are already added in SOURCES

# css/CSSDefaultStyleSheets.cpp \
# accessibility/AXObjectCache.cpp \
# accessibility/AccessibilityNodeObject.cpp \
# accessibility/AccessibilityObject.cpp \
# accessibility/AccessibilityRenderObject.cpp \
# dom/Document.cpp \

Do not add this file which has been renamed to MathMLPresentationElement

# mathml/MathMLInlineContainerElement.cpp \
scientificware commented 6 years ago
scientificware commented 6 years ago

I'm looking for what is the function of WTFMove. WTF : Web Template Framework WTFMove is a macro defined in StdLibExtras.h : #define WTFMove(value) std::move<WTF::CheckMoveParameter>(value)

Bug 152601 - Use of WTF::move prevents clang's move diagnostics from warning about several classes of mistakes

scientificware commented 6 years ago
scientificware commented 6 years ago 
void RenderMathMLToken::layoutBlock(bool relayoutChildren, LayoutUnit pageLogicalHeight)
{
    ASSERT(needsLayout());

    if (!relayoutChildren && simplifiedLayout())
        return;

    GlyphData mathVariantGlyph;
    if (m_mathVariantCodePoint)
        mathVariantGlyph = style().fontCascade().glyphDataForCharacter(m_mathVariantCodePoint.value(), m_mathVariantIsMirrored);

    if (!mathVariantGlyph.font) {
        RenderMathMLBlock::layoutBlock(relayoutChildren, pageLogicalHeight);
        return;
    }

    for (auto* child = firstChildBox(); child; child = child->nextSiblingBox())
        child->layoutIfNeeded();

    setLogicalWidth(mathVariantGlyph.font->widthForGlyph(mathVariantGlyph.glyph));

    setLogicalHeight(mathVariantGlyph.font->boundsForGlyph(mathVariantGlyph.glyph).height());
    clearNeedsLayout();
}
    for (auto* child = firstChildBox(); child; child = child->nextSiblingBox())
        child->layoutIfNeeded();

    setLogicalWidth(mathVariantGlyph.font->widthForGlyph(mathVariantGlyph.glyph));

    setLogicalHeight(mathVariantGlyph.font->boundsForGlyph(mathVariantGlyph.glyph).height());
    clearNeedsLayout();
scientificware commented 6 years ago

Read previous comments above : For Java platform FontJava.cpp platformBoundsForGlyph doesn't return the bounds of the glyph but a (0,0,0,0) rectangle. May be that one who wrote this comment :

//That is OK! platformWidthForGlyph impl is enough.

Doesn't take in account that height values are used in MathML Rendering ?

So Font.h boundsForGlyph returns 0 for the height.

Try to explore that, this weekend.

scientificware commented 6 years ago

Morning suggestion, before breakfast (draft for this weekend):

FontJava.cpp return FloatRect(); //That is OK! platformWidthForGlyph impl is enough

FloatRect Font::platformBoundsForGlyph(Glyph) const
    {
    // return FloatRect(); //That is OK! platformWidthForGlyph impl is enough.

    JNIEnv* env = WebCore_GetJavaEnv();

    RefPtr<RQRef> jFont = m_platformData.nativeFontData();
    if (!jFont) {
        return 0.0f;
    }

    static jmethodID getGlyphBoundingBox_mID = env->GetMethodID(PG_GetFontClass(env), "getGlyphBoundingBox", "(I)D");
    ASSERT(getGlyphBoundingBox_mID);

    float boudingBox[4];

    (jfloatArray) res = (jfloatArray)env->CallObjectMethod(*jFont, getGlyphBoundingBox_mID, (jint)c);
    CheckAndClearException(env);

    return FloatRect(res[0], res[1], res[2], res[3]);
}

WCFontImpl.java 

 @Override public float[] getGlyphBoundingBox(int glyph) {
        float[] bb = new float[4];
        bb = getFontStrike().getFontResource().getGlyphBoundingBox(glyph, font.getSize(), bb);
        return bb;
    }
scientificware commented 6 years ago

First tests confirm my analysis. I finally found what's wrong with MathML lines overlapping. In fact MathML needs platformBoundsForGlyph implementation in FontJava.cpp.

With patchs describe in this and previous message, fraction numerator and denominator like matrix lines are in there right positions.

That's not solve all problems, but it's a great progress.

WCFontPerfLogger.java

    public float[] getGlyphBoundingBox(int glyph) {
        logger.resumeCount("GETGLYPHBOUNDINGBOX");
        float[] res = fnt.getGlyphBoundingBox(glyph);
        logger.suspendCount("GETGLYPHBOUNDINGBOX");
        return res;
    }

WCFont.java

public abstract float[] getGlyphBoundingBox(int glyph);
scientificware commented 6 years ago

Morning correction from previous morning suggestion, before breakfast too :

FontJava.cpp return FloatRect(); //That is OK! platformWidthForGlyph impl is enough

FloatRect Font::platformBoundsForGlyph(Glyph c) const
    {
    //return FloatRect(); //That is OK! platformWidthForGlyph impl is enough.

    JNIEnv* env = WebCore_GetJavaEnv();

    RefPtr<RQRef> jFont = m_platformData.nativeFontData();
    if (!jFont) {
        return FloatRect();
    }

    static jmethodID getGlyphBoundingBox_mID = env->GetMethodID(PG_GetFontClass(env), "getGlyphBoundingBox", "(I)[F");
    ASSERT(getGlyphBoundingBox_mID);

    jfloatArray boundingBox = (jfloatArray)env->CallObjectMethod(*jFont, getGlyphBoundingBox_mID, (jint)c);

    jfloat *bBox = env->GetFloatArrayElements(boundingBox,0);

    CheckAndClearException(env);

    return FloatRect(bBox[0], bBox[1], bBox[2], bBox[3]);
    }
scientificware commented 6 years ago

The remaining problem seems to be a Glyph ascent and descent, I'm still working on it. It seems that FloatRect have to contain ascent and descent.

MathOperator.cpp

static inline void getAscentAndDescentForGlyph(const GlyphData& data, LayoutUnit& ascent, LayoutUnit& descent)
{
    FloatRect bounds = boundsForGlyph(data);
    ascent = -bounds.y();
    descent = bounds.maxY();
}

So, I presume, I had to simply put ascent in y() and descent in maxY().

In my current patch :

So, I should use getAscent() and getDescent() values rather than getGlyphBoundingBox values ?

scientificware commented 6 years ago

I'm right !

mathml_javafx_20180609

This patch doens't solve all problems but we can at least display elementary school mathematic formulars.

I think like Arunprasad Rajkumar that to go further, javafx will have to support math table.

Touch down !

scientificware commented 6 years ago

WCFontImpl.java

The ascent and descent problem has been resolved by :

@Override public float[] getGlyphBoundingBox(int glyph) {
        float[] bb = new float[4];
        bb = getFontStrike().getFontResource().getGlyphBoundingBox(glyph, font.getSize(), bb);
        bb[1]= -getAscent();
        bb[3]= getDescent();
        return bb;
    }
kevinrushforth commented 6 years ago

@scientificware if you plan to contribute your fix for this issue to FX, we will need a signed OCA on file before it can be reviewed and accepted; see CONTRIBUTING.md.

muralibilla commented 6 years ago

@scientificware Nice to see that you made good progress on this issue. We can treat this issue separately from webkit upgrade. Once you are ready with complete patch (including unit tests), you can generate a pull request to initiate review.

scientificware commented 6 years ago

Abossolo Foh Guy - NB I already signed the OCA.

kevinrushforth commented 6 years ago

Yes, I see you on the list of OCA signatories. Thanks.

scientificware commented 6 years ago

I don't understand why everybody adopts this approch of stretchy characters ! Implement a METAFONT like language would had be more efficient and simple. It's really a shame that the WebCore part was not written en Java. Java Swing text package seems to have been written for D. Knuth.

Stretchy Characters UTF-16 Code :

Name Char. Sy. Top Char. Extension Char. Bottom Char. Middle Char.
left parenthesis 0x28 ( 0x239b 0x239c 0x239d 0x0
right parenthesis 0x29 ) 0x239e 0x239f 0x23a0 0x0
left square bracket 0x5b [ 0x23a1 0x23a2 0x23a3 0x0
right square bracket 0x5d ] 0x23a4 0x23a5 0x23a6 0x0
left curly bracket 0x7b { 0x23a7 ⎧ 0x23aa ⎪ 0x23a9 ⎩ 0x23a8 ⎨
right curly bracket 0x7d } 0x23ab ⎫ 0x23aa ⎪ 0x23ad ⎭ 0x23ac ⎬
left ceiling 0x2308 ⌈ 0x23a1 0x23a2 0x23a2 0x0
right ceiling 0x2309 ⌉ 0x23a4 0x23a5 0x23a5 0x0
left floor 0x230a ⌈ 0x23a2 0x23a2 0x23a3 0x0
right floor 0x230b ⌋ 0x23a5 0x23a5 0x23a6 0x0
vertical bar 0x7c | 0x7c 0x7c 0x7c 0x0
double vertical line 0x2016 ‖ 0x2016 0x2016 0x2016 0x0
parallel to 0x2225 ∥ 0x2225 0x2225 0x2225 0x0
integral sign 0x222b ∫ 0x2320 0x23ae 0x2321 0x0
scientificware commented 6 years ago

Yes !

With this small correction matrix delimiters are well displayed !

WCFontImpl.java 

        bb[1]= -getCapHeight();
        bb[3]= getDescent()-bb[1];

In fact, I don't understand exactly what getGlyphBoundingBox should return ! ?

TeX on the left and JavaFX / OpenJFX on the right.

screenshot_20180611_201434

screenshot_20180612_032744

scientificware commented 6 years ago

screenshot_20180612_052032

scientificware commented 6 years ago

Ok, I've just understand what's wrong !

Name Char. Sy. Top Char. Extension Char. Bottom Char. Middle Char.
left curly bracket 0x7b { ? ⎧ ? ⎪ ? ⎩ ? ⎨
right curly bracket ? } ? ⎫ ? ⎪ ? ⎭ ? ⎬
left ceiling ? ⌈ ? ? ? ?
right ceiling ? ⌉ ? ? ? ?
left floor ? ⌈ ? ? ? ?
right floor ? ⌋ ? ? ? ?
vertical bar ? | ? ? ? ?
double vertical line ? ‖ ? ? ? ?
parallel to ? ∥ ? ? ? ?
integral sign ? ∫ ? ? ? ?
radical sign ? √

Well have got all pieces of this puzzle ! All that's left to do, try to build something resilient.

The picture has been obtained after change order in the list of font in mathml.css.

screenshot_20180612_180253

scientificware commented 6 years ago

The relative test :

import javafx.application.Application; import javafx.application.Platform; import javafx.scene.Scene; import javafx.scene.layout.StackPane; import javafx.scene.web.HTMLEditor; import javafx.scene.web.WebEngine; import javafx.scene.web.WebView; import javafx.stage.Stage;

/**

}

scientificware commented 6 years ago

Exactly what I need and what I has been waiting since few years ! Hope, I could use it in my projects soon.

screenshot_20180613_123931

Hi ! screenshot_20180613_125304

The code of this page.

      <p>
    Here is an example of Presentation MathML:
    <math>
      <mrow>
        <mfenced>
          <mtable>
        <mtr><mtd><mi> a </mi></mtd> <mtd><mi> b </mi></mtd></mtr>
        <mtr><mtd><mi> c </mi></mtd> <mtd><mi> d </mi></mtd></mtr>
          </mtable>
        </mfenced>
        <mo> &#x2062;<!--INVISIBLE TIMES--> </mo>
        <mfenced>
          <mtable>
        <mtr><mtd><mi> x </mi></mtd></mtr>
        <mtr><mtd><mi> y </mi></mtd></mtr>
          </mtable>
        </mfenced>
      </mrow>
      <mo> = </mo>
      <mfenced>
        <mtable>
          <mtr><mtd><mi> e </mi></mtd></mtr>
          <mtr><mtd><mi> f </mi></mtd></mtr>
        </mtable>
      </mfenced>
    </math>
        and another:
        <math>
         <mfenced open="{" close="">
          <mtable>
           <mtr><mtd>
             <mrow>
              <mrow><mi>a</mi><mo>&#x2062;</mo><mi>x</mi></mrow>
              <mo>+</mo>
              <mrow><mi>b</mi><mo>&#x2062;</mo><mi>y</mi></mrow>
             </mrow>
             <mo> = </mo>
             <mi> e </mi>
           </mtd></mtr>
           <mtr><mtd>
             <mrow>
              <mrow><mi>c</mi><mo>&#x2062;</mo><mi>x</mi></mrow>
              <mo>+</mo>
              <mrow><mi>d</mi><mo>&#x2062;</mo><mi>y</mi></mrow>
             </mrow>
             <mo> = </mo>
             <mi> f </mi>
           </mtd></mtr>
          </mtable>
         </mfenced>
        </math>
      </p>

screenshot_20180613_130720

scientificware commented 6 years ago

The web browser I used to test the patch over the net.

import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.StackPane;
import javafx.scene.web.HTMLEditor;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

public class NavigateurTest extends Application {

    final HTMLEditor htmlEditor = new HTMLEditor();
    final StackPane root = new StackPane();
    final WebView webView = (WebView) htmlEditor.lookup(".web-view");

    public void init() {
        htmlEditor.setHtmlText("");
        root.getChildren().add(htmlEditor);
        htmlEditor.setPrefWidth(800);
        htmlEditor.setPrefHeight(600);
    }

    @Override
    public void start(Stage primaryStage) {

        webView.getEngine().load("https://duckduckgo.com");

        primaryStage.setTitle("OpenJFX MathML Rendering Issue WebBrowser Test");
        primaryStage.setScene(new Scene(root));
        primaryStage.show();

    }

    /**
     * @param args the command line arguments
     */
    public static void main(String[] args) {
        launch(args);
    }

}
scientificware commented 6 years ago

JDK-8147476 Rendering issues with MathML token elements. JDK-8165520 Several mathml/ DRT tests fail JDK-8089878 HTMLEditor messes up MathML markup. JDK-8090887 Support MathML

scientificware commented 6 years ago

Another question was not resolved :

Why WebCore authors didn't used the same logic in normal text and mathml text ? They call bounding box height dimension rather than sum the text ascent and descent. One who wrapped webkit in JavaFX must have been thinking that it was the same logic.

My response : In fact the problem with mathematical glyphs is that they go upper and lower than ascent or the descent. Only the bounding box height gives this right measure of the glyph.

JavaFX prism and WebCore Text Bounding Box implementation are different.

Use of getAscent() and getDescent() is quite a good interim solution while I'm looking for what's wrong with JavaFX text bounding box informations.

scientificware commented 6 years ago

To do :

scientificware commented 6 years ago

PrismFontFile

Try to understand this :

createGlyphBoundingBox

    @Override
    protected synchronized int[] createGlyphBoundingBox](int gc) {
        int flags = OSFreetype.FT_LOAD_NO_SCALE;
        OSFreetype.FT_Load_Glyph(face, gc, flags);
        int[] bbox = new int[4];
        FT_GlyphSlotRec glyphRec = OSFreetype.getGlyphSlot(face);
        if (glyphRec != null && glyphRec.metrics != null) {
            FT_Glyph_Metrics gm = glyphRec.metrics;
            bbox[0] = (int)gm.horiBearingX;
            bbox[1] = (int)(gm.horiBearingY - gm.height);
            bbox[2] = (int)(gm.horiBearingX + gm.width);
            bbox[3] = (int)gm.horiBearingY;
        }
        return bbox;
}

It can explain what kind of informations is returned by getBoundingBox in FontResource.

Explanations :

Freetype definitions :

Picture giving all the JavaFX metric details for horizontal layout horizontal_layout_metrics

Picture giving all the JavaFX metric details for vertical layout vertical_layout_metrics

Picture giving all the WebCore metric details for horizontal layout webcore_glyph_metrics

So I understand why when using GlyphBoundingBox, I didn't succeed to perfrom right alignment. Then I used getAscent() and getDescent() which work well for usual glyphs but not for mathematical glyphs as explains in the previous comment.

The rectangle definition are in different order :

Finally no more need of getAscent() and getDescent(), I used, pending best JavaFX metrics understanding. Modify getGlyphBoundingBox

in WCFontImpl.java

Reorder values and compute height :

@Override public float[] getGlyphBoundingBox(int glyph) {
        float[] bb = new float[4];
        bb = getFontStrike().getFontResource().getGlyphBoundingBox(glyph, font.getSize(), bb);
        //bb[1]= -getAscent();
        //bb[3]= getDescent();
        return new float[]{bb[0],-bb[3],bb[2],bb[3]-bb[1]};
    }

Great ... adopted !

screenshot_20180619_191413

scientificware commented 6 years ago

OpenJFX unit tests

Building OpenJFX and testing

The junit test is ready to be tested

gradle -PFULL_TEST=true :systemTests:test --tests MathMLRenderTest

Test Succeeded !

screenshot_20180621_010700

/*
 * Copyright (c) 2017, 2018, Oracle and/or its affiliates. All rights reserved.
 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 *
 * This code is free software; you can redistribute it and/or modify it
 * under the terms of the GNU General Public License version 2 only, as
 * published by the Free Software Foundation.  Oracle designates this
 * particular file as subject to the "Classpath" exception as provided
 * by Oracle in the LICENSE file that accompanied this code.
 *
 * This code is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
 * version 2 for more details (a copy is included in the LICENSE file that
 * accompanied this code).
 *
 * You should have received a copy of the GNU General Public License version
 * 2 along with this work; if not, write to the Free Software Foundation,
 * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
 *
 * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA
 * or visit www.oracle.com if you need additional information or have any
 * questions.
 */

import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.scene.Scene;
import javafx.scene.layout.StackPane;
import javafx.scene.web.HTMLEditor;
import javafx.scene.web.WebEngine;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.Assert;

import static junit.framework.TestCase.fail;

import static test.util.Util.TIMEOUT;

public class MathMLRenderTest {

    private static final CountDownLatch launchLatch = new CountDownLatch(1);

    // Document test
    static String BodyContent
            = "<math display=\"block\">"
            + "   <mrow>"
            + "      <mi>x</mi>"
            + "      <mo>=</mo>"
            + "      <mfrac>"
            + "         <mrow>"
            + "            <mo>−</mo>"
            + "            <mi>b</mi>"
            + "            <mo>±</mo>"
            + "            <msqrt>"
            + "               <mrow>"
            + "                  <msup>"
            + "                     <mi>b</mi>"
            + "                     <mn>2</mn>"
            + "                  </msup>"
            + "                  <mo>−</mo>"
            + "                  <mn>4</mn>"
            + "                  <mi>a</mi>"
            + "                  <mi>c</mi>"
            + "               </mrow>"
            + "            </msqrt>"
            + "         </mrow>"
            + "         <mrow>"
            + "            <mn>2</mn>"
            + "            <mi>a</mi>"
            + "         </mrow>"
            + "      </mfrac>"
            + "   </mrow>"
            + "</math>";

    static String content = "<!doctype html>"
            + "<html>"
            + "   <head>"
            + "      <meta charset=\"UTF-8\">"
            + "      <title>OpenJFX and MathML</title>"
            + "   </head>"
            + "   <body>"
            + "      <p>"
            + BodyContent
            + "      </p>"
            + "   </body>"
            + "</html>";

    // Application instance
    static TestApp testApp;

    public static class TestApp extends Application {

        final HTMLEditor htmlEditor = new HTMLEditor();
        final StackPane root = new StackPane();
        final WebView webView = (WebView) htmlEditor.lookup(".web-view");

        public void init() {
            MathMLRenderTest.testApp = this;
            htmlEditor.setHtmlText(content);
            root.getChildren().add(htmlEditor);
            htmlEditor.setPrefWidth(400);
            htmlEditor.setPrefHeight(150);
        }

        @Override
        public void start(Stage primaryStage) {
            primaryStage.setTitle("OpenJFX MathML Rendering Issue Test");
            primaryStage.setScene(new Scene(root));
            primaryStage.show();
            launchLatch.countDown();
        }

        // Get height of the first token element <mo>
        public int getTokenHeight(){
            WebEngine we = webView.getEngine();
            int height = (int) we.executeScript(
                "elements = document.getElementsByTagName('mo');"
                + "element = elements[0].clientHeight;"
            );
            return height;
        }
    }

    @BeforeClass
    public static void setupOnce() {

        // Start the Test Application
        new Thread(() -> Application.launch(TestApp.class,
            (String[]) null)).start();

        try {
            if (!launchLatch.await(TIMEOUT, TimeUnit.MILLISECONDS)) {
                fail("Timeout waiting for FX runtime to start");
            }
        } catch (InterruptedException exception) {
            fail("Unexpected exception: " + exception);
        }    
    }

    @AfterClass
    public static void tearDownOnce() {
        Platform.exit();
    }

    /**
     * @test
     * @bug JDK-8147476 Rendering issues with MathML token elements
     */
    @Test
    public void testgetTokenHeight() throws Exception {

        final CountDownLatch editorStateLatch = new CountDownLatch(1);
        final AtomicBoolean rightRender = new AtomicBoolean(false);
        final AtomicInteger tokenHeight = new AtomicInteger(0);
        Platform.runLater(() -> {
            tokenHeight.set(testApp.getTokenHeight());
            rightRender.set(!(tokenHeight.get()<2));
            editorStateLatch.countDown();
        });

        try {
            editorStateLatch.await(TIMEOUT, TimeUnit.MILLISECONDS);
        } catch (InterruptedException ex) {
            throw new AssertionError(ex);
        } finally {
            Assert.assertTrue("Check MathML token height : " + tokenHeight.get() + " is much smaller than the expected size." , rightRender.get());
        }
    }
}
scientificware commented 6 years ago

Close Buzilla Webkit Report.

ogimage

https://bugs.webkit.org/show_bug.cgi?id=185417

scientificware commented 6 years ago

Bug report related to this bug :