rubyFeedback commented 2 years ago

Feature request

Please include the following information:

Is your feature request related to a problem? Please describe.

The below example, shown after a ---- line, can not be compiled via native-image.

It would be convenient if this would work out of the box for default/"vanilla" swing applications (at the least simple ones).

Describe the solution you'd like.

I would appreciate if some minimal swing support would be possible. There are other issues and blog posts where people were able to get at the least some swing applications to work, but these are a bit tedious to do (for instance, one required downloading some external program, so I'd prefer if I could just ask native-image to download what it needs optionally, and proceed with it, rather than have to collect what I need on my own, if possible).

If someone on the GraalVM team has time and motivation, perhaps it could be done to see so that at the least the basic functionality, buttons etc... may work via swing + native-image. People say that swing is outdated and deprecated compared to JavaFX and that may be, but it would still be nice at the least for simple applications to work.

Describe who do you think will benefit the most.

I believe the net benefit for the GraalVM users would be that they could get native swing apps in one binary to work; which they could distribute (e. g. my use case is to get some functionality into it, and have that be usable for a few elderly people in my local area. I can work around it, that's ok, default java works just fine, but I'd love to be able to switch to GraalVM completely for everything related to java in the long run.).

Describe alternatives you've considered.

Well - I can use alternatives of course, which I am already doing, but I am not interested in the alternatives. I want this to work out-of-the-box on GraalVM eventually. ;)

Additional context.

That's it - see the content below. This example works as-is via standalone java. Via GraalVM compile errors occur though.

Express whether you'd like to help contributing this feature

Unfortunately I am still a bit of a Java noob so I don't think I can contribute much. And it's ok to meta-merge this perhaps, add "swing" or "GUI" label to it to track progress, or move it to some other category. There are talks about this in other issues, but I wanted to add my own little example here to get the GraalVM devs to not forget about swing too. :D

Thanks for reading! Now the content.

import java.awt.BorderLayout;
import java.awt.Container;

import javax.swing.JFrame;
import javax.swing.JTextPane;
import javax.swing.text.Element;
import javax.swing.text.html.HTMLDocument;
import javax.swing.text.html.HTMLEditorKit;
import javax.swing.text.html.StyleSheet;

public class CssExample extends JFrame {

  StyleSheet styleSheet = new StyleSheet();
  HTMLDocument htmlDocument;
  HTMLEditorKit htmlEditorKit = new HTMLEditorKit();
  Element bodyElement;

  public static void main(String[] args) throws Exception {
    CssExample jTextPaneApp = new CssExample();
    jTextPaneApp.setVisible(true);

    Thread.currentThread().sleep(5000);

    jTextPaneApp.change();
  }

  public CssExample() {
    setSize(500, 500);
    styleSheet.addRule(".someclass1 {color: blue; font-size: 50px;}");
    styleSheet.addRule(".someclass2 {color: green;}");

    htmlEditorKit.setStyleSheet(styleSheet);
    htmlDocument = (HTMLDocument) htmlEditorKit.createDefaultDocument();
    JTextPane jTextPane = new JTextPane();
    jTextPane.setEditorKit(htmlEditorKit);
    jTextPane.setDocument(htmlDocument);

    try {
      Element htmlElement = htmlDocument.getRootElements()[0];
      bodyElement = htmlElement.getElement(0);

      Container contentPane = getContentPane();
      contentPane.add(jTextPane, BorderLayout.CENTER);
      super.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

      addContent("<span class=someclass1>test 1</span><br>");
      addContent("<span class=someclass2>test 2</span><br>");

    } catch (Exception e) {
      e.printStackTrace();
    }
  }

  public void reapplyStyles() {
    Element sectionElem = bodyElement.
                          getElement(bodyElement.getElementCount() - 1);
    int paraCount = sectionElem.getElementCount();
    for (int i = 0; i < paraCount; i++) {
      Element e = sectionElem.getElement(i);
      int rangeStart = e.getStartOffset();
      int rangeEnd = e.getEndOffset();
      htmlDocument.setParagraphAttributes(rangeStart, rangeEnd - rangeStart,
          e.getAttributes(), true);
    }
  }

  public void change() throws Exception {
    styleSheet = htmlEditorKit.getStyleSheet();
    styleSheet.addRule(".someclass1 {color: red;}");
    reapplyStyles();
    addContent("<span class=someclass1>test 3</span><br>");
  }

  private void addContent(String content) throws Exception {
    Element contentElement = bodyElement.getElement(bodyElement
        .getElementCount() - 1);

    StringBuffer sbHtml = new StringBuffer();
    sbHtml.append(
      "<span class=someclass>" + content + "</span><br>"
    );

    htmlDocument.insertBeforeEnd(contentElement, sbHtml.toString());
  }
}

Compile error was:

native-image CssExample

[cssexample:25476] classlist: 2,855.69 ms, 0.96 GB [cssexample:25476] (cap): 1,514.07 ms, 0.96 GB [cssexample:25476] setup: 4,479.58 ms, 0.96 GB [cssexample:25476] analysis: 13,446.90 ms, 1.24 GB Fatal error:com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing sun.awt.X11.XClipboard.getClipboardFormats() Parsing context: at sun.awt.X11.XClipboard.getClipboardFormats(XClipboard.java:118) at sun.awt.datatransfer.ClipboardTransferable.(ClipboardTransferable.java:80) at sun.awt.X11.XClipboard.getContents(XClipboard.java:108) at javax.swing.text.DefaultCaret.mouseClicked(DefaultCaret.java:493) at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:278) at java.awt.Component.processMouseEvent(Component.java:6629) at java.awt.Component.processEvent(Component.java:6391) at java.awt.Container.processEvent(Container.java:2266) at java.awt.Component.dispatchEventImpl(Component.java:5001) at java.awt.Container.dispatchEventImpl(Container.java:2324) at java.awt.Window.dispatchEventImpl(Window.java:2780) at java.awt.Component.dispatchEvent(Component.java:4833)

at com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:133)
at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:311)
at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureTypeFlowCreated(MethodTypeFlow.java:282)
at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:103)
at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:222)
at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:490)
at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:559)
at com.oracle.graal.pointsto.PointsToAnalysis$2.run(PointsToAnalysis.java:595)
at com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:188)
at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:172)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)

Caused by: com.oracle.graal.pointsto.util.AnalysisError: parsing had failed in another thread at com.oracle.graal.pointsto.util.AnalysisError.shouldNotReachHere(AnalysisError.java:153) at com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:656) at com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.lookupEncodedGraph(InlineBeforeAnalysis.java:182) at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.doInline(PEGraphDecoder.java:1120) at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.tryInline(PEGraphDecoder.java:1103) at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.trySimplifyInvoke(PEGraphDecoder.java:961) at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.handleInvoke(PEGraphDecoder.java:915) at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.processNextNode(GraphDecoder.java:791) at com.oracle.graal.pointsto.phases.InlineBeforeAnalysisGraphDecoder.processNextNode(InlineBeforeAnalysis.java:242) at jdk.internal.vm.compiler/org.graalvm.compiler.nodes.GraphDecoder.decode(GraphDecoder.java:532) at jdk.internal.vm.compiler/org.graalvm.compiler.replacements.PEGraphDecoder.decode(PEGraphDecoder.java:787) at com.oracle.graal.pointsto.phases.InlineBeforeAnalysis.decodeGraph(InlineBeforeAnalysis.java:99) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:171) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:321) at com.oracle.graal.pointsto.flow.MethodTypeFlow.createTypeFlow(MethodTypeFlow.java:293) ... 14 more Caused by: org.graalvm.compiler.java.BytecodeParser$BytecodeParserError: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported type sun.awt.X11.XBaseWindow is reachable To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time. at parsing sun.awt.X11.XToolkit.getCurrentServerTime(XToolkit.java:1464) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2624) at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:107) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3485) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3437) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3282) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1145) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1030) at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84) at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:81) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:212) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42) at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38) at com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:132) at com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:621) ... 27 more Caused by: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported type sun.awt.X11.XBaseWindow is reachable To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time. at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.lookup(AnnotationSubstitutionProcessor.java:120) at com.oracle.graal.pointsto.infrastructure.SubstitutionProcessor$ChainedSubstitutionProcessor.lookup(SubstitutionProcessor.java:125) at com.oracle.graal.pointsto.infrastructure.SubstitutionProcessor$ChainedSubstitutionProcessor.lookup(SubstitutionProcessor.java:125) at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:199) at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:179) at com.oracle.graal.pointsto.meta.AnalysisMethod.(AnalysisMethod.java:118) at com.oracle.graal.pointsto.meta.AnalysisUniverse.createMethod(AnalysisUniverse.java:429) at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:417) at com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.lookupMethod(WrappedConstantPool.java:125) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.lookupMethodInPool(BytecodeParser.java:4350) at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.lookupMethodInPool(SharedGraphBuilderPhase.java:140) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.lookupMethod(BytecodeParser.java:4344) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1649) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5419) at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3477) ... 39 more [cssexample:25476] [total]: 21,237.03 ms, 1.24 GB

Printing build artifacts to: /home/x/programming/java/src/swing/cssexample.build_artifacts.txt

Error: Image build request failed with exit status 1

kkriske commented 2 years ago

Tried out your example:

Linux:

Note that for linux, the value of the java.awt.headless property is baked in because different libraries have to be linked based on if it is to run headless or not. So what you're missing here is pretty much just -Djava.awt.headless=false on the native-image command.

# collect config for the app
java -agentlib:native-image-agent=config-merge-dir=. CssExample
# compile native-image, note the `-Djava.awt.headless=false` flag
native-image --no-fallback --no-server --verbose -Djava.awt.headless=false -H:JNIConfigurationFiles=./jni-config.json -H:ReflectionConfigurationFiles=./reflect-config.json -H:ResourceConfigurationFiles=./resource-config.json CssExample
# run native binary
./cssexample

Functionality looks identical as running from java. I did run from WSL2 in windows, with an XServer, but I don't think that should matter anyway.

Windows:

REM collect config for the app
java -agentlib:native-image-agent=config-merge-dir=. CssExample
REM compile native-image
native-image --no-fallback --no-server --verbose -H:JNIConfigurationFiles=./jni-config.json -H:ReflectionConfigurationFiles=./reflect-config.json -H:ResourceConfigurationFiles=./resource-config.json CssExample
REM copy required resources
mkdir conf\fonts
copy "%JAVA_HOME%\lib\fontconfig.bfc" conf\fonts\fontconfig.bfc
REM run native binary
cssexample.exe -Djava.home=.

The windows binary looks for a conf/fonts/fontconfig.bfc file by default, in the java home location. So that's why -Djava.home=. and the extra resource relative to the binary are required. This could be fixed fairly easy by a feature that includes that file in the image resources, and a simple substitution that grabs that resource instead of looking in the filesystem.

On windows, the styled default text doesn't get added in the text pane, but other than that, functionality seems the same as in java.

MacOS

I don't have a MacOS device so I have no idea if that works.

rodrigar-mx commented 2 years ago

@rubyFeedback did the suggested approach work for you? Otherwise, please share the graalvm version and the OS you are using.

kkriske commented 2 years ago

@rodrigar-mx for the windows case, the feature & substitution that I mention is something that I think should be included in native-image out of the box. So in that regard there is still an issue to solve here.

As for the linux case, it is not obvious that you need to set the java.awt.headless flag so it would help if that can be documented somewhere.

rodrigar-mx commented 2 years ago

@rubyFeedback this feature request has been reported internally. Any updates or questions we will let you know.

oracle / graal

[GR-36435] Swing support: example file (its content) in the issue itself (via "native-image") #4165

Printing build artifacts to: /home/x/programming/java/src/swing/cssexample.build_artifacts.txt

Linux:

Windows:

MacOS