search

eclipse / xtext

Eclipse Xtext™ is a language development framework
http://www.eclipse.org/Xtext
Eclipse Public License 2.0
754 stars 317 forks source link

Error creating PDA for syntactic sequencer using Maven #2394

Open S-wald opened 1 year ago

S-wald commented 1 year ago

As discussed in https://www.eclipse.org/forums/index.php/t/1112014/ building a xtext DSL with three grammars in the hierarchy (MyDsl -with-> Base2 -with-> Base) leads to to an error during the workflow for the last grammar (MyDsl). @cdietrich suggested to ping @szarnekow.

Sample project: project.zip

Error:

1137 ERROR SyntacticSequencerPDAProvider - Error creating PDA for syntactic sequencer for contexts: Basic2 returns Basic2: no type found for name=unresolved_rule
java.lang.RuntimeException: no type found for name=unresolved_rule
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.getType(SyntacticSequencerPDAProvider.java:703)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.createAbsorberState(SyntacticSequencerPDAProvider.java:564)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.createAbsorberState(SyntacticSequencerPDAProvider.java:569)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.getSyntacticSequencerPDAs(SyntacticSequencerPDAProvider.java:631)
        at org.eclipse.xtext.serializer.analysis.SemanticSequencerNfaProvider.getSemanticSequencerNFAs(SemanticSequencerNfaProvider.java:240)
        at org.eclipse.xtext.serializer.analysis.GrammarConstraintProvider.getConstraints(GrammarConstraintProvider.java:624)
        at org.eclipse.xtext.xtext.generator.serializer.SemanticSequencerExtensions.getGrammarConstraints(SemanticSequencerExtensions.java:112)
        at org.eclipse.xtext.xtext.generator.serializer.SerializerFragment2.generateAbstractSemanticSequencer(SerializerFragment2.java:417)
        at org.eclipse.xtext.xtext.generator.serializer.SerializerFragment2.generate(SerializerFragment2.java:211)
        at org.eclipse.xtext.xtext.generator.CompositeGeneratorFragment2.generate(CompositeGeneratorFragment2.java:43)
        at org.eclipse.xtext.xtext.generator.XtextGenerator.invokeInternal(XtextGenerator.java:228)
        at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:133)
        at org.eclipse.emf.mwe.core.lib.Mwe2Bridge.invoke(Mwe2Bridge.java:35)
        at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:213)
        at org.eclipse.emf.mwe2.runtime.workflow.AbstractCompositeWorkflowComponent.invoke(AbstractCompositeWorkflowComponent.java:38)
        at org.eclipse.emf.mwe2.runtime.workflow.Workflow.run(Workflow.java:21)
        at org.eclipse.emf.mwe2.launch.runtime.M
we2Runner.run(Mwe2Runner.java:103)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:63)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:53)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:78)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:36)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:254)
        at java.base/java.lang.Thread.run(Thread.java:829)
1142 ERROR SyntacticSequencerPDAProvider - Error creating PDA for syntactic sequencer for contexts: KEYWORDID2 returns null: no type found for unresolved_rule
java.lang.RuntimeException: no type found for unresolved_rule
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.getType(SyntacticSequencerPDAProvider.java:703)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.createEmitterStates(SyntacticSequencerPDAProvider.java:605)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.createAbsorberState(SyntacticSequencerPDAProvider.java:574)
        at org.eclipse.xtext.serializer.analysis.SyntacticSequencerPDAProvider.getSyntacticSequencerPDAs(SyntacticSequencerPDAProvider.java:631)
        at org.eclipse.xtext.serializer.analysis.SemanticSequencerNfaProvider.getSemanticSequencerNFAs(SemanticSequencerNfaProvider.java:240)
        at org.eclipse.xtext.serializer.analysis.GrammarConstraintProvider.getConstraints(GrammarConstraintProvider.java:624)
        at org.eclipse.xtext.xtext.generator.serializer.SemanticSequencerExtensions.getGrammarConstraints(SemanticSequencerExtensions.java:112)
        at org.eclipse.xtext.xtext.generator.serializer.SerializerFragment2.generateAbstractSemanticSequencer(SerializerFragment2.java:417)
        at org.eclipse.xtext.xtext.generator.serializer.SerializerFragment2.generate(SerializerFragment2.java:211)
        at org.eclipse.xtext.xtext.generator.CompositeGeneratorFragment2.generate(CompositeGeneratorFragment2.java:43)
        at org.eclipse.xtext.xtext.generator.XtextGenerator.invokeInternal(XtextGenerator.java:228)
        at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:133)
        at org.eclipse.emf.mwe.core.lib.Mwe2Bridge.invoke(Mwe2Bridge.java:35)
        at org.eclipse.emf.mwe.core.lib.AbstractWorkflowComponent.invoke(AbstractWorkflowComponent.java:213)
        at org.eclipse.emf.mwe2.runtime.workflow.AbstractCompositeWorkflowComponent.invoke(AbstractCompositeWorkflowComponent.java:38)
        at org.eclipse.emf.mwe2.runtime.workflow.Workflow.run(Workflow.java:21)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:103)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:63)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:53)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:78)
        at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:36)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:254)
        at java.base/java.lang.Thread.run(Thread.java:829)
cdietrich commented 1 year ago

i also wonder if the clone methods in the SyntacticSequencerPDAProvider are bogus. maybe it should load all supergrammars into the clone

what i dont understand. sometimes it works from eclipse. sometimes it does not. e.g. after running maven, eclipse does not run anymore. in the error case the copied base2.xtext has no supergrammar. i also wonder if this is a problem between http and platform:/resource uris for the supergrammars (have no idea how these are mapped back and forth - normalization map)

URI uri = Iterables.getFirst(grammar.getUsedGrammars(), null).eResource().getURI();
        ResourceSet resourceSet = cloneResourceSet(grammar.eResource().getResourceSet());
        for(Grammar used: GrammarUtil.allUsedGrammars(grammar)) {
            URI uri2 = used.eResource().getURI();
            System.err.println(uri2);
            resourceSet.getResource(uri2 , true);
        }
        Resource resource = resourceSet.getResource(uri, true);

in maven it never does

cdietrich commented 1 year ago

diffs in normalizationMaps

{platform:/resource/org.xtext.example.mydsl/src/org/xtext/example/base2/Base2.xtext=platform:/resource/org.xtext.example.mydsl/src/org/xtext/example/base2/Base2.xtext, file:/home/dietrich/eclipse-workspaces/202212m3/org.eclipse.xtext/bin/org/eclipse/xtext/common/Terminals.xtext=file:/home/dietrich/eclipse-workspaces/202212m3/org.eclipse.xtext/bin/org/eclipse/xtext/common/Terminals.xtext, file:/home/dietrich/Downloads/org.xtext.example.mydsl.parent/org.xtext.example.mydsl/target/classes/org/xtext/example/base/Base.xtext=file:/home/dietrich/Downloads/org.xtext.example.mydsl.parent/org.xtext.example.mydsl/target/classes/org/xtext/example/base/Base.xtext, http://www.xtext.org/example/base/MyBase=http://www.xtext.org/example/base/MyBase, http://www.xtext.org/example/base/MyBase2=http://www.xtext.org/example/base/MyBase2}

{platform:/resource/org.xtext.example.mydsl/src/org/xtext/example/base2/Base2.xtext=platform:/resource/org.xtext.example.mydsl/src/org/xtext/example/base2/Base2.xtext}

there is also org.eclipse.xtext.xtext.generator.ecore.EMFGeneratorFragment2.cloneGrammarIntoNewResourceSet(Grammar)

org.eclipse.xtext.xtext.XtextLinkingService.getUsedGrammar(Grammar, INode) does some fallback to classpath uris, maybe that explains that it is working in eclipse

S-wald commented 1 year ago

The problem can be solved by putting the src-folder as an additionalClassPathElement inside the configuration of the mwe2Launcher execution in the pom.xml:

<additionalClasspathElements>
  <additionalClasspathElement>${project.basedir}/src</additionalClasspathElement>
</additionalClasspathElements>
cdietrich commented 1 year ago

we should think about this anyway. cc @szarnekow

cosminpolifronie commented 3 months ago

@cdietrich @szarnekow Can confirm @S-wald solution fixed my issue as well. Also, it removes the need to define multiple referencedResource for each language in the .mwe2 file (in my case the grammars couldn't resolve references to other grammars).

Everything just worked inside Eclipse. In Maven... you need the fix above. Maybe this could be fixed somehow inside Xtext.

More details: https://stackoverflow.com/q/77236560

szarnekow commented 3 months ago

When running the workflow from Eclipse, the grammars have already been copied to the binary output folder - they are at that point on the classpath. In mvn, the generator runs before anything was copied to the output folder.

A workaround is to not use classpath URIs in the workflow but point to the grammar in the source folders. In any case. I'm happy that you found a solution.

LorenzoBettini commented 3 months ago

@szarnekow I think there's another problem here if I understand the context correctly: the exec-maven-plugin has no such concept as the "sources are part of the classpath of the current project" because it's a general-purpose plugin. In fact, as reported in the documentation, when it is used to run a code generator, like in our case, there are a few suggested options to force the current project's element be part of the classpath, e.g., https://www.mojohaus.org/exec-maven-plugin/java-mojo.html#addResourcesToClasspath (unfortunately there's no addSourcesToClasspath) and https://www.mojohaus.org/exec-maven-plugin/java-mojo.html#sourceRoot, where it is explicitly stated "This folder is added to the list of those folders containing source to be compiled. Use this if your plugin generates source code."

From our side, adding this

<additionalClasspathElements>
  <additionalClasspathElement>${project.basedir}/src</additionalClasspathElement>
</additionalClasspathElements>

to the POM generated by the wizard might be a good solution.

Again, if I understand the context correctly ;)