JPL-IMCE / gov.nasa.jpl.imce.oml

Ontology Modeling Language (OML) Workbench
14 stars 1 forks source link

Problems serializing OML raw strings with a trailing '"' character. #239

Closed NicolasRouquette closed 6 years ago

NicolasRouquette commented 6 years ago
java.lang.RuntimeException: Error while converting value 'Latch Valve: LP - 3/8"' for grammar rule 'QUOTED_STRING_VALUE' to string via ValueConverter.
Caused By: org.eclipse.xtext.conversion.ValueConverterException: The value 'Latch Valve: LP - 3/8"' is an invalid QUOTED_STRING_VALUE
Semantic Object: Extent.modules[0]->TerminologyGraph.annotations[0]->AnnotationPropertyValue.value->LiteralQuotedString
URI: file%3AOMLAnnoationTest3.oml
EStructuralFeature: omlc::LiteralQuotedString.string
    at org.eclipse.xtext.serializer.diagnostic.ISerializationDiagnostic$ExceptionThrowingAcceptor.accept(ISerializationDiagnostic.java:129)
    at org.eclipse.xtext.serializer.tokens.ValueSerializer.serializeAssignedValue(ValueSerializer.java:82)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.getToken(SequenceFeeder.java:489)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.accept(SequenceFeeder.java:238)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence_LiteralQuotedString(OMLSemanticSequencer.java:937)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence(OMLSemanticSequencer.java:162)
    at org.eclipse.xtext.serializer.sequencer.AbstractSemanticSequencer.createSequence(AbstractSemanticSequencer.java:67)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptEObjectRuleCall(SequenceFeeder.java:325)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptRuleCall(SequenceFeeder.java:352)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.accept(SequenceFeeder.java:239)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence_AnnotationPropertyValue(OMLSemanticSequencer.java:368)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence(OMLSemanticSequencer.java:144)
    at org.eclipse.xtext.serializer.sequencer.AbstractSemanticSequencer.createSequence(AbstractSemanticSequencer.java:67)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptEObjectRuleCall(SequenceFeeder.java:325)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptRuleCall(SequenceFeeder.java:352)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.accept(SequenceFeeder.java:263)
    at org.eclipse.xtext.serializer.sequencer.BacktrackingSemanticSequencer.accept(BacktrackingSemanticSequencer.java:434)
    at org.eclipse.xtext.serializer.sequencer.BacktrackingSemanticSequencer.createSequence(BacktrackingSemanticSequencer.java:501)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence_TerminologyGraph(OMLSemanticSequencer.java:1527)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence(OMLSemanticSequencer.java:225)
    at org.eclipse.xtext.serializer.sequencer.AbstractSemanticSequencer.createSequence(AbstractSemanticSequencer.java:67)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptEObjectRuleCall(SequenceFeeder.java:325)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.acceptRuleCall(SequenceFeeder.java:352)
    at org.eclipse.xtext.serializer.acceptor.SequenceFeeder.accept(SequenceFeeder.java:263)
    at org.eclipse.xtext.serializer.sequencer.BacktrackingSemanticSequencer.accept(BacktrackingSemanticSequencer.java:434)
    at org.eclipse.xtext.serializer.sequencer.BacktrackingSemanticSequencer.createSequence(BacktrackingSemanticSequencer.java:501)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence_Extent(OMLSemanticSequencer.java:794)
    at gov.nasa.jpl.imce.oml.dsl.serializer.OMLSemanticSequencer.sequence(OMLSemanticSequencer.java:147)
    at org.eclipse.xtext.serializer.sequencer.AbstractSemanticSequencer.createSequence(AbstractSemanticSequencer.java:67)
    at org.eclipse.xtext.serializer.impl.Serializer.serialize(Serializer.java:115)
    at org.eclipse.xtext.serializer.impl.Serializer.serializeToRegions(Serializer.java:136)
    at org.eclipse.xtext.serializer.impl.Serializer.serialize(Serializer.java:142)
    at org.eclipse.xtext.serializer.impl.Serializer.serialize(Serializer.java:189)
    at org.eclipse.xtext.resource.XtextResource.doSave(XtextResource.java:386)
    at org.eclipse.emf.ecore.resource.impl.ResourceImpl.save(ResourceImpl.java:1430)
...
Caused by: org.eclipse.xtext.conversion.ValueConverterException: The value 'Latch Valve: LP - 3/8"' is an invalid QUOTED_STRING_VALUE
    at org.eclipse.xtext.conversion.impl.AbstractLexerBasedConverter.createTokenContentMismatchException(AbstractLexerBasedConverter.java:82)
    at gov.nasa.jpl.imce.oml.dsl.util.GenericValueConverter.assertTokens(GenericValueConverter.java:42)
    at org.eclipse.xtext.conversion.impl.AbstractLexerBasedConverter.assertTokens(AbstractLexerBasedConverter.java:58)
    at org.eclipse.xtext.conversion.impl.AbstractLexerBasedConverter.toString(AbstractLexerBasedConverter.java:49)
    at gov.nasa.jpl.imce.oml.dsl.util.OMLValueConverterService.toString(OMLValueConverterService.java:205)
    at org.eclipse.xtext.serializer.tokens.ValueSerializer.serializeAssignedValue(ValueSerializer.java:74)
    ... 58 more
NicolasRouquette commented 6 years ago

Roughly, this is enough to reproduce this:

        def void test() {

        val rs = resourceSetProvider.get
        val r = rs.createResource(URI.createFileURI("file:OMLAnnoationTest3.oml"))

        val Extent e = commonF.createExtent()
        r.getContents.add(e)

        val TerminologyGraph g = graphsF.createTerminologyGraph()
        e.getModules.add(g)
        g.setIri("http://www.example.org/OMLAnnotationTest3")

        val AnnotationProperty ap1 = commonF.createAnnotationProperty
        ap1.module = g
        ap1.abbrevIRI = "test:doc1"
        ap1.iri = "http://www.example.org/OMLAnnotationTest3#doc1"

        addAnnotation(g, ap1, '''Latch Valve: LP - 3/8''')

        val ByteArrayOutputStream bos = new ByteArrayOutputStream()
        val Builder builder = SaveOptions.newBuilder()
        builder.format()
        val SaveOptions s = builder.getOptions();
        r.save(bos, s.toOptionsMap());
        }

    def void addAnnotation(LogicalElement e, AnnotationProperty ap, String v) {
        val AnnotationPropertyValue av = commonF.createAnnotationPropertyValue
        e.annotations += av
        av.property = ap
        val s = commonF.createLiteralQuotedString()
        s.string = new QuotedStringValue(v)
        av.value = s
    }
NicolasRouquette commented 6 years ago

This turns out to be a limitation of the Xtext 2.12 grammar where the rule for raw strings is non-greedy. For technical details, see: https://www.eclipse.org/forums/index.php/m/1781919/#msg_1781919

NicolasRouquette commented 6 years ago

Christian Dietrich suggests switching to a custom lexer: https://www.eclipse.org/forums/index.php?t=msg&th=1091720&goto=1781921&#msg_1781921

He wrote a blog article about this: https://typefox.io/taming-the-lexer

And provided some concrete suggestions how to do this: https://www.eclipse.org/forums/index.php?t=msg&th=1091720&goto=1781926&#msg_1781926

A possible short-term hack would be to run the current lexer generator and post-process the generated output to change the mRULE_RAW_STRING_VALUE() method as described here: https://www.eclipse.org/forums/index.php?t=msg&th=1091720&goto=1781919&#msg_1781919

Another option is to follow Christian's blog article and switch from Antlr3 to Antlr4 or JFlex.

NicolasRouquette commented 6 years ago

Switching to a custom lexer makes sense; however, this Xtext surgery needs to be deferred. For now, this issue can be reasonably resolved with a simple workaround:

In OML, make sure that the last character of an OML RawString is anything but ".