search

vlingo / xoom-schemata

The VLINGO XOOM Schema Registry.
https://vlingo.io
Mozilla Public License 2.0
17 stars 9 forks source link

ANTLR parser is failing when one spec depends on another #183

Closed hurelhuyag closed 3 years ago

hurelhuyag commented 3 years ago

For example, currently, schemata can't parse this VSS file.

data ContactInformation {
  version semanticVersion
  Vlingo:Developers:io.vlingo.developers.petclinic:PostalAddress:1.0.0 postalAddress
  Vlingo:Developers:io.vlingo.developers.petclinic:Telephone:1.0.0 telephone
}

Stacktrace:

line 3:22 no viable alternative at input 'Vlingo:Developers:io.'
line 4:22 no viable alternative at input 'Vlingo:Developers:io.'
12:38:08.860 [pool-2-thread-2] ERROR io.vlingo.xoom.actors.Logger - EXCEPTION: java.lang.NullPointerException
java.lang.NullPointerException: null
    at io.vlingo.xoom.schemata.codegen.parser.AntlrTypeParser.parseComplexTypeAttribute(AntlrTypeParser.java:158)
    at io.vlingo.xoom.schemata.codegen.parser.AntlrTypeParser.parseFieldDefinition(AntlrTypeParser.java:89)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
    at io.vlingo.xoom.schemata.codegen.parser.AntlrTypeParser.parseTypeDeclaration(AntlrTypeParser.java:80)
    at io.vlingo.xoom.schemata.codegen.parser.AntlrTypeParser.parseTypeDefinition(AntlrTypeParser.java:60)
    at io.vlingo.xoom.schemata.codegen.TypeDefinitionCompilerActor.compile(TypeDefinitionCompilerActor.java:37)
    at io.vlingo.xoom.schemata.resource.CodeResource.compile(CodeResource.java:135)
    at io.vlingo.xoom.schemata.resource.CodeResource.lambda$queryCodeForLanguage$0(CodeResource.java:93)
    at io.vlingo.xoom.common.completes.FutureCompletes$State.lambda$functionWrapper$8(FutureCompletes.java:576)
    at io.vlingo.xoom.common.completes.FutureCompletes$State.lambda$composableFunction$5(FutureCompletes.java:496)
    at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:966)
    at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:940)
    at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:457)
    at java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
12:38:08.860 [pool-2-thread-2] ERROR io.vlingo.xoom.actors.Logger - Exception thrown by Resource execution
VaughnVernon commented 3 years ago

@hurelhuyag This is not a problem with events that use data.

hurelhuyag commented 3 years ago

@VaughnVernon Same exception raising with this

event PetRegistered {
  version semanticVersion
  string id
  Vlingo:Developers:io.vlingo.developers.petclinic:Name:1.0.0 name
  long birth
  Vlingo:Developers:io.vlingo.developers.petclinic:Kind:1.0.0 kind
  Vlingo:Developers:io.vlingo.developers.petclinic:Owner:1.0.0 owner
}
VaughnVernon commented 3 years ago

@hurelhuyag Why? Please fix the problem.

/cc @danilo-ambrosio

danilo-ambrosio commented 3 years ago

@VaughnVernon I could not reproduce this problem. The only thing I've found on the pet clinic example was a difference between the org name in the pom and in the VSS files. This inconsistency causes an error while pushing the schemas. Once I fixed the org name in the pom file., all schemas (including the ones that depend on another) were successfully pushed.
The image below shows all pet clinic schemas that I published during the tests: image

danilo-ambrosio commented 3 years ago

@VaughnVernon If you get a chance, just in case, start the Schemata service then pull the fixed xoom-petclinic-2 and run mvn clean install. If the build succeeds, we can close this issue.

VaughnVernon commented 3 years ago

@danilo-ambrosio Test and confirmed. No problems on my side.

@hamzajg Looks like this is not a real problem.

VaughnVernon commented 3 years ago

Reopening for possible secondary cause. Seeking test case to find how it can happen and how it can be fixed.

/cc @hamzajg

VaughnVernon commented 3 years ago

Confirmed by @hamzajg that @danilo-ambrosio fix was the same one he was pursuing.

VaughnVernon commented 3 years ago

This is a problem when parsing the schema version in order to generate Java or C# code. I've investigated this and generated a decent error message. The parser doesn't support type references that are either fully-qualified or prefixed with category type.

Apparently the definition isn't parsed before schema-push, otherwise the same definitions wouldn't be pushed in.

I will push the error message support.

/cc @jakzal @danilo-ambrosio

VaughnVernon commented 3 years ago

I added better parser error output, as follows:

00:55:33.272 [pool-2-thread-7] ERROR io.vlingo.xoom.actors.Logger - Parsing failed for schema: xoom:apps:io.vlingo.xoom.examples.petclinic:VeterinarianRegistered:1.0.0
Error [4,2]: mismatched input 'xoom' expecting {'boolean', 'byte', 'char', 'double', 'float', 'int', 'long', 'short', 'string', 'timestamp', 'type', 'version', '=', '}', TYPE_IDENTIFIER}null
org.antlr.v4.runtime.DefaultErrorStrategy.recoverInline(DefaultErrorStrategy.java:485)
org.antlr.v4.runtime.Parser.match(Parser.java:206)
io.vlingo.xoom.schemata.codegen.antlr.SchemaVersionDefinitionParser.typeBody(SchemaVersionDefinitionParser.java:276)
io.vlingo.xoom.schemata.codegen.antlr.SchemaVersionDefinitionParser.typeDeclaration(SchemaVersionDefinitionParser.java:140)
io.vlingo.xoom.schemata.codegen.parser.AntlrTypeParser.parseTypeDefinition(AntlrTypeParser.java:60)
io.vlingo.xoom.schemata.codegen.TypeDefinitionCompilerActor.compile(TypeDefinitionCompilerActor.java:37)
io.vlingo.xoom.schemata.resource.CodeResource.compile(CodeResource.java:135)
io.vlingo.xoom.schemata.resource.CodeResource.lambda$queryCodeForLanguage$0(CodeResource.java:93)
io.vlingo.xoom.common.completes.FutureCompletes$State.lambda$functionWrapper$8(FutureCompletes.java:576)
io.vlingo.xoom.common.completes.FutureCompletes$State.lambda$composableFunction$5(FutureCompletes.java:496)
java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:952)
java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:443)
java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Also every time a new type is parsed and instantiated as an object, it's definition is printed to the console. (This can be removed after debugging.) From this example you can see that the type is not fully loaded because the complex types are missing (not even sure why it gets that far).

==========
TypeDefinition [category=Event, fullyQualifiedTypeName=xoom:apps:io.vlingo.xoom.examples.petclinic:VeterinarianRegistered:1.0.0, typeName=VeterinarianRegistered, children=[FieldDefinition [type=BasicType [typeName=version], version=Optional.empty, name=semanticVersion, defaultValue=Optional.empty], FieldDefinition [type=BasicType [typeName=string], version=Optional.empty, name=id, defaultValue=Optional[NullValue []]]]]
==========

The VeterinarianRegistered event should have all of the following, but only the semanticVersion and id are seen in the above TypeDefinition log:

event VeterinarianRegistered {
  version semanticVersion
  string id
  xoom:apps:io.vlingo.xoom.examples.petclinic:FullName:1.0.0 name
  xoom:apps:io.vlingo.xoom.examples.petclinic:ContactInformation:1.0.0 contactInformation
  xoom:apps:io.vlingo.xoom.examples.petclinic:Specialty:1.0.0 specialty
}
jakzal commented 3 years ago

The way parser rules are currently defined require that:

https://github.com/vlingo/xoom-schemata/blob/c0ece4cde836412c205b7db8eb392d44b639fb89/src/main/antlr4/io/vlingo/xoom/schemata/codegen/antlr/SchemaVersionDefinitionLexer.g4#L128-L138

Part of the problem might be that the top level type definition:

https://github.com/vlingo/xoom-schemata/blob/c0ece4cde836412c205b7db8eb392d44b639fb89/src/main/antlr4/io/vlingo/xoom/schemata/codegen/antlr/SchemaVersionDefinitionParser.g4#L12-L14

https://github.com/vlingo/xoom-schemata/blob/c0ece4cde836412c205b7db8eb392d44b639fb89/src/main/antlr4/io/vlingo/xoom/schemata/codegen/antlr/SchemaVersionDefinitionParser.g4#L25-L27

is shared with attribute type definition:

https://github.com/vlingo/xoom-schemata/blob/c0ece4cde836412c205b7db8eb392d44b639fb89/src/main/antlr4/io/vlingo/xoom/schemata/codegen/antlr/SchemaVersionDefinitionParser.g4#L64-L67

Perhaps decoupling the two type definition is necessary to make the attribute type definition more elastic.

VaughnVernon commented 3 years ago

@jakzal The schema definitions now support only category.Type, category.Type:m.m.p, etc.

VaughnVernon commented 3 years ago

Fixed by #191 #192 #193 #194 #195