search

Apicurio / apicurio-registry

An API/Schema registry - stores APIs and Schemas.
https://www.apicur.io/registry/
Apache License 2.0
596 stars 265 forks source link

Allow to upload new version of artifact with compatibility rules enabled. #1686

Open tillias opened 3 years ago

tillias commented 3 years ago

Hello folks,

I have version 1 of Artifact (JsonSchema) without timestamp field. Whenever I activate any compatibility rule and try uploading new version with optional timestamp I got exception: 

org.everit.json.schema.SchemaException: #: could not determine version
    at org.everit.json.schema.loader.SchemaLoader.<init>(SchemaLoader.java:338)
    at org.everit.json.schema.loader.SchemaLoader$SchemaLoaderBuilder.build(SchemaLoader.java:149)
    at io.apicurio.registry.rules.compatibility.jsonschema.JsonSchemaDiffLibrary.findDifferences(JsonSchemaDiffLibrary.java:54)
    at io.apicurio.registry.rules.compatibility.jsonschema.JsonSchemaDiffLibrary.getIncompatibleDifferences(JsonSchemaDiffLibrary.java:76)
    at io.apicurio.registry.rules.compatibility.JsonSchemaCompatibilityChecker.testCompatibility(JsonSchemaCompatibilityChecker.java:57)
    at io.apicurio.registry.rules.compatibility.CompatibilityChecker.testCompatibility(CompatibilityChecker.java:48)
    at io.apicurio.registry.rules.compatibility.CompatibilityRuleExecutor.execute(CompatibilityRuleExecutor.java:64)
    at io.apicurio.registry.rules.compatibility.CompatibilityRuleExecutor_Subclass.execute$$superaccessor2(CompatibilityRuleExecutor_Subclass.zig:350)
    at io.apicurio.registry.rules.compatibility.CompatibilityRuleExecutor_Subclass$$function$$2.apply(CompatibilityRuleExecutor_Subclass$$function$$2.zig:33)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
    at io.apicurio.registry.logging.LoggingInterceptor.logMethodEntry(LoggingInterceptor.java:55)
    at io.apicurio.registry.logging.LoggingInterceptor_Bean.intercept(LoggingInterceptor_Bean.zig:275)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
    at io.apicurio.registry.rules.compatibility.CompatibilityRuleExecutor_Subclass.execute(CompatibilityRuleExecutor_Subclass.zig:298)
    at io.apicurio.registry.rules.compatibility.CompatibilityRuleExecutor_ClientProxy.execute(CompatibilityRuleExecutor_ClientProxy.zig:129)
    at io.apicurio.registry.rules.RulesServiceImpl.applyRule(RulesServiceImpl.java:130)
    at io.apicurio.registry.rules.RulesServiceImpl.applyGlobalAndArtifactRules(RulesServiceImpl.java:97)
    at io.apicurio.registry.rules.RulesServiceImpl.applyRules(RulesServiceImpl.java:71)
    at io.apicurio.registry.rules.RulesServiceImpl_ClientProxy.applyRules(RulesServiceImpl_ClientProxy.zig:214)
    at io.apicurio.registry.rest.v2.GroupsResourceImpl.createArtifactVersion(GroupsResourceImpl.java:583)
    at io.apicurio.registry.rest.v2.GroupsResourceImpl_Subclass.createArtifactVersion$$superaccessor13(GroupsResourceImpl_Subclass.zig:3613)
    at io.apicurio.registry.rest.v2.GroupsResourceImpl_Subclass$$function$$13.apply(GroupsResourceImpl_Subclass$$function$$13.zig:53)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:54)
    at io.smallrye.metrics.interceptors.CountedInterceptor.countedCallable(CountedInterceptor.java:95)
    at io.smallrye.metrics.interceptors.CountedInterceptor.countedMethod(CountedInterceptor.java:70)
    at io.smallrye.metrics.interceptors.CountedInterceptor_Bean.intercept(CountedInterceptor_Bean.zig:366)
    at io.quarkus.arc.impl.InitializedInterceptor.intercept(InitializedInterceptor.java:79)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
    at io.smallrye.metrics.interceptors.TimedInterceptor.timedCallable(TimedInterceptor.java:95)
    at io.smallrye.metrics.interceptors.TimedInterceptor.timedMethod(TimedInterceptor.java:70)
    at io.smallrye.metrics.interceptors.TimedInterceptor_Bean.intercept(TimedInterceptor_Bean.zig:366)
    at io.quarkus.arc.impl.InitializedInterceptor.intercept(InitializedInterceptor.java:79)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
    at io.smallrye.metrics.interceptors.ConcurrentGaugeInterceptor.concurrentCallable(ConcurrentGaugeInterceptor.java:96)
    at io.smallrye.metrics.interceptors.ConcurrentGaugeInterceptor.countedMethod(ConcurrentGaugeInterceptor.java:69)
    at io.smallrye.metrics.interceptors.ConcurrentGaugeInterceptor_Bean.intercept(ConcurrentGaugeInterceptor_Bean.zig:366)
    at io.quarkus.arc.impl.InitializedInterceptor.intercept(InitializedInterceptor.java:79)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
    at io.apicurio.registry.auth.AuthorizedInterceptor.authorizeMethod(AuthorizedInterceptor.java:76)
    at io.apicurio.registry.auth.AuthorizedInterceptor_Bean.intercept(AuthorizedInterceptor_Bean.zig:786)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
    at io.apicurio.registry.metrics.RestMetricsInterceptor.intercept(RestMetricsInterceptor.java:82)
    at io.apicurio.registry.metrics.RestMetricsInterceptor_Bean.intercept(RestMetricsInterceptor_Bean.zig:327)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:50)
    at io.apicurio.registry.logging.LoggingInterceptor.logMethodEntry(LoggingInterceptor.java:55)
    at io.apicurio.registry.logging.LoggingInterceptor_Bean.intercept(LoggingInterceptor_Bean.zig:275)
    at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:41)
    at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:41)
    at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:32)
    at io.apicurio.registry.rest.v2.GroupsResourceImpl_Subclass.createArtifactVersion(GroupsResourceImpl_Subclass.zig:3564)
    at io.apicurio.registry.rest.v2.GroupsResourceImpl_ClientProxy.createArtifactVersion(GroupsResourceImpl_ClientProxy.zig:690)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
    at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
    at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:643)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:507)
    at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:457)
    at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:459)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:419)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:393)
    at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:68)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:261)
    at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:161)
    at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
    at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:164)
    at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:247)
    at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:249)
    at io.quarkus.resteasy.runtime.ResteasyFilter.doFilter(ResteasyFilter.java:35)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at io.apicurio.registry.ui.servlets.ResourceCacheControlFilter.doFilter(ResourceCacheControlFilter.java:83)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at io.apicurio.registry.rest.RegistryApplicationServletFilter.doFilter(RegistryApplicationServletFilter.java:113)
    at io.apicurio.registry.rest.RegistryApplicationServletFilter_ClientProxy.doFilter(RegistryApplicationServletFilter_ClientProxy.zig:225)
    at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
    at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
    at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:63)
    at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
    at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:67)
    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:133)
    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:65)
    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
    at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
    at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:247)
    at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:56)
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:111)
    at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:108)
    at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
    at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
    at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$9$1.call(UndertowDeploymentRecorder.java:574)
    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227)
    at io.undertow.servlet.handlers.ServletInitialHandler.handleRequest(ServletInitialHandler.java:152)
    at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$1.handleRequest(UndertowDeploymentRecorder.java:117)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:290)
    at io.undertow.server.DefaultExchangeHandler.handle(DefaultExchangeHandler.java:18)
    at io.quarkus.undertow.runtime.UndertowDeploymentRecorder$5$1.run(UndertowDeploymentRecorder.java:400)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at org.jboss.threads.JBossThread.run(JBossThread.java:501)

Here is schema I'm using:

{
  "$id": "https://raw.githubusercontent.com/tillias/api-first-insurance/main/claim-accepted-schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "ClaimAccepted",
  "type": "object",
  "properties": {
    "claimId": {
      "type": "string",
      "description": "Id of a claim"
    },
    "email": {
      "type": "string",
      "format": "email",
      "description": "Official in charge"
    },
    "timestamp": {
      "type": "string",
      "format": "date-time",
      "description": "indicates timestamp when claim was accepted by official in charge"
    },
    "required": ["claimId", "email"]
  },
  "examples": [
    {
      "claimId": "ABC-123",
      "email": "john-doe@foo.com"
    }
  ]
}
stephennimmo commented 1 year ago

This continues to be a problem and should be addressed. The problem is the underlying library being used does not support the newer version of the schema definition and does NOT plan on doing so. https://github.com/everit-org/json-schema/issues/441#issuecomment-949585872

If a customer comes with schemas using 2020-12, then using the product is a non-starter.

jsenko commented 1 year ago

We're working on our own JSON schema library, since we did not find an existing one suitable for our needs. We'll make it a priority.

Mainstay-Noah-Huppert commented 1 year ago

I've found a workaround is to use Draft 7 ("$schema": "https://json-schema.org/draft-07/schema"). Of course then you can't use 2020-12 features but at least you can use apicurio.

EricWittmann commented 1 year ago

Yeah - thanks @Mainstay-Noah-Huppert for updating this. The library we're currently using for this functionality does not support the most recent versions of JSON Schema. As @jsenko reported we have plans to write our own, but we have limited bandwidth for this activity at the moment.

stephennimmo commented 1 year ago

Understood. Perhaps a quick win would be an initial submit validation for json schemas that looks at the version and provides feedback regarding support.

What’s weird about this is the error doesn’t manifest until after you attempt to upload a new version. This could put users into an awkward position as they won’t have any warning of the issues until possibly after the schema is being used in production.

janvyhnanek commented 1 year ago

Hi, I have run into a weird behaviour related to this issue. I have a global rule in registry to check FULL schema validity and if I try to upload a new version of json schema (2020-12), I get the same error as above. But what is strange - if I set schema validity rule on the artifact itself, the registration is suddenly working :) And it seems that the validation itself is working, at least for some json schema constructions (e.g. if I modify type string to string2, schema registry returns validation error).

Does it happen to anyone else as well? We are going to use it as a workaround - if a new json schema of version 2020-12 is registered, the validation rule has to be set explicitely on it and then everything seems to be working.