confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.23k stars 1.11k forks source link

Error registering Avro schema: Register schema operation failed while writing to the Kafka store; error code: 50001 #1176

Open jdang67 opened 5 years ago

jdang67 commented 5 years ago

Hello Everyone,

I have a weird issue to send an object that generated from the .avsc file.
On my local, there is no problem but on the dev server, it throws an exception when I use producer to send this type of record to any topic or the streaming app to forward to a topic. This type of object already consumes by many other apps.

thanks, Jdang

Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Register schema operation failed while writing to the Kafka store; error code: 50001

{
  "type": "record",
  "name": "object-name",
  "namespace": "com.compnay.schemas.java",
  "fields": [
    {
      "name": "message_type",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "repeat",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "scaled",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "turn",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "speed",
      "type": [
        "null",
        "double"
      ],
      "default": null
    },
    {
      "name": "second",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "maneuver",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "radio",
      "type": [
        "null",
        "int"
      ],
      "default": null
    },
    {
      "name": "source_id",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "status",
      "type": [
        "null",
        "int"
      ],
      "default": null
    }
  ]
}

Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Register schema operation failed while writing to the Kafka store; error code: 50001
    at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:209)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:235)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:326)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:318)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:313)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:119)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:156)
    at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:79)
    at io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53)
    at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:65)
    at io.confluent.kafka.streams.serdes.avro.SpecificAvroSerializer.serialize(SpecificAvroSerializer.java:38)
    at org.apache.kafka.common.serialization.Serializer.serialize(Serializer.java:60)
    at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.send(RecordCollectorImpl.java:157)
    at org.apache.kafka.streams.processor.internals.RecordCollectorImpl.send(RecordCollectorImpl.java:101)
    at org.apache.kafka.streams.processor.internals.SinkNode.process(SinkNode.java:89)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:146)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:129)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:93)
    at org.apache.kafka.streams.kstream.internals.KStreamMap$KStreamMapProcessor.process(KStreamMap.java:42)
    at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:117)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:146)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:129)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:93)
    at org.apache.kafka.streams.kstream.internals.KStreamPassThrough$KStreamPassThroughProcessor.process(KStreamPassThrough.java:33)
    at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:117)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:146)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:125)
    at org.apache.kafka.streams.kstream.internals.KStreamBranch$KStreamBranchProcessor.process(KStreamBranch.java:48)
    at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:117)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:146)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:129)
    at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:93)
    at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:84)
    at org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:351)
    at org.apache.kafka.streams.processor.internals.AssignedStreamsTasks.process(AssignedStreamsTasks.java:104)
    at org.apache.kafka.streams.processor.internals.TaskManager.process(TaskManager.java:413)
    at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:862)
    at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:777)
    at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:747)
alokpaul commented 5 years ago

Seeing the same issue. There is no details on the schema registry log. even in debug mode. what could be the cause of this. I see some post saying the error may be caused by having "-" in schema name but no that issue.

tsrikanth06 commented 5 years ago

Any update on this issue, my producer is encountering the same issue. Its not a schema issue as I was able to post the same schema successfully using post (SCHEMA_REGSTRY_URL + "/subjects/" + subjectName + "/versions/). But It fails when the producer is trying to write data and when it tries to post schema.

tomaszszlek commented 4 years ago

I have the same issue, it happens from time to time, the logs are as follows:

io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Register schema operation failed while writing to the Kafka store; error code: 50001 at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:202) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:229) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:320) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:312) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:307) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:115) ~[kafka-schema-registry-client-4.1.0.jar:na] at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:154) ~[kafka-schema-registry-client-4.1.0.jar:na]

andrewkandzuba commented 4 years ago

Same here:

Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Register schema operation failed while writing to the Kafka store; error code: 50001
    at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:209)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:235)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:326)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:318)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:313)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:119)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:156)

There is no way for steady reproducing of the issue so I anticipate some data racing conditions in KafkaStore class. Can you add some logging here: https://github.com/confluentinc/schema-registry/blob/master/core/src/main/java/io/confluent/kafka/schemaregistry/storage/KafkaStore.java#L358?

jdang67 commented 4 years ago

I am not sure how about the others but in my case, we reordered the properties within the schema causing that exception, the ordering of the fields within the schema is important. I hope this might help someone.

For examples, the following 2 schemas are NOT the same

  "fields": [
    {
      "name": "id",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "name",
      "type": [
        "null",
        "string"
      ],
      "default": null
    }
]
 "fields": [
    {
      "name": "name",
      "type": [
        "null",
        "string"
      ],
      "default": null
    },
    {
      "name": "id",
      "type": [
        "null",
        "string"
      ],
      "default": null
    }
]
andrewkandzuba commented 4 years ago

@jdang67 - Confluent has different exceptions to classify issues. Had it be something wrong with the schema itself it should (a) has a steady reproduction and (b) different exception to be thrown.

Anyway do not want to spread a guesswork ...

jdang67 commented 4 years ago

@jdang67 - Confluent has different exceptions to classify issues. Had it be something wrong with the schema itself it should (a) has a steady reproduction and (b) different exception to be thrown.

Anyway do not want to spread a guesswork ...

Well, we stuck with that exception for weeks, until we rolled back our schema to the previous version then the problem went away.

Jdang

andrewkandzuba commented 4 years ago

@jdang67 - FYI we had this problem with 5.0.x in the past, then no issues with 5.2.x and now it reappeared again with 5.3.x. Strange thing I do not trace much changes between this two versions i.e. 5.2.x vs 5.3.x (at least in that KafkaSchemaRegistry class).

johnstonpn commented 4 years ago

I was running into the same problem on 5.3.1. Per comment above, I tried reverting to 5.2.3 and continued to see the same intermittent error during schema registration, so it didn't appear to be version specific.

What DID appear to resolve the issue was increasing the memory and cpu resources. After I removed some fairly low cpu/memory limits across the confluent components, this error went away. I was able to register schemas with no error on the first attempt. This was a quick and dirty solution - it would be better to do more detail resource utilization analysis and see exactly what was maxing out and where (schema registry, broker response for storing the schemas, etc.) at the time the error occurs.

I'd recommend looking closely at resource utilization across your cluster at the time you see this error and adjust up accordingly.

kaushiksrinivas commented 4 years ago

One of the instances where we have seen this issue was when running schema registry version 5.1.0 or lower. We have seen this after kafka broker restart in case of kubernetes deployment, the schema registry fails to register schema with 50001 kafka store failure. This is something to do with the DNS resolution of the kafka broker after restart. kafka clients have undergone a fix with the issue here : https://issues.apache.org/jira/browse/KAFKA-7755 prior to this bug fix, the kafka clients (schema registry in this case, and master schema pod mostly) needed a manual restart to get back to its operations with kafka. but with this fix, this issue has been taken care of. So from schema registry 5.1.1 onwards we can expect this error not to be happening due to the DNS resolve issue upon kafka broker restart.

danistrebel commented 4 years ago

I ran into this issue with the new Schema Registry 5.3.x and Client tools 5.3.x. inside Kubernetes. Apparently the client is able to resolve the Kafka Bootstrap URL via DNS but not the Schema Registry. The solution for me is to use the svc.cluster.local form of the schema registry service hostname.

GiridharSamanth commented 3 years ago

A simple restart of schema-registry had solved the problem for me.

Valentin-archi commented 3 years ago

Hello! I encountered the exact same problem o.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Register schema operation failed while writing to the Kafka store; error code: 50001 at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:295) at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:365) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:508) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:499) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:472) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:213) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:275) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:251) at io.conduktor.schemaregistry.SchemaRegistryController$createOrUpdateSubject$4$1.invokeSuspend(SchemaRegistryController.kt:358) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)

Attempts to create a scheme made using conductor, akhq and curl - POST request.

Example:

curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
--data '{"schema": "{\"type\":\"record\",\"name\":\"CarrierContract\",\"namespace\":\"kafka.types.tms\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"vendorId\",\"type\":\"string\"},{\"name\":\"code\",\"type\":\"string\"},{\"name\":\"startingDate\",\"type\":\"string\"},{\"name\":\"VAT\",\"type\":\"int\"},{\"name\":\"endingDate\",\"type\":\"string\"},{\"name\":\"extensionDate\",\"type\":\"string\"}],\"connect.name\":\"kafka.types.tms.CarrierContract\"}"}' \
http://******:8081/subjects/Carrier_Contract-value/versions

Result {"error_code":50001,"message":"Register schema operation failed while writing to the Kafka store"}%

For the schema service we ran the debug, at the time of schema creation we see the following errors:

DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=0/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=0,c=false,c=false/false,a=IDLE,uri=null,age=0} filled 689 HeapByteBuffer@53991672[p=0,l=689,c=8192,r=689]={<<<POST /subjects/Ca...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,681] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=0/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=0,c=false,c=false/false,a=IDLE,uri=null,age=0} parse HeapByteBuffer@53991672[p=0,l=689,c=8192,r=689]={<<<POST /subjects/Ca...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} {} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,682] DEBUG parseNext s=START HeapByteBuffer@53991672[p=0,l=689,c=8192,r=689]={<<<POST /subjects/Ca...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG START --> SPACE1 (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG SPACE1 --> URI (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG URI --> SPACE2 (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG SPACE2 --> REQUEST_VERSION (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG REQUEST_VERSION --> HEADER (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:Host --> VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:Host --> IN_VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:Host --> FIELD (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:User-Agent --> VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:User-Agent --> IN_VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,682] DEBUG HEADER:User-Agent --> FIELD (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Accept: */* --> IN_VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Accept: */* --> FIELD (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Type --> VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Type --> IN_VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Type --> FIELD (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Length --> VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Length --> IN_VALUE (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER:Content-Length --> FIELD (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG HEADER --> CONTENT (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,683] DEBUG REQUEST for //**********:8081/subjects/CarrierContractvalue/versions/ on HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=0}
POST //**********:8081/subjects/CarrierContractvalue/versions/ HTTP/1.1
Host: **********:8081
User-Agent: curl/7.68.0
Accept: */*
Content-Type: application/vnd.schemaregistry.v1+json
Content-Length: 477

 (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,683] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=2/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CONTENT,0 of 477},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=0} parsed true HttpParser{s=CONTENT,0 of 477} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,684] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=1} handle //**********:8081/subjects/CarrierContractvalue/versions/  (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,684] DEBUG handling HttpChannelState@34879dcb{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,684] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=1} action DISPATCH (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,684] DEBUG REQUEST POST /subjects/CarrierContractvalue/versions/ on HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=1} (org.eclipse.jetty.server.Server)
[2021-08-25 12:45:44,684] DEBUG GzipHandler@f99f5e0{STARTED} handle Request(POST //**********:8081/subjects/CarrierContractvalue/versions/)@47b0ac54 in null (org.eclipse.jetty.server.handler.gzip.GzipHandler)
[2021-08-25 12:45:44,684] DEBUG scope null||/subjects/CarrierContractvalue/versions/ @ o.e.j.s.ServletContextHandler@51c693d{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
[2021-08-25 12:45:44,684] DEBUG context=||/subjects/CarrierContractvalue/versions/ @ o.e.j.s.ServletContextHandler@51c693d{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
[2021-08-25 12:45:44,684] DEBUG SessionHandler.doScope (org.eclipse.jetty.server.session)
[2021-08-25 12:45:44,684] DEBUG sessionHandler=org.eclipse.jetty.server.session.SessionHandler1784131088==dftMaxIdleSec=-1 session=null (org.eclipse.jetty.server.session)
[2021-08-25 12:45:44,685] DEBUG servlet ||/subjects/CarrierContractvalue/versions/ -> default@5c13d641==org.eclipse.jetty.servlet.DefaultServlet,jsp=null,order=-1,inst=true,async=true (org.eclipse.jetty.servlet.ServletHandler)
[2021-08-25 12:45:44,685] DEBUG chain=org.glassfish.jersey.servlet.ServletContainer-71075444@71075444==org.glassfish.jersey.servlet.ServletContainer,inst=true,async=true->default@5c13d641==org.eclipse.jetty.servlet.DefaultServlet,jsp=null,order=-1,inst=true,async=true (org.eclipse.jetty.servlet.ServletHandler)
[2021-08-25 12:45:44,685] DEBUG call filter org.glassfish.jersey.servlet.ServletContainer-71075444@71075444==org.glassfish.jersey.servlet.ServletContainer,inst=true,async=true (org.eclipse.jetty.servlet.ServletHandler)
[2021-08-25 12:45:44,686] DEBUG startAsync HttpChannelState@34879dcb{s=DISPATCHED a=NOT_ASYNC i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,686] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=5/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CONTENT,0 of 477},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=3} parse HeapByteBuffer@53991672[p=212,l=689,c=8192,r=477]={POST /subjects/Ca...Length: 477\r\n\r\n<<<{"schema": "{\\"ty...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} {} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,686] DEBUG parseNext s=CONTENT HeapByteBuffer@53991672[p=212,l=689,c=8192,r=477]={POST /subjects/Ca...Length: 477\r\n\r\n<<<{"schema": "{\\"ty...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,687] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=4} onContent Content@4781f1af{HeapByteBufferR@556027d4[p=212,l=689,c=8192,r=477]={POST /subjects/Ca...Length: 477\r\n\r\n<<<{"schema": "{\\"ty...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}} (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,681] DEBUG ReservedThreadExecutor@5a18cd76{s=1/6,p=0}@5738009e task=EatWhatYouKill@543e593/SelectorProducer@4e628b52/IDLE/p=true/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.681+03:00 (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,687] DEBUG EatWhatYouKill@543e593/SelectorProducer@4e628b52/IDLE/p=true/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.687+03:00 tryProduce true (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill)
[2021-08-25 12:45:44,687] DEBUG updateable 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,687] DEBUG updates 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,688] DEBUG Key interests updated 1 -> 0 on NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=7/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CONTENT,477 of 477},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=5} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,688] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 waiting with 1 keys (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,687] DEBUG HttpInputOverHTTP@578fac56[c=0,q=0,[0]=null,s=STREAM] addContent Content@4781f1af{HeapByteBufferR@556027d4[p=212,l=689,c=8192,r=477]={POST /subjects/Ca...Length: 477\r\n\r\n<<<{"schema": "{\\"ty...erContract\\"}"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}} (org.eclipse.jetty.server.HttpInput)
[2021-08-25 12:45:44,688] DEBUG CONTENT --> END (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,688] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=5} onContentComplete (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,689] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=6} onRequestComplete (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,689] DEBUG HttpInputOverHTTP@578fac56[c=0,q=1,[0]=EOF,s=STREAM] addContent EOF (org.eclipse.jetty.server.HttpInput)
[2021-08-25 12:45:44,689] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=8/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=END,477 of 477},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=6} parsed false HttpParser{s=END,477 of 477} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,689] DEBUG HttpInputOverHTTP@578fac56[c=477,q=1,[0]=EOF,s=STREAM] read 477 from Content@4781f1af{HeapByteBufferR@556027d4[p=689,l=689,c=8192,r=0]={POST /subjects/Ca...erContract\\"}"}<<<>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}} (org.eclipse.jetty.server.HttpInput)
[2021-08-25 12:45:44,690] DEBUG releaseRequestBuffer HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=9/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=END,477 of 477},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=7} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,691] INFO Wait to catch up until the offset of the last message at 167 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
[2021-08-25 12:45:44,691] DEBUG Reached offset at 167 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
[2021-08-25 12:45:44,692] DEBUG org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor@159f68ff exclude by status 500 (org.eclipse.jetty.server.handler.gzip.GzipHttpOutputInterceptor)
[2021-08-25 12:45:44,692] DEBUG sendResponse info=null content=HeapByteBuffer@5ddbef6a[p=0,l=98,c=8192,r=98]={<<<{"error_code":500...e Kafka store"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} complete=true committing=true callback=Blocker@9513d82{null} (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,692] DEBUG COMMIT for /subjects/CarrierContractvalue/versions/ on HttpChannelOverHttp@b9e0680{r=1,c=true,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=9}
500 Internal Server Error HTTP/1.1
Date: Wed, 25 Aug 2021 09:45:44 GMT
Content-Length: 98
Content-Type: application/vnd.schemaregistry.v1+json

 (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,692] DEBUG org.eclipse.jetty.server.HttpConnection$SendCallback@2316dde2[PROCESSING][i=HTTP/1.1{s=500,h=3,cl=98},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@4d2091ef] generate: NEED_HEADER (null,[p=0,l=98,c=8192,r=98],true)@START (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,692] DEBUG generateHeaders HTTP/1.1{s=500,h=3,cl=98} last=true content=HeapByteBuffer@5ddbef6a[p=0,l=98,c=8192,r=98]={<<<{"error_code":500...e Kafka store"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpGenerator)
[2021-08-25 12:45:44,693] DEBUG Date: Wed, 25 Aug 2021 09:45:44 GMT
Content-Length: 98
Content-Type: application/vnd.schemaregistry.v1+json

 (org.eclipse.jetty.http.HttpGenerator)
[2021-08-25 12:45:44,693] DEBUG CONTENT_LENGTH (org.eclipse.jetty.http.HttpGenerator)
[2021-08-25 12:45:44,693] DEBUG org.eclipse.jetty.server.HttpConnection$SendCallback@2316dde2[PROCESSING][i=HTTP/1.1{s=500,h=3,cl=98},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@4d2091ef] generate: FLUSH ([p=0,l=182,c=8192,r=182],[p=0,l=98,c=8192,r=98],true)@COMPLETING (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,693] DEBUG write: WriteFlusher@4f02979e{IDLE}->null [HeapByteBuffer@53991672[p=0,l=182,c=8192,r=182]={<<<HTTP/1.1 500 Inte....v20190429)\r\n\r\n>>>+json\r\nContent-Le...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00},HeapByteBuffer@5ddbef6a[p=0,l=98,c=8192,r=98]={<<<{"error_code":500...e Kafka store"}>>>\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00}] (org.eclipse.jetty.io.WriteFlusher)
[2021-08-25 12:45:44,693] DEBUG update WriteFlusher@4f02979e{WRITING}->null:IDLE-->WRITING (org.eclipse.jetty.io.WriteFlusher)
[2021-08-25 12:45:44,693] DEBUG flushed 182 NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=W,to=12/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=END,477 of 477},g=HttpGenerator@2752bb85{s=COMPLETING}]=>HttpChannelOverHttp@b9e0680{r=1,c=true,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=10} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,693] DEBUG flushed 98 NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=W,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=END,477 of 477},g=HttpGenerator@2752bb85{s=COMPLETING}]=>HttpChannelOverHttp@b9e0680{r=1,c=true,c=false/false,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=10} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,694] DEBUG Flushed=true written=280 remaining=0 WriteFlusher@4f02979e{WRITING}->null (org.eclipse.jetty.io.WriteFlusher)
[2021-08-25 12:45:44,694] DEBUG update WriteFlusher@4f02979e{IDLE}->null:WRITING-->IDLE (org.eclipse.jetty.io.WriteFlusher)
[2021-08-25 12:45:44,694] DEBUG org.eclipse.jetty.server.HttpConnection$SendCallback@2316dde2[PROCESSING][i=HTTP/1.1{s=500,h=3,cl=98},cb=org.eclipse.jetty.server.HttpChannel$SendCallback@4d2091ef] generate: DONE ([p=182,l=182,c=8192,r=0],[p=98,l=98,c=8192,r=0],true)@END (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,694] DEBUG complete HttpChannelState@34879dcb{s=DISPATCHED a=STARTED i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,694] DEBUG FinalSession=null, old_session_handler=null, this=org.eclipse.jetty.server.session.SessionHandler1784131088==dftMaxIdleSec=-1, calling complete=true (org.eclipse.jetty.server.session)
[2021-08-25 12:45:44,694] DEBUG Adding AsyncListener for Request[POST //**********:8081/subjects/CarrierContractvalue/versions/]@47b0ac54 (org.eclipse.jetty.server.session)
[2021-08-25 12:45:44,694] DEBUG handled=true async=true committed=true on HttpChannelOverHttp@b9e0680{r=1,c=true,c=false/true,a=DISPATCHED,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=11} (org.eclipse.jetty.server.Server)
[2021-08-25 12:45:44,694] DEBUG unhandle HttpChannelState@34879dcb{s=DISPATCHED a=COMPLETE i=true r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,694] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=true,c=false/true,a=COMPLETING,uri=//**********:8081/subjects/CarrierContractvalue/versions/,age=11} action COMPLETE (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,695] DEBUG onComplete HttpChannelState@34879dcb{s=COMPLETING a=NOT_ASYNC i=false r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,695] DEBUG COMPLETE for /subjects/CarrierContractvalue/versions/ written=98 (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,695] INFO ******** - - [25/авг/2021:09:45:44 +0000] "POST /subjects/CarrierContractvalue/versions/ HTTP/1.1" 500 98  12 (io.confluent.rest-utils.requests)
[2021-08-25 12:45:44,695] DEBUG recycle HttpChannelState@34879dcb{s=COMPLETED a=NOT_ASYNC i=false r=IDLE w=false} (org.eclipse.jetty.server.HttpChannelState)
[2021-08-25 12:45:44,695] DEBUG reset HttpParser{s=END,477 of 477} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,695] DEBUG END --> START (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,695] DEBUG HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} handle exit, result COMPLETE (org.eclipse.jetty.server.HttpChannel)
[2021-08-25 12:45:44,695] DEBUG filled 0 HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,695] DEBUG filled 0 HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,695] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=1/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} filled 0 HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,695] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=1/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} parse HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} {} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,695] DEBUG parseNext s=START HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,695] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=1/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} parsed false HttpParser{s=START,0 of -1} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,696] DEBUG releaseRequestBuffer HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=2/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,696] DEBUG fillInterested HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=2/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractConnection)
[2021-08-25 12:45:44,696] DEBUG interested FillInterest@5c8303e9{AC.ReadCB@437141b4{HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=0/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0}}} (org.eclipse.jetty.io.FillInterest)
[2021-08-25 12:45:44,696] DEBUG changeInterests p=false 0->1 for NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=0/30000}{io=0/1,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,696] DEBUG Queued change org.eclipse.jetty.io.ChannelEndPoint$1@5462b477 on ManagedSelector@51ec2df1{STARTED} id=2 keys=1 selected=0 updates=0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,696] DEBUG Wakeup on submit ManagedSelector@51ec2df1{STARTED} id=2 keys=1 selected=0 updates=1 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,696] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 woken with none selected (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,696] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 woken up from select, 0/0/1 selected (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,696] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 processing 0 keys, 1 updates (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,696] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=0/30000}{io=0/1,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} onFillable exit HttpChannelState@34879dcb{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,696] DEBUG ReservedThreadExecutor@5a18cd76{s=2/6,p=0}@2f01377 waiting (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,696] DEBUG updateable 1 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,697] DEBUG update org.eclipse.jetty.io.ChannelEndPoint$1@5462b477 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,697] DEBUG Key interests updated 0 -> 1 on NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=1/30000}{io=1/1,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,697] DEBUG updates 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,697] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 waiting with 1 keys (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,710] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 woken up from select, 1/1/1 selected (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,710] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 processing 1 keys, 0 updates (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,710] DEBUG selected 1 sun.nio.ch.SelectionKeyImpl@7d8fec52 NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=14/30000}{io=1/1,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0}  (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,710] DEBUG onSelected 1->0 r=true w=false for NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=14/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,710] DEBUG task CEP:NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=14/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,710] DEBUG ReservedThreadExecutor@5a18cd76{s=2/6,p=0} tryExecute EatWhatYouKill@543e593/SelectorProducer@4e628b52/PRODUCING/p=false/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=2/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.71+03:00 (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,710] DEBUG ReservedThreadExecutor@5a18cd76{s=1/6,p=0}@2f01377 offer EatWhatYouKill@543e593/SelectorProducer@4e628b52/PRODUCING/p=false/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.71+03:00 (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,710] DEBUG EatWhatYouKill@543e593/SelectorProducer@4e628b52/IDLE/p=true/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.71+03:00 m=EXECUTE_PRODUCE_CONSUME t=CEP:NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=14/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING/BLOCKING (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill)
[2021-08-25 12:45:44,711] DEBUG fillable FillInterest@5c8303e9{AC.ReadCB@437141b4{HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=FI,flush=-,to=14/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0}}} (org.eclipse.jetty.io.FillInterest)
[2021-08-25 12:45:44,710] DEBUG ReservedThreadExecutor@5a18cd76{s=1/6,p=0}@2f01377 task=EatWhatYouKill@543e593/SelectorProducer@4e628b52/IDLE/p=true/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=117]@2021-08-25T12:45:44.71+03:00 (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,711] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=15/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} onFillable enter HttpChannelState@34879dcb{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,711] DEBUG EatWhatYouKill@543e593/SelectorProducer@4e628b52/IDLE/p=true/QueuedThreadPool[qtp987249254]@3ad83a66{STARTED,8<=8<=200,i=3,r=6,q=0}[ReservedThreadExecutor@5a18cd76{s=1/6,p=0}][pc=0,pic=0,pec=45,epc=118]@2021-08-25T12:45:44.711+03:00 tryProduce true (org.eclipse.jetty.util.thread.strategy.EatWhatYouKill)
[2021-08-25 12:45:44,711] DEBUG updateable 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,711] DEBUG updates 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,711] DEBUG shutdownInput NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=15/30000}{io=1/0,kio=1,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractEndPoint)
[2021-08-25 12:45:44,711] DEBUG filled -1 HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,711] DEBUG Key interests updated 1 -> 0 on NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,OPEN,fill=-,flush=-,to=15/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,711] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 waiting with 1 keys (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,711] DEBUG atEOF HttpParser{s=START,0 of -1} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,711] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,ISHUT,fill=-,flush=-,to=15/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} filled -1 HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,712] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,ISHUT,fill=-,flush=-,to=16/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=START,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} parse HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} {} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,712] DEBUG parseNext s=START HeapByteBuffer@53991672[p=0,l=0,c=8192,r=0]={<<<>>>HTTP/1.1 500 Inte...\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00} (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,712] DEBUG START --> CLOSED (org.eclipse.jetty.http.HttpParser)
[2021-08-25 12:45:44,712] DEBUG close NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,ISHUT,fill=-,flush=-,to=16/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractEndPoint)
[2021-08-25 12:45:44,712] DEBUG close(null) NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,ISHUT,fill=-,flush=-,to=16/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractEndPoint)
[2021-08-25 12:45:44,712] DEBUG doClose NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=16/30000}{io=0/0,kio=0,kro=1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ChannelEndPoint)
[2021-08-25 12:45:44,712] DEBUG ignored: WriteFlusher@4f02979e{IDLE}->null (org.eclipse.jetty.io.WriteFlusher)
java.nio.channels.ClosedChannelException
    at org.eclipse.jetty.io.WriteFlusher.onClose(WriteFlusher.java:492)
    at org.eclipse.jetty.io.AbstractEndPoint.onClose(AbstractEndPoint.java:353)
    at org.eclipse.jetty.io.ChannelEndPoint.onClose(ChannelEndPoint.java:215)
    at org.eclipse.jetty.io.NetworkTrafficSelectChannelEndPoint.onClose(NetworkTrafficSelectChannelEndPoint.java:98)
    at org.eclipse.jetty.io.AbstractEndPoint.doOnClose(AbstractEndPoint.java:225)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:192)
    at org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:175)
    at org.eclipse.jetty.io.AbstractConnection.close(AbstractConnection.java:248)
    at org.eclipse.jetty.server.HttpChannelOverHttp.earlyEOF(HttpChannelOverHttp.java:234)
    at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1551)
    at org.eclipse.jetty.server.HttpConnection.parseRequestBuffer(HttpConnection.java:369)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
    at java.lang.Thread.run(Thread.java:748)
[2021-08-25 12:45:44,713] DEBUG onClose FillInterest@5c8303e9{null} (org.eclipse.jetty.io.FillInterest)
[2021-08-25 12:45:44,713] DEBUG Wakeup ManagedSelector@51ec2df1{STARTED} id=2 keys=1 selected=0 updates=0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG queue org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@3cc2d641 (org.eclipse.jetty.util.thread.QueuedThreadPool)
[2021-08-25 12:45:44,713] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 woken with none selected (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 woken up from select, 0/0/0 selected (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 processing 0 keys, 0 updates (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG updateable 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG updates 0 (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG Selector sun.nio.ch.EPollSelectorImpl@5e4c3fb9 waiting with 0 keys (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,713] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=17/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} parsed false HttpParser{s=CLOSED,0 of -1} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,713] DEBUG releaseRequestBuffer HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=17/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,714] DEBUG shutdownOutput NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=18/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractEndPoint)
[2021-08-25 12:45:44,714] DEBUG HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=18/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} onFillable exit HttpChannelState@34879dcb{s=IDLE a=NOT_ASYNC i=true r=IDLE w=false} null (org.eclipse.jetty.server.HttpConnection)
[2021-08-25 12:45:44,714] DEBUG ReservedThreadExecutor@5a18cd76{s=2/6,p=0}@5738009e waiting (org.eclipse.jetty.util.thread.ReservedThreadExecutor)
[2021-08-25 12:45:44,713] DEBUG run org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@3cc2d641 (org.eclipse.jetty.util.thread.QueuedThreadPool)
[2021-08-25 12:45:44,715] DEBUG Destroyed NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=19/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.ManagedSelector)
[2021-08-25 12:45:44,715] DEBUG onClose HttpConnection@437141b4::NetworkTrafficSelectChannelEndPoint@27e875fa{/********:56106<->/********:8081,CLOSED,fill=-,flush=-,to=19/30000}{io=0/0,kio=-1,kro=-1}->HttpConnection@437141b4[p=HttpParser{s=CLOSED,0 of -1},g=HttpGenerator@2752bb85{s=START}]=>HttpChannelOverHttp@b9e0680{r=1,c=false,c=false/false,a=IDLE,uri=null,age=0} (org.eclipse.jetty.io.AbstractConnection)
[2021-08-25 12:45:44,716] DEBUG ran org.eclipse.jetty.io.ManagedSelector$DestroyEndPoint@3cc2d641 (org.eclipse.jetty.util.thread.QueuedThreadPool)

The tips posted above did not help.

Thanks!

ashuvviet commented 2 years ago

I am also getting error during registration of new schema inside my schema registry pod running in AWS cluster

{"error_code":50004,"message":"Leader not known."}

curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema":"{\"type\":\"record\",\"name\":\"ExampleUser\",\"namespace\":\"Schemas.Events\",\"fields\":[{\"name\":\"id\",\"type\":\"string\",\"doc\":\"Id of this message\"},{\"name\":\"operationid\",\"type\":[\"string\",\"null\"],\"doc\":\"id from Automation test for tear down\",\"default\":null},{\"name\":\"tenantid\",\"type\":\"string\",\"doc\":\"Id of the user\"},{\"name\":\"userid\",\"type\":\"string\",\"doc\":\"User id of the removed User\"},{\"name\":\"version\",\"type\":\"string\",\"doc\":\"Version of the message\"}]}"}' http://localhost:8081/subjects/ExampleUser-value/versions {"error_code":50004,"message":"Leader not known."}

Jacob-Holland-CKO commented 2 years ago

I am also getting error during registration of new schema inside my schema registry pod running in AWS cluster

{"error_code":50004,"message":"Leader not known."}

curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" --data '{"schema":"{"type":"record","name":"ExampleUser","namespace":"Schemas.Events","fields":[{"name":"id","type":"string","doc":"Id of this message"},{"name":"operationid","type":["string","null"],"doc":"id from Automation test for tear down","default":null},{"name":"tenantid","type":"string","doc":"Id of the user"},{"name":"userid","type":"string","doc":"User id of the removed User"},{"name":"version","type":"string","doc":"Version of the message"}]}"}' http://localhost:8081/subjects/ExampleUser-value/versions {"error_code":50004,"message":"Leader not known."}

FYI, this can happen if you're running multiple instances of the schema registry. when encountering this issue, scaling down the number of schema registry tasks resolved the issue.