hortonworks / registry

Schema Registry
Apache License 2.0
15 stars 8 forks source link

high concurrency r/w trigger the full table scan which got stuck #810

Open stephonzhao opened 1 year ago

stephonzhao commented 1 year ago

high read/write frequency leads to full "metadata info " load into cache by executing "select * from schema_metadata_info" query without any "where clause" which got stuck. There are 400 millions schema metadata in the database, the full table scan caused the "registry" unable to accept other requests, The cpu usage raise nearly to 2000% . The details described as follows: ERROR [15:52:03.102] [dw-300 - GET /api/v1/schemaregistry/schemas/] o.g.j.s.ServerRuntime$Responder - An I/O error has occurred while writing a response message entity to the container output stream. org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.je