OpenSPG / openspg

OpenSPG is a Knowledge Graph Engine developed by Ant Group in collaboration with OpenKG, based on the SPG (Semantic-enhanced Programmable Graph) framework. Core Capabilities: 1) domain model constrained knowledge modeling, 2) facts and logic fused representation, 3) natively support KAG...
https://spg.openkg.cn/en-US
Apache License 2.0
798 stars 101 forks source link

ERROR: createIdx error, and errorMsg= #35

Closed zhangcongxiao closed 11 months ago

zhangcongxiao commented 11 months ago

在重启过docker-compose.yml之后,执行knext schema commit报错 image 一直必现,对应spg日志如下,es日志没有发现报错,请问下可能是什么原因: 2023-12-13 16:11:36,224 [c0a84005170248386569110111] [] [http-nio-8887-exec-4] ERROR c.a.o.a.h.s.HttpBizTemplate - execute http biz callback error com.antgroup.openspg.core.spgschema.model.SchemaException: createIdx error, and errorMsg= at com.antgroup.openspg.core.spgschema.model.SchemaException.alterError(SchemaException.java:33) at com.antgroup.openspg.biz.spgschema.impl.SchemaManagerImpl.alterSchema(SchemaManagerImpl.java:84) at com.antgroup.openspg.biz.spgschema.impl.SchemaManagerImpl$$FastClassBySpringCGLIB$$71da7c33.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) at com.antgroup.openspg.biz.spgschema.impl.SchemaManagerImpl$$EnhancerBySpringCGLIB$$c618afb4.alterSchema() at com.antgroup.openspg.api.http.server.openapi.SchemaController$1.action(SchemaController.java:127) at com.antgroup.openspg.api.http.server.openapi.SchemaController$1.action(SchemaController.java:59) at com.antgroup.openspg.api.http.server.HttpBizTemplate.execute(HttpBizTemplate.java:41) at com.antgroup.openspg.api.http.server.openapi.SchemaController.alterSchema(SchemaController.java:59) at com.antgroup.openspg.api.http.server.openapi.SchemaController$$FastClassBySpringCGLIB$$6ca55c2b.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at com.antgroup.openspg.api.http.server.ControllerLogAspect.doAround(ControllerLogAspect.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708) at com.antgroup.openspg.api.http.server.openapi.SchemaController$$EnhancerBySpringCGLIB$$d7ffbb27.alterSchema() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1071) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:964) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) at javax.servlet.http.HttpServlet.service(HttpServlet.java:696) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) at javax.servlet.http.HttpServlet.service(HttpServlet.java:779) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at com.alipay.sofa.tracer.plugins.springmvc.SpringMvcSofaTracerFilter.doFilter(SpringMvcSofaTracerFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:891) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1784) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.RuntimeException: createIdx error, and errorMsg= at com.antgroup.openspg.cloudext.impl.searchengine.elasticsearch.util.ElasticSearchSchemaUtils.createIdx(ElasticSearchSchemaUtils.java:49) at com.antgroup.openspg.cloudext.impl.searchengine.elasticsearch.ElasticSearchEngineClient.alterSchema(ElasticSearchEngineClient.java:85) at com.antgroup.openspg.cloudext.interfaces.searchengine.BaseIdxSearchEngineClient.alterSchema(BaseIdxSearchEngineClient.java:44) at com.antgroup.openspg.core.spgschema.service.alter.sync.SearchEngineSyncer.syncSchema(SearchEngineSyncer.java:32) at com.antgroup.openspg.core.spgschema.service.alter.stage.PostProcessStage.syncSchema(PostProcessStage.java:69) at com.antgroup.openspg.core.spgschema.service.alter.stage.PostProcessStage.execute(PostProcessStage.java:57) at com.antgroup.openspg.core.spgschema.service.alter.SchemaAlterPipeline.run(SchemaAlterPipeline.java:43) at com.antgroup.openspg.biz.spgschema.impl.SchemaManagerImpl.alterSchema(SchemaManagerImpl.java:82) ... 94 common frames omitted

zhangcongxiao commented 11 months ago

我试了下将schema内容删除只剩一个实体提交可以成功,再恢复schema全量的内容再提交也可以提交成功了。也就是不能在重启后第一次一次性提交较多的schema内容,否则就会出现上面这个报错

zhangcongxiao commented 11 months ago

刚才又重启了 试图重现上面的操作,但发现失败了。第一次提交较少内容的schema失败,重试后可以成功,但是较多的内容的schema仍然失败

baifuyu commented 11 months ago

刚才又重启了 试图重现上面的操作,但发现失败了。第一次提交较少内容的schema失败,重试后可以成功,但是较多的内容的schema仍然失败

执行下 sudo docker ps 看下是否es的container在运行。在之前的已知的问题是当机器内存比较小的时候,es有时候启动不了。这个时候需要在docker-compose配置文件中添加这样一行再试下 image

baifuyu commented 11 months ago

我试了下将schema内容删除只剩一个实体提交可以成功,再恢复schema全量的内容再提交也可以提交成功了。也就是不能在重启后第一次一次性提交较多的schema内容,否则就会出现上面这个报错

很抱歉给您带来不好的体验,这些问题我们会在下一版本中都修复掉

zhangcongxiao commented 11 months ago

经过定位应该是磁盘空间的问题,清理了磁盘空间,并将tugraph, mysql, es数据目录都挂载到主机上后,问题得到解决。

baifuyu commented 11 months ago

经过定位应该是磁盘空间的问题,清理了磁盘空间,并将tugraph, mysql, es数据目录都挂载到主机上后,问题得到解决。

好的,收到。我们在安装文档到时候强调下

zhangcongxiao commented 11 months ago

谢谢