search

HamaWhiteGG / langchain-java

Java version of LangChain, while empowering LLM for Big Data.
Apache License 2.0
545 stars 106 forks source link

retrofit2.adapter.rxjava2.HttpException: HTTP 500 #79

Closed TheBlindM closed 1 year ago

TheBlindM commented 1 year ago

我想创建一个知识库,但是异常

var client = PineconeClient.builder().pineconeApiKey("xx").pineconeEnv("us-west1-gcp-free").requestTimeout(30).build().init();
        if (!client.listIndexes().contains(INDEX_NAME)) {
            // the text-embedding-ada-002 model has an output dimension of 1536.
            var request = CreateIndexRequest.builder()
                    .name(INDEX_NAME)
                    .build();
            client.createIndex(request);
            awaitIndexReady(client);
        }

        var embeddings = OpenAIEmbeddings.builder().openaiApiBase("https://xx/v1/").openaiApiKey("xx").requestTimeout(600).build().init();
        var pinecone = Pinecone.builder()
                .client(client)
                .indexName(INDEX_NAME)
                .namespace(namespace)
                .embeddingFunction(embeddings::embedQuery)
                .build().init();

        var request = new DescribeIndexStatsRequest();

        var response = pinecone.getIndex().describeIndexStats(request);

        if (!response.getNamespaces().containsKey(namespace)) {
            pinecone.fromDocuments(docs,embeddings);
        }

errMsg：

2023/08/16 15:24:54.439 ERROR [http-nio-8089-exec-3] c.b.a.c.ControllerExceptionHandler : 捕获异常:
retrofit2.adapter.rxjava2.HttpException: HTTP 500 
    at retrofit2.adapter.rxjava2.BodyObservable$BodyObserver.onNext(BodyObservable.java:57)
    at retrofit2.adapter.rxjava2.BodyObservable$BodyObserver.onNext(BodyObservable.java:38)
    at retrofit2.adapter.rxjava2.CallExecuteObservable.subscribeActual(CallExecuteObservable.java:48)
    at io.reactivex.Observable.subscribe(Observable.java:12284)
    at retrofit2.adapter.rxjava2.BodyObservable.subscribeActual(BodyObservable.java:35)
    at io.reactivex.Observable.subscribe(Observable.java:12284)
    at io.reactivex.internal.operators.observable.ObservableSingleSingle.subscribeActual(ObservableSingleSingle.java:35)
    at io.reactivex.Single.subscribe(Single.java:3666)
    at io.reactivex.Single.blockingGet(Single.java:2869)
    at com.hw.openai.OpenAiClient.embedding(OpenAiClient.java:247)
    at com.hw.langchain.embeddings.openai.OpenAIEmbeddings.embedWithRetry(OpenAIEmbeddings.java:214)
    at com.hw.langchain.embeddings.openai.OpenAIEmbeddings.getLenSafeEmbeddings(OpenAIEmbeddings.java:140)
    at com.hw.langchain.embeddings.openai.OpenAIEmbeddings.embedDocuments(OpenAIEmbeddings.java:195)
    at com.hw.langchain.vectorstores.pinecone.Pinecone.fromTexts(Pinecone.java:202)
    at com.hw.langchain.vectorstores.base.VectorStore.fromDocuments(VectorStore.java:195)
    at com.beaver.asura.service.impl.FileServiceImpl.initializePineconeIndex(FileServiceImpl.java:98)
    at com.beaver.asura.service.impl.FileServiceImpl.upload(FileServiceImpl.java:65)
    at com.beaver.asura.service.impl.FileServiceImpl$$FastClassBySpringCGLIB$$496fdd7c.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)
    at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704)
    at com.beaver.asura.service.impl.FileServiceImpl$$EnhancerBySpringCGLIB$$e70f8772.upload(<generated>)
    at com.beaver.asura.controller.AppController.upload(AppController.java:84)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at com.github.xiaoymin.knife4j.spring.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:87)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:833)
2023/08/16 15:24:54.440 WARN  [http-nio-8089-exec-3] o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver : Resolved [retrofit2.adapter.rxjava2.HttpException: HTTP 500 ]
HamaWhiteGG commented 1 year ago

The suggestion is to adjust the log level to DEBUG, analyze and pinpoint the issue after viewing more detailed logs.

TheBlindM commented 1 year ago

500可能是网络问题，后面运行出现新的异常

2023/08/16 16:13:05.547 INFO  [http-nio-8089-exec-3] o.n.l.f.Nd4jBackend : Loaded [CpuBackend] backend
2023/08/16 16:13:06.631 INFO  [http-nio-8089-exec-3] o.n.n.NativeOpsHolder : Number of threads used for linear algebra: 8
2023/08/16 16:13:06.632 INFO  [http-nio-8089-exec-3] o.n.l.c.n.CpuNDArrayFactory : Binary level Generic x86 optimization level AVX512
2023/08/16 16:13:06.648 INFO  [http-nio-8089-exec-3] o.n.n.Nd4jBlas : Number of threads used for OpenMP BLAS: 8
2023/08/16 16:13:06.658 INFO  [http-nio-8089-exec-3] o.n.l.a.o.e.DefaultOpExecutioner : Backend used: [CPU]; OS: [Windows 10]
2023/08/16 16:13:06.658 INFO  [http-nio-8089-exec-3] o.n.l.a.o.e.DefaultOpExecutioner : Cores: [16]; Memory: [10.0GB];
2023/08/16 16:13:06.658 INFO  [http-nio-8089-exec-3] o.n.l.a.o.e.DefaultOpExecutioner : Blas vendor: [OPENBLAS]
2023/08/16 16:13:06.661 INFO  [http-nio-8089-exec-3] o.n.l.c.n.CpuBackend : Backend build information:
 GCC: "12.1.0"
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS
2023/08/16 16:13:06.725 ERROR [http-nio-8089-exec-3] c.b.a.c.ControllerExceptionHandler : 捕获异常:
java.lang.IllegalStateException: Shapes do not match: dimensions[0] - x[1] must match y[0], x shape [11, 1536], y shape [11], dimensions [1]
    at org.nd4j.common.base.Preconditions.throwStateEx(Preconditions.java:639)
    at org.nd4j.common.base.Preconditions.checkState(Preconditions.java:427)
    at org.nd4j.linalg.factory.Broadcast.validateBroadcastDims(Broadcast.java:290)
    at org.nd4j.linalg.api.ops.BaseBroadcastOp.<init>(BaseBroadcastOp.java:134)
    at org.nd4j.linalg.api.ops.impl.broadcast.BroadcastMulOp.<init>(BroadcastMulOp.java:36)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.applyBroadcastOp(BaseNDArray.java:2669)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.doRowWise(BaseNDArray.java:2646)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.muliRowVector(BaseNDArray.java:2830)
    at org.nd4j.linalg.api.ndarray.BaseNDArray.mulRowVector(BaseNDArray.java:2836)
    at com.hw.langchain.embeddings.openai.OpenAIEmbeddings.getLenSafeEmbeddings(OpenAIEmbeddings.java:161)
    at com.hw.langchain.embeddings.openai.OpenAIEmbeddings.embedDocuments(OpenAIEmbeddings.java:195)
    at com.hw.langchain.vectorstores.pinecone.Pinecone.fromTexts(Pinecone.java:202)
    at com.hw.langchain.vectorstores.base.VectorStore.fromDocuments(VectorStore.java:195)
    at com.beaver.asura.service.impl.FileServiceImpl.initializePineconeIndex(FileServiceImpl.java:98)
    at com.beaver.asura.service.impl.FileServiceImpl.upload(FileServiceImpl.java:65)
    at com.beaver.asura.service.impl.FileServiceImpl$$FastClassBySpringCGLIB$$496fdd7c.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at org.springframework.aop.framework.CglibAopProxy.invokeMethod(CglibAopProxy.java:386)
    at org.springframework.aop.framework.CglibAopProxy.access$000(CglibAopProxy.java:85)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:704)
    at com.beaver.asura.service.impl.FileServiceImpl$$EnhancerBySpringCGLIB$$45cdd01e.upload(<generated>)
    at com.beaver.asura.controller.AppController.upload(AppController.java:92)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1072)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:965)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:555)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:623)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:209)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at com.github.xiaoymin.knife4j.spring.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:87)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:178)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:153)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:481)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:130)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:390)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:926)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:833)
2023/08/16 16:13:06.732 WARN  [http-nio-8089-exec-3] o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver : Resolved [java.lang.IllegalStateException: Shapes do not match: dimensions[0] - x[1] must match y[0], x shape [11, 1536], y shape [11], dimensions [1]]
TheBlindM commented 1 year ago

这是测试的文本 001儿童口腔科诊疗指南与护理常规.txt

HamaWhiteGG commented 1 year ago

Well, you're dealing with the issue of indexing dimensions now.

java.lang.IllegalStateException: Shapes do not match: dimensions[0] - x[1] must match y[0], x shape [11, 1536], y shape [11], dimensions [1]

HamaWhiteGG commented 1 year ago

这是测试的文本 001儿童口腔科诊疗指南与护理常规.txt

You can add me on WeChat at HamaWhite for a more detailed discussion.

HamaWhiteGG commented 1 year ago

image

TheBlindM commented 1 year ago

好的