search

milvus-io / milvus-sdk-java

Java SDK for Milvus.
https://milvus.io
Apache License 2.0
383 stars 155 forks source link

2.3版本sdk和xxxjob有jar包冲突 #744

Closed zihao2981 closed 8 months ago

zihao2981 commented 8 months ago
com.xuxueli xxl-job-core 2.4.0 io.milvus milvus-sdk-java 2.3.4

请问有没有好的解决方案，谢谢

xiaofan-luan commented 8 months ago

具体是哪个包冲突，拉个maven dependency看看？

zihao2981 commented 8 months ago

具体是哪个包冲突，拉个maven dependency看看？

netty-buffer应该是。不太会看maven dependency

yhmo commented 8 months ago

你把你的pom.xml贴出来看看

zihao2981 commented 8 months ago

你把你的pom.xml贴出来看看

    <dependency>
        <groupId>io.milvus</groupId>
        <artifactId>milvus-sdk-java</artifactId>
        <version>2.3.1</version>
    </dependency>
    <dependency>
        <groupId>com.xuxueli</groupId>
        <artifactId>xxl-job-core</artifactId>
        <version>2.4.0</version>
    </dependency>
zihao2981 commented 8 months ago

如果milvusjar在前 Exception in thread "xxl-job, EmbedServer bizThreadPool-866716712" java.lang.NoSuchMethodError: io.netty.util.internal.PlatformDependent.byteArrayBaseOffset()J at io.netty.buffer.ByteBufUtil.writeUtf8(ByteBufUtil.java:713) at io.netty.buffer.ByteBufUtil.reserveAndWriteUtf8Seq(ByteBufUtil.java:686) at io.netty.buffer.ByteBufUtil.writeUtf8(ByteBufUtil.java:642) at io.netty.buffer.Unpooled.copiedBufferUtf8(Unpooled.java:599) at io.netty.buffer.Unpooled.copiedBuffer(Unpooled.java:582) at com.xxl.job.core.server.EmbedServer$EmbedHttpServerHandler.writeResponse(EmbedServer.java:214) at com.xxl.job.core.server.EmbedServer$EmbedHttpServerHandler.access$300(EmbedServer.java:130) at com.xxl.job.core.server.EmbedServer$EmbedHttpServerHandler$1.run(EmbedServer.java:164) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

如果是xxxjob在前 [Channel<1>: (172.18.100.148:19530)] Uncaught exception in the SynchronizationContext. Panic!

java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.(ZIIIIIIZ)V at io.grpc.netty.Utils.createByteBufAllocator(Utils.java:176) at io.grpc.netty.Utils.access$000(Utils.java:75) at io.grpc.netty.Utils$ByteBufAllocatorPreferDirectHolder.(Utils.java:97) at io.grpc.netty.Utils.getByteBufAllocator(Utils.java:144) at io.grpc.netty.NettyClientTransport.start(NettyClientTransport.java:233) at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:34) at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:34) at io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:259) at io.grpc.internal.InternalSubchannel.access$400(InternalSubchannel.java:66) at io.grpc.internal.InternalSubchannel$2.run(InternalSubchannel.java:201) at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:95) at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:127) at io.grpc.internal.ManagedChannelImpl$NameResolverListener.onResult(ManagedChannelImpl.java:1878) at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:333) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

GetLoadStateRequest RPC failed! Collection name:knowledge_collection, partition names:[]

io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug! at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271) at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252) at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165) at io.milvus.grpc.MilvusServiceGrpc$MilvusServiceBlockingStub.getLoadState(MilvusServiceGrpc.java:4505) at io.milvus.client.AbstractMilvusGrpcClient.getLoadState(AbstractMilvusGrpcClient.java:2798) at io.milvus.client.MilvusServiceClient.lambda$getLoadState$63(MilvusServiceClient.java:623) at io.milvus.client.MilvusServiceClient$$Lambda$900/1983429073.call(Unknown Source) at io.milvus.client.MilvusServiceClient.retry(MilvusServiceClient.java:259) at io.milvus.client.MilvusServiceClient.getLoadState(MilvusServiceClient.java:623) at com.app.knowledge.service.impl.MilvusServiceImpl.hasLoadCollection(MilvusServiceImpl.java:307) at com.app.knowledge.service.impl.MilvusServiceImpl.queryMilvusIdByKnowledgeIds(MilvusServiceImpl.java:257) at com.app.knowledge.service.impl.ScopeServiceImpl.deleteByIds(ScopeServiceImpl.java:50) at com.app.knowledge.job.SyncMilvusJob.syncMilvusHandler(SyncMilvusJob.java:43) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.xxl.job.core.handler.impl.MethodJobHandler.execute(MethodJobHandler.java:31) at com.xxl.job.core.thread.JobThread.run(JobThread.java:166) Caused by: java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.(ZIIIIIIZ)V at io.grpc.netty.Utils.createByteBufAllocator(Utils.java:176) at io.grpc.netty.Utils.access$000(Utils.java:75) at io.grpc.netty.Utils$ByteBufAllocatorPreferDirectHolder.(Utils.java:97) at io.grpc.netty.Utils.getByteBufAllocator(Utils.java:144) at io.grpc.netty.NettyClientTransport.start(NettyClientTransport.java:233) at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:34) at io.grpc.internal.ForwardingConnectionClientTransport.start(ForwardingConnectionClientTransport.java:34) at io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:259) at io.grpc.internal.InternalSubchannel.access$400(InternalSubchannel.java:66) at io.grpc.internal.InternalSubchannel$2.run(InternalSubchannel.java:201) at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:95) at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:127) at io.grpc.internal.ManagedChannelImpl$NameResolverListener.onResult(ManagedChannelImpl.java:1878) at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:333) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)

yhmo commented 8 months ago

我试了一下，不论是哪个在前哪个在后都没问题啊... 这两个库都依赖io.netty:netty，一个是依赖4.1.90，另一个是依赖4.1.97，这两个版本的netty没太多区别，我这边复杂的case都能跑。

zihao2981 commented 8 months ago

我试了一下，不论是哪个在前哪个在后都没问题啊... 这两个库都依赖io.netty:netty，一个是依赖4.1.90，另一个是依赖4.1.97，这两个版本的netty没太多区别，我这边复杂的case都能跑。

我能不能看看你的pom

yhmo commented 8 months ago

我试了一下，不论是哪个在前哪个在后都没问题啊... 这两个库都依赖io.netty:netty，一个是依赖4.1.90，另一个是依赖4.1.97，这两个版本的netty没太多区别，我这边复杂的case都能跑。

我能不能看看你的pom

用java sdk example里的pom改的，就加了一条xxl-job-core

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ Licensed to the Apache Software Foundation (ASF) under one
  ~ or more contributor license agreements.  See the NOTICE file
  ~ distributed with this work for additional information
  ~ regarding copyright ownership.  The ASF licenses this file
  ~ to you under the Apache License, Version 2.0 (the
  ~ "License"); you may not use this file except in compliance
  ~ with the License.  You may obtain a copy of the License at
  ~
  ~   http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing,
  ~ software distributed under the License is distributed on an
  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  ~ KIND, either express or implied.  See the License for the
  ~ specific language governing permissions and limitations
  ~ under the License.
  -->

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.milvus</groupId>
    <artifactId>milvus-sdk-java-examples</artifactId>
    <version>2.3.4</version>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
                <version>3.8.1</version>
            </plugin>
        </plugins>
    </build>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <!--  For development use  -->
    <!--  Include this if you want to try the latest SNAPSHOT version  -->
    <repositories>
        <repository>
            <id>oss.sonatype.org-snapshot</id>
            <url>http://oss.sonatype.org/content/repositories/snapshots</url>
            <releases>
                <enabled>false</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>io.milvus</groupId>
            <artifactId>milvus-sdk-java</artifactId>
            <version>2.3.4</version>
        </dependency>
        <dependency>
            <groupId>com.xuxueli</groupId>
            <artifactId>xxl-job-core</artifactId>
            <version>2.4.0</version>
        </dependency>
        <dependency>
            <groupId>com.google.code.gson</groupId>
            <artifactId>gson</artifactId>
            <version>2.8.9</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
        </dependency>
    </dependencies>

</project>
zihao2981 commented 8 months ago

我试了一下，不论是哪个在前哪个在后都没问题啊... 这两个库都依赖io.netty:netty，一个是依赖4.1.90，另一个是依赖4.1.97，这两个版本的netty没太多区别，我这边复杂的case都能跑。

我能不能看看你的pom

用java sdk example里的pom改的，就加了一条xxl-job-core


<?xml version="1.0" encoding="UTF-8"?>

<!--

  ~ Licensed to the Apache Software Foundation (ASF) under one

  ~ or more contributor license agreements.  See the NOTICE file

  ~ distributed with this work for additional information

  ~ regarding copyright ownership.  The ASF licenses this file

  ~ to you under the Apache License, Version 2.0 (the

  ~ "License"); you may not use this file except in compliance

  ~ with the License.  You may obtain a copy of the License at

  ~

  ~   http://www.apache.org/licenses/LICENSE-2.0

  ~

  ~ Unless required by applicable law or agreed to in writing,

  ~ software distributed under the License is distributed on an

  ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

  ~ KIND, either express or implied.  See the License for the

  ~ specific language governing permissions and limitations

  ~ under the License.

  -->

<project xmlns="http://maven.apache.org/POM/4.0.0"

         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>io.milvus</groupId>

    <artifactId>milvus-sdk-java-examples</artifactId>

    <version>2.3.4</version>

    <build>

        <plugins>

            <plugin>

                <groupId>org.apache.maven.plugins</groupId>

                <artifactId>maven-compiler-plugin</artifactId>

                <configuration>

                    <source>8</source>

                    <target>8</target>

                </configuration>

                <version>3.8.1</version>

            </plugin>

        </plugins>

    </build>

    <properties>

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

    </properties>

    <!--  For development use  -->

    <!--  Include this if you want to try the latest SNAPSHOT version  -->

    <repositories>

        <repository>

            <id>oss.sonatype.org-snapshot</id>

            <url>http://oss.sonatype.org/content/repositories/snapshots</url>

            <releases>

                <enabled>false</enabled>

            </releases>

            <snapshots>

                <enabled>true</enabled>

            </snapshots>

        </repository>

    </repositories>

    <dependencies>

        <dependency>

            <groupId>io.milvus</groupId>

            <artifactId>milvus-sdk-java</artifactId>

            <version>2.3.4</version>

        </dependency>

        <dependency>

            <groupId>com.xuxueli</groupId>

            <artifactId>xxl-job-core</artifactId>

            <version>2.4.0</version>

        </dependency>

        <dependency>

            <groupId>com.google.code.gson</groupId>

            <artifactId>gson</artifactId>

            <version>2.8.9</version>

        </dependency>

        <dependency>

            <groupId>org.slf4j</groupId>

            <artifactId>slf4j-api</artifactId>

            <version>1.7.30</version>

        </dependency>

    </dependencies>

</project>

感谢分享，已解决，解决办法为pom中的properties中引入netty.version。