search

epam / ai-dial-core

The main component of AI DIAL, which provides unified API to different chat completion and embedding models, assistants, and applications
https://epam-rail.com
Apache License 2.0
358 stars 19 forks source link

GCP. File API fails to upload a file if an absolute path to the file is not encoded #250

Open astsiapanay opened 8 months ago

astsiapanay commented 8 months ago

The test com.epam.aidial.core.FileApiTest#testFileUpload fails with the error:

SEVERE: error after writing 389/409 bytes to https://www.googleapis.com/upload/storage/v1/b/gcp-dev-dial/o?uploadType=multipart
java.io.IOException: too many bytes written
    at java.base/sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3812)
    at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:54)
    at org.jclouds.io.ByteStreams2.copy(ByteStreams2.java:73)
    at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:302)
    at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:175)
    at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:66)
    at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97)
    at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:91)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:74)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:45)
    at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
    at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
    at jdk.proxy3/jdk.proxy3.$Proxy86.multipartUpload(Unknown Source)
    at org.jclouds.googlecloudstorage.blobstore.GoogleCloudStorageBlobStore.putBlob(GoogleCloudStorageBlobStore.java:235)
    at org.jclouds.googlecloudstorage.blobstore.GoogleCloudStorageBlobStore.putBlob(GoogleCloudStorageBlobStore.java:207)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:50)
    at jdk.proxy3/jdk.proxy3.$Proxy78.putBlob(Unknown Source)
    at com.epam.aidial.core.storage.BlobStorage.store(BlobStorage.java:140)
    at com.epam.aidial.core.storage.BlobWriteStream.lambda$end$0(BlobWriteStream.java:106)
    at io.vertx.core.impl.ContextBase.lambda$executeBlocking$0(ContextBase.java:167)
    at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:277)
    at io.vertx.core.impl.ContextBase.lambda$internalExecuteBlocking$2(ContextBase.java:199)
    at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:833)
astsiapanay commented 7 months ago

There is a bug in JCloud library which Core uses to work with cloud providers. Specifically this bug relates to GCP component of JClouds. The bug is reported https://issues.apache.org/jira/browse/JCLOUDS-1632 and the PR is created https://github.com/apache/jclouds/pull/201

@sdryapko ^^

astsiapanay commented 7 months ago

The PR is already merged https://github.com/apache/jclouds/pull/201 Waiting for the next release of JClouds

adubovik commented 1 month ago

@astsiapanay Could you please bump to jclouds>=2.6.1. It appears to contain the fix we need.

https://github.com/epam/ai-dial-core/blob/df982081d966da811c0d638146ccc60a92174b6f/build.gradle#L49-L50

astsiapanay commented 1 month ago

@adubovik the latest version is 2.6.0. See https://github.com/apache/jclouds/tags There is no the required fix in the latest version. We've to apply a patch from 0.10.1