put file with http Expect: 100-continue fails with timeout on tryCreate tempBlob

ca-stefan-cordes commented 10 months ago

What problem are you trying to solve?

When putting a file to nexus3 with

curl --location --request PUT 'http://test.nexus.canda.com:8081/repository/Snapshots/com/canda/mule4/mule4-template/1.0.0-local-SNAPSHOT/mule4-template-1.0.0-local-20231027.164040-1.pom' \
--header 'Expect: 100-continue' \
--header 'Content-Length: 10999' \

the request fails after 30 seconds with a http 500 server error and the server shows

2023-11-03 10:59:27,838+0100 WARN  [qtp-1066085455-93]  org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: PUT /repository/Snapshots/com/canda/mule4/mule4-template/1.0.0-local-SNAPSHOT/mule4-template-1.0.0-local-20231027.164040-1.pom
org.sonatype.nexus.blobstore.api.BlobStoreException: BlobId: tmp$b2339fd7-1b4b-4a1f-b370-31084ba1cc74, java.io.IOException: java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms, Cause: java.util.concurrent.TimeoutException: Idle timeout expired: 30000/30000 ms
    at org.sonatype.nexus.blobstore.file.FileBlobStore.tryCreate(FileBlobStore.java:438)
    at org.sonatype.nexus.blobstore.file.FileBlobStore.create(FileBlobStore.java:368)
    at org.sonatype.nexus.blobstore.file.FileBlobStore.doCreate(FileBlobStore.java:348)
    at org.sonatype.nexus.blobstore.metrics.BlobStoreAnalyticsInterceptor.invoke(BlobStoreAnalyticsInterceptor.java:60)
    at org.sonatype.nexus.blobstore.BlobStoreSupport.create(BlobStoreSupport.java:125)
    at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
    at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
    at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:54)
    at org.sonatype.nexus.blobstore.BlobStoreSupport.create(BlobStoreSupport.java:109)
    at org.sonatype.nexus.common.stateguard.MethodInvocationAction.run(MethodInvocationAction.java:39)
    at org.sonatype.nexus.common.stateguard.StateGuard$GuardImpl.run(StateGuard.java:272)
    at org.sonatype.nexus.common.stateguard.GuardedInterceptor.invoke(GuardedInterceptor.java:54)
    at org.sonatype.nexus.repository.storage.StorageFacetImpl.createTempBlob(StorageFacetImpl.java:266)

(full stacktace, see nexus-stacktrace-trycreate.txt )

when removing the --header 'Expect: 100-continue' it works as expected.

Do you have a workaround you are using at present? The error was in nexus before, too but previously on maven 3.8 we configured wagon not to send http 100-expect continue as workaround. With maven 3.9 the transport was changed and now on putting files the with Expect: 100-continue always. So with maven 3.9 default http transport and nexus3 we no longer have a workaround. The maven 3.9 workaround is to switch back to the (outdated) wagon protocol via -Dmaven.resolver.transport=wagon
What feature or behavior is this required for? Uploading any artifact with maven 3.9
How could we solve this issue? (Not knowing is okay!) Expect: 100-continue should respond with HTTP/1.1 100 Continue.
Tell us about your Nexus Repository deployment: what version, operating system, and database are you using? Nexus: 3.60.0-02 Edition OSS java-1.8.0-ibm-1.8.0 (java.runtime.version 8.0.8.10 - pxz6480sr8fp10-20230703_02(SR8 FP10)) karaf.version 4.3.9 nexus-db-feature nexus-orient nexus.change.repo.blobstore.task.enabled false nexus.hazelcast.discovery.isEnabled true os.arch s390x os.name Linux os.version 5.14.21-150500.55.31-default sun.java.command com.install4j.runtime.launcher.UnixLauncher start 9d17dc87 0 0 org.sonatype.nexus.karaf.NexusMain ORIENTDB_HOME /srv/nexus/sonatype-work/nexus3/orient java.io.tmpdir ../sonatype-work/nexus3/tmp
Anything else? The problem is not related to our reverse proxy in front of nexus as I connect directly to the nexus instance http://test.nexus.canda.com:8081/ (application-port 8081) It happens independent of file size to be put. You cannot access the server from internet.

cstamas commented 10 months ago

This is clearly nexus bug, as it violates steps as described here: https://stackoverflow.com/questions/35305084/100-continue-with-jersey2-with-jetty9

As can be seen from sources:

It DOES gets the parameters, hence, triggers request consumption while the full request is "not yet here".

ca-stefan-cordes commented 10 months ago

In the meantime I found another workaround for us:

The maven 3.9 workaround is to switch back to the (outdated) wagon protocol via -Dmaven.resolver.transport=wagon

(Proposed solution is to fully support "http 100 expect continue")

nblair commented 9 months ago

Thanks for opening an issue @ca-stefan-cordes - we plan to investigate our compatibility with Maven 3.9 in the near future. In the interim, thank you for sharing the workaround you've discovered.

ca-stefan-cordes commented 2 weeks ago

Do you know when approximately is "near future" ?

sonatype / nexus-public

put file with http Expect: 100-continue fails with timeout on tryCreate tempBlob #287