Error when using S3 blobstore: java.io.IOException: failed to read entry trailer. Occured at byte: 46324

[ju2wheels]

Thanks for creating an issue! Please fill out this form so we can be sure to have all the information we need, and to minimize back and forth.

• What are you trying to do? Upload a deb file to an apt hosted repo.

• What feature or behavior is this required for? Upload a deb

• How could we solve this issue? (Not knowing is okay!) Im currently getting the following error:

2018-03-13 02:03:46,868+0000 INFO  [qtp520303092-386]  admin org.sonatype.nexus.repository.manager.internal.RepositoryManagerImpl - Updating repository: myrepo -> Configuration{repositoryName='myrepo', recipeName='apt-hosted', attributes={apt={distribution=bionic}, aptHosted={assetHistoryLimit=null}, aptSigning={keypair=-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: GnuPG v1

redacted
-----END PGP PRIVATE KEY BLOCK-----
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1

redacted
-----END PGP PUBLIC KEY BLOCK-----, passphrase=redacted}, storage={blobStoreName=my-s3, strictContentTypeValidation=true, writePolicy=ALLOW}}}
2018-03-13 02:04:00,852+0000 WARN  [qtp520303092-474]  me org.sonatype.nexus.repository.httpbridge.internal.ViewServlet - Failure servicing: POST /repository/myrepo/
java.io.IOException: failed to read entry trailer. Occured at byte: 46324
    at org.apache.commons.compress.archivers.ar.ArArchiveInputStream.getNextArEntry(ArArchiveInputStream.java:125)
    at org.apache.commons.compress.archivers.ar.ArArchiveInputStream.getNextEntry(ArArchiveInputStream.java:221)
    at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedFacet.ingestAsset(AptHostedFacet.java:134)
    at org.sonatype.nexus.transaction.TransactionalWrapper.proceedWithTransaction(TransactionalWrapper.java:56)
    at org.sonatype.nexus.transaction.TransactionInterceptor.invoke(TransactionInterceptor.java:54)
    at net.staticsnow.nexus.repository.apt.internal.hosted.AptHostedHandler.handle(AptHostedHandler.java:59)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at net.staticsnow.nexus.repository.apt.internal.gpg.AptSigningHandler.handle(AptSigningHandler.java:43)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at net.staticsnow.nexus.repository.apt.internal.snapshot.AptSnapshotHandler.handle(AptSnapshotHandler.java:60)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.storage.UnitOfWorkHandler.handle(UnitOfWorkHandler.java:39)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.view.handlers.ContentHeadersHandler.handle(ContentHeadersHandler.java:44)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.http.PartialFetchHandler.handle(PartialFetchHandler.java:55)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.view.handlers.ConditionalRequestHandler.handle(ConditionalRequestHandler.java:72)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.view.handlers.ExceptionHandler.handle(ExceptionHandler.java:44)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.security.SecurityHandler.handle(SecurityHandler.java:52)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.view.handlers.TimingHandler.handle(TimingHandler.java:46)
    at org.sonatype.nexus.repository.view.Context.proceed(Context.java:80)
    at org.sonatype.nexus.repository.view.Context.start(Context.java:114)
    at org.sonatype.nexus.repository.view.Router.dispatch(Router.java:63)
    at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:52)
    at org.sonatype.nexus.repository.view.ConfigurableViewFacet.dispatch(ConfigurableViewFacet.java:43)
    at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.dispatchAndSend(ViewServlet.java:211)
    at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.doService(ViewServlet.java:173)
    at org.sonatype.nexus.repository.httpbridge.internal.ViewServlet.service(ViewServlet.java:126)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:286)
    at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:276)
    at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:181)
    at com.google.inject.servlet.DynamicServletPipeline.service(DynamicServletPipeline.java:71)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
    at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:112)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:61)
    at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
    at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
    at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
    at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
    at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
    at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
    at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
    at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
    at org.apache.shiro.web.servlet.AdviceFilter.executeChain(AdviceFilter.java:108)
    at org.apache.shiro.web.servlet.AdviceFilter.doFilterInternal(AdviceFilter.java:137)
    at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
    at org.apache.shiro.web.servlet.ProxiedFilterChain.doFilter(ProxiedFilterChain.java:66)
    at org.apache.shiro.web.servlet.AbstractShiroFilter.executeChain(AbstractShiroFilter.java:449)
    at org.sonatype.nexus.security.SecurityFilter.executeChain(SecurityFilter.java:85)
    at org.apache.shiro.web.servlet.AbstractShiroFilter$1.call(AbstractShiroFilter.java:365)
    at org.apache.shiro.subject.support.SubjectCallable.doCall(SubjectCallable.java:90)
    at org.apache.shiro.subject.support.SubjectCallable.call(SubjectCallable.java:83)
    at org.apache.shiro.subject.support.DelegatingSubject.execute(DelegatingSubject.java:383)
    at org.apache.shiro.web.servlet.AbstractShiroFilter.doFilterInternal(AbstractShiroFilter.java:362)
    at org.sonatype.nexus.security.SecurityFilter.doFilterInternal(SecurityFilter.java:101)
    at org.apache.shiro.web.servlet.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:125)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.sonatype.nexus.repository.httpbridge.internal.ExhaustRequestFilter.doFilter(ExhaustRequestFilter.java:71)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at com.sonatype.nexus.licensing.internal.LicensingRedirectFilter.doFilter(LicensingRedirectFilter.java:108)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.sonatype.nexus.internal.web.ErrorPageFilter.doFilter(ErrorPageFilter.java:68)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.sonatype.nexus.internal.web.EnvironmentFilter.doFilter(EnvironmentFilter.java:102)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at org.sonatype.nexus.internal.web.HeaderPatternFilter.doFilter(HeaderPatternFilter.java:98)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
    at com.google.inject.servlet.DynamicFilterPipeline.dispatch(DynamicFilterPipeline.java:104)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:135)
    at org.sonatype.nexus.bootstrap.osgi.DelegatingFilter.doFilter(DelegatingFilter.java:73)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:175)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:530)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
    at java.lang.Thread.run(Thread.java:748)

I dont understand from the error if this is a misconfiguration on my part or a bug. It would really help if the documentation at least gave a hint as to what is wanted by the PGP signing key pair box despite what was said in #20 , I cant be sure im providing what it wants otherwise (e.g. if the above is actually what it wants have the documentation at least state that you should export your GPG armored public/private keys into that box or something).

I gave it a RSA 4096 signing key. I get the same result even if I only provide the private key.

• Anything else?

[mpoindexter]

It should look like:

-----BEGIN PGP PRIVATE KEY BLOCK-----
Version: GnuPG v1

<REDACTED>
-----END PGP PRIVATE KEY BLOCK-----

It should be an encrypted private key that can be decrypted using the password supplied in the passphrase field.

My comment in #20 was more around "how to create and export a GPG key" (well documented by GPG) and "how to add a repo to the apt sources list" (well documented by APT), if it's not clear what should go in this field, that definitely deserves documentation!

[ju2wheels]

Ill try that again, but I was getting the same error.

[Edit] Still the same error with just the exported private key, and even with a 2048 key.

Steps:

gpg --gen-key  # (option 4 RSA signing key only)
gpg --armor --export-secret-keys <key_id>  # Output placed in the key box in APT config

[DarthHater]

@mpoindexter I'm sorta useless here (have not made it to this section of the codebase yet). Any thing I can do to help?

[ju2wheels]

@DarthHater How much effort would it be for me to build nexus with a newer version of org.apache.commons commons-compress so I can see if that makes any difference (if that wont break anything)? The stack trace seems to be in the code for reading the archive and maybe not GPG related?

Either that or it does not like an already signed deb possibly? I just downloaded a random deb from one of my configured PPAs to upload to test my repo.

[mpoindexter]

Good point that it looks like its a decompression problem! I've uploaded a lot of debs and not run into this. As far as an already signed deb, I doubt that would cause any trouble, but signed debs are pretty uncommon. Most things use repository signing (which is what this plugin does too). Is it a public deb file that you're trying to upload? If so could you link to it? If not, could you grab a random public deb (like http://archive.ubuntu.com/ubuntu/pool/main/a/a11y-profile-manager/a11y-profile-manager-doc_0.1.10-0ubuntu3_all.deb for example) and try to upload that to see if that causes trouble?

[ju2wheels]

Same result with that deb. I was originally trying yubikey-neo-manager_1.4.0-1_all.deb from PPA:

$ cat /etc/apt/sources.list.d/yubico-ubuntu-stable-xenial.list
deb http://ppa.launchpad.net/yubico/stable/ubuntu xenial main
# deb-src http://ppa.launchpad.net/yubico/stable/ubuntu xenial main

Using:

curl -v --user "${REPO_USER}" -X POST -H "Content-Type: multipart/form-data" --data-binary "@a11y-profile-manager-doc_0.1.10-0ubuntu3_all.deb" https://myrepo.com/repository/myrepo/

REPO_USER is user:pass assigned by user token.

[DarthHater]

@ju2wheels your guess is as good as mine, I don't think it would be TOO bad. I would fork nexus-public, adjust the pom, add a Dockerfile to it, run mvn and then set up Nexus similar to what we do in docker-nexus3. That'd be my best guess in terms of doing it. I would use nexus-public to do it since the change I made recently here makes apt available to base/core feature, so running those commands should work on nexus-public.

[DarthHater]

Pressed wrong button PS, sorry for artificially closing this.

[mpoindexter]

Hmmmm, I tried uploading that package into my local copy and it worked fine.

I looked at the code of commons-compress (https://github.com/apache/commons-compress/blob/master/src/main/java/org/apache/commons/compress/archivers/ar/ArArchiveInputStream.java#L125) and it seems like maybe your upload is getting truncated? So I'd look into making sure the file is downloaded properly onto your machine, and that nothing in between you and your Nexus instance is interfering with the upload (load balancers, proxies, etc), but not sure how much help I can provide beyond that given that it works on my instance.

The good news is that at least it's pretty clearly a archive problem, not a problem with the GPG stuff.

[ju2wheels]

I tried uploading it locally on the host to 8081 but I get the same thing. That should rule out my ELB.

I also tried using the Admin user's user token instead of a regular user with specific role permissions just to see if somehow I missed some permissions that might be causing the problem but that doesnt work either.

@DarthHater I tried building the last commit that had 3.9.0, not too bad but failed to build both times with tests on and off.

Tests on (mvn package):

nexus_1  | SEVERE: org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "/nexus/components/nexus-rapture/target/phantomjs-maven-plugin/phantomjs-2.1.1-linux-x86_64/bin/phantomjs" (in directory "."): error=2, No such file or directory)

Tests off (mvn package -Dmaven.test.skip=true):

nexus_1  | [INFO] ------------------------------------------------------------------------
nexus_1  | [INFO] BUILD FAILURE
nexus_1  | [INFO] ------------------------------------------------------------------------
nexus_1  | [INFO] Total time: 05:53 min
nexus_1  | [INFO] Finished at: 2018-03-13T18:42:52Z
nexus_1  | [INFO] Final Memory: 172M/1465M
nexus_1  | [INFO] ------------------------------------------------------------------------
nexus_1  | [ERROR] Failed to execute goal on project nexus-repository-maven: Could not resolve dependencies for project org.sonatype.nexus.plugins:nexus-repository-maven:bundle:3.9.0-SNAPSHOT: Could not find artifact org.sonatype.nexus:nexus-repository:jar:tests:3.9.0-SNAPSHOT in rso-public-grid (https://repository.sonatype.org/content/groups/sonatype-public-grid/) -> [Help 1]
nexus_1  | [ERROR] 
nexus_1  | [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
nexus_1  | [ERROR] Re-run Maven using the -X switch to enable full debug logging.
nexus_1  | [ERROR] 
nexus_1  | [ERROR] For more information about the errors and possible solutions, please read the following articles:
nexus_1  | [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
nexus_1  | [ERROR] 
nexus_1  | [ERROR] After correcting the problems, you can resume the build with the command
nexus_1  | [ERROR]   mvn <goals> -rf :nexus-repository-maven

Ditto for mvn clean install and mvn release:prepare.

[DarthHater]

That's wild. It's looking for a snapshot of 3.9.0, which would be gone from RSO (we are doing 3.10.0 work internally now). I'll ping a few people and let them know!

[ju2wheels]

I was specifically on a checkout of 3.9.0 (38a81af031551d0e9e32e474cc2d1dcd55579275), I didnt use master because my current nexus-data dir is based off 3.9.0 and I dont know what going back and forth would do the data.

[DarthHater]

Gotcha, thanks for the extra info. Either way, it's looking for snapshot files rather than released ones, so that's likely the issue? I rang the bell internally, I'll see what happens.

[DarthHater]

You are using this tag correct? https://github.com/sonatype/nexus-public/releases/tag/release-3.9.0-01

[DarthHater]

Thanks for the commit @ju2wheels , there's certainly something amiss with nexus-public, I'll see what we can do to figure it out.

[michael-bucher]

@ju2wheels I've updated the release to use the proper version and ran the build/tests locally. Give it a try now.

[ju2wheels]

@DarthHater @anubistheta OK I grabbed that tarball above and extracted it into a branch and got it to build successfully, however Im not convinced the produced artifact will actually run given that its missing a bunch of libraries. I updated buildsupport/commons/pom.xml to change the commons-compress version and its dependency to a newer one.

What Im seeing is that aside from the .install4j files in the upstream official release tarball which can probably be safely ignored, there are missing libraries in the generated build (e.g. commons-compress, commons-collections4, and a few others). I dont have enough Java-fu to figure out whats going on here.

Attached a file showing the diff between upstream tarball (left) and the generated tarball (right): upstreaam_vs_manual_build_tar_contents.txt

Can be reproduced using this branch using docker-compose build; docker-compose up.

[michael-bucher]

@ju2wheels @DarthHater Thanks for the update. I'm looking into it now.

[michael-bucher]

@ju2wheels @DarthHater Is the upstream tarball the one download from the Sonatype website? i.e. this link: https://www.sonatype.com/download-oss-sonatype

If so, I think I know what causes the discrepancy. We have some plugins that are part of the OSS binary, but aren't open sourced. e.g. nexus-proui-plugin, nexus-hazelcast-plugin, nexus-repository-pypi. It looks like those are the ones that are missing according to the diff.

Thus, the tarball from the website should have a superset of the features/files of the generated tarball.

I think that most of the missing jar's that are in a subdir of nexus-3.9.0-01/system/com/sonatype/nexus/plugins are OSGI bundles.
So if you need some of them to do your testing you can install them into your custom built nexus install by following https://help.sonatype.com/repomanager3/bundle-development/installing-bundles

[ju2wheels]

@DarthHater @anubistheta that would explain the missing plugins but not the other libraries unless the libraries are only included by your internal build process that includes those non OSS plugins. As far as I understand how things are packaged im not sure thats the case.

For example this Apt plugin specifies a dependency on commons-compress which to my limited Java knowledge would mean the parent (nexus-public) has to provide it as part of its common installed libraries (which it currently isnt according to the diff)?

At a minimum, I know these missing ones are important for this plugin:

nexus-3.9.0-01/system/org/apache/commons/commons-collections4 <
nexus-3.9.0-01/system/org/apache/commons/commons-compress/1.1 <
nexus-3.9.0-01/system/org/tukaani/xz/1.5/xz-1.5.jar       <

If these are not currently installed as part of the apt plugin build process, how would these get there (these are defined in buildsupport/commons/pom.xml in nexus-public)?

If this is the expected build output then that means I cant test what I wanted to (building Nexus 3 with newer commons-compress version).

Im not sure those bundle instructions are that useful in my case, im not trying to do bundle development, im just trying to rebuild a usable Nexus 3 instance with different dependency versions.

[ju2wheels]

@mpoindexter what version of Nexus + repository-apt are you on?

[mpoindexter]

I'm using 3.7.1/1.0.4 currently. But I don't think much has changed in the code aside from adding some tests and updating method signatures to match newer versions of Nexus.

[DarthHater]

Interesting @ju2wheels , I see it in buildsupport/commons/pom.xml , let me build nexus-public and see what I find out.

[ju2wheels]

I just realized from looking at the file sizes and the error messages that its not truncation of the upload that seems to be the problem. Its trying to read past the end of the archive for the trailer marker (assuming that means byte sequence for EOF of archive). For example, in the error I posted above, java.io.IOException: failed to read entry trailer. Occured at byte: 46324, 46324 is the exact size of the deb file I upload and according to the source the value it outputs is the number of bytes read.

$ ll yubikey-neo-manager_1.4.0-1_all.deb 
-rw-r--r-- 1 jlajara jlajara 46324 Nov 25  2015 yubikey-neo-manager_1.4.0-1_all.deb

This issue seems similar to whats being hit above except its for the TAR format instead of the AR format. Based on that discussion, could it be possible we are hitting the same problem on this line and that we arent getting a proper return value of null even though theres no next entry, so its trying to read past EOF to find a non existent next entry?

[DarthHater]

@ju2wheels separately to this issue, I am digging around internally to see what people think about how to test out upgrading commons-compress and how to best do so. I'll share some info when I know more (wanted to make sure you all know I'm not just watching YouTube over here :) )

[ju2wheels]

OK, I identified the problem based on what I found in the last post.

Using an S3 blobstore with the apt repository seems to cause it to be unable to find the EOF of file I guess. Using a normal file blobstore with apt allows me to upload the deb fine, so its between S3 blob store and commons-compress.

I suspect this is related to this commit that changed the temp upload from being a stream to based on the blob store.

@DarthHater @mpoindexter I havent looked further but let me know if you consider this to be a bug in nexus-blobstore-s3.

[mpoindexter]

Great detective work @ju2wheels! I think that commit was because the TempStream based one went away in newer versions of Nexus. For the standard blob store the revised code seems to work fine, but definitely haven't tested with the s3 based one.

[DarthHater]

I would raise it for s3, probably send an email to the nexus-users list and maybe create a JIRA ticket. We are in the process of bringing s3 into the core product, so I'm sure they will want to know about that. And wow, yeah! Great detective work!

[mpoindexter]

The interwebs seem to indicate that this could possibly be a bad interaction between the JDK's built in GZIPInputStream and streaming files from S3. If someone ever wants to look at this it might be a good experiment to replace the use of java.util.zip.GZIPInputStream with the version from commons-compress.

[mpoindexter]

Not sure if the above change will fix this, if anyone finds this still happening with the latest code feel free to reopen!

[ju2wheels]

Im unfortunately hitting a convoluted mess trying to test this out to see if it resolves the issue, although nothing directly to due with this plugin. Is 3.11.0 a hard requirement for this one? Any chance it will work with 3.10.0 ?

Im not able to test this in combination with S3 blobstore because that plugin wont compile against 3.11.0 .

[brianpatrickreavey]

I get the same error on OSS 3.13.0-01, and the most recent apt repository plugin.

Any attempts to upload a .deb to an apt repo with the S3-blob backend fails with an error:

java.io.IOException: failed to read entry trailer. Occured at byte: <size of file>

Using the default blob store works fine. This occurs via the UI, or curl.

[mpoindexter]

Well, bummer. Kind of at a loss here, I don't use the S3 blobstore. As near as I can tell, the code here is right, but PRs to fix this welcome if anyone else has a good idea

[jarro2783]

I'm getting something similar using the S3 blobstore. It fails with "truncated ar archive". It seems to work fine when I use local storage.

[kadosh1000]

I get the same issue as @jarro2783

[abuzhynsky]

@mpoindexter Any news on this issue?

[mpoindexter]

Nope. I don't use s3 blob store, don't have any way to reproduce, and I don't see anything wrong in the code. Someone who uses s3 blob store will have to look into this if it's to get fixed.

[gdzien]

Just wanted to confirm this was happening to me using S3 blobstore, switching to local one fixed it.

[mpoindexter]

I've finally gotten around to taking a look at this. It's a bug in commons-compress in the ArArchiveInputStream class. commons-compress seems to have a fix for it that was done Feb 11, 2019, but it's not in a public release yet. I'll take a stab at integrating the fixed class into this plugin temporarily until a fixed version of commons-compress is released.

[bhamail]

APT is now part of Nexus Repository Manager. Version 3.17.0 includes the APT plugin by default. If this is still an issue if using 3.17.0 or later please file an issue at https://issues.sonatype.org/. Links to the new source code location are in the top level README.md