search

sonatype / nexus-public

Sonatype Nexus Repository Open-source codebase mirror
https://www.sonatype.com/products/repository-oss-download
Eclipse Public License 1.0
1.83k stars 557 forks source link

Yum group repo returns http 500 error when inner repo uses xzst compression #392

Open seregaizsbera opened 1 month ago

seregaizsbera commented 1 month ago

What problem are you trying to solve?

Yum group repository returns http 500 error.

Steps to reproduce

  1. Create yum proxy repository with zst compressed metadata, for example, Remote storage: https://repo1.red-soft.ru/redos/8.0/x86_64/os/
  2. Add proxy repository to group
  3. Try to download metadata from group repository dnf makecache.

Expected result

  1. Metadata cached successfully

Actual result

  1. DNF writes error messages Failed to synchronize cache for repo 'test', ignoring this repo.
  2. nexus.log contains the following exception:
XMLStreamException 

java.lang.RuntimeException: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,1]
Message: Invalid byte 1 of 1-byte UTF-8 sequence.
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMetadataMerger.merge(YumMetadataMerger.java:235)
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMetadataMerger.mergeStreams(YumMetadataMerger.java:171)
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMetadataMerger.mergeStreamsAndUpdate(YumMetadataMerger.java:150)
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMetadataMerger.merge(YumMetadataMerger.java:139)
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMergeServiceImpl.generateMergedFile(YumMergeServiceImpl.java:78)
        at org.sonatype.nexus.repository.yum.internal.mergerepo.YumMergeServiceImpl.merge(YumMergeServiceImpl.java:57)
        at org.sonatype.nexus.repository.yum.internal.group.YumGroupMergerImpl.mergeResponses(YumGroupMergerImpl.java:121)
        at org.sonatype.nexus.repository.yum.internal.group.YumGroupMergerImpl.lambda$2(YumGroupMergerImpl.java:71)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
        at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1764)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
        at org.sonatype.nexus.repository.yum.internal.group.YumGroupMergerImpl.fetchAndMergeMetadata(YumGroupMergerImpl.java:73)
        at org.sonatype.nexus.repository.yum.orient.internal.group.OrientYumGroupFacetImpl.buildRepomd(OrientYumGroupFacetImpl.java:150)
        at org.sonatype.nexus.repository.yum.internal.group.YumMergingGroupHandlerSupport.merge(YumMergingGroupHandlerSupport.java:67)
        at org.sonatype.nexus.repository.group.MergingGroupHandlerSupport.lambda$8(MergingGroupHandlerSupport.java:147)

Do you have a workaround you are using at present?

No

What feature or behavior is this required for?

Yum group repository.

How could we solve this issue? (Not knowing is okay!)

if (repomdFile.endsWith(".zst") {
    var unpackedFile = unpack(repomdFile);
    repomdFile = unpackedFile;
}
yumMetadataMerger.merge(repomdFile);

Tell us about your Nexus Repository deployment: what version, operating system, and database are you using?

Sonatype Nexus Repository OSS 3.68.0-04, OrientDB, Fedora 39.

Anything else?

The problem looks very much like issue #338.

nblair commented 1 month ago

Thanks for opening an issue @seregaizsbera . I don't think this is related to #338, which is focused on nuget support. Are there other examples of this xzst compression used by packages in other providers, like http://mirror.centos.org/centos/? If it's unique to that provider, it may not be something Nexus Repository has support for, and would be more akin to an enhancement request.

seregaizsbera commented 1 month ago

@nblair I had been observing the same in https://archives.fedoraproject.org/pub/fedora/linux/releases/40/Server/x86_64/os/repodata/. But now it looks like they switched back to good old gz compression.