repository parsing fails due to unexpected HTTP headers

[olafhering]

I have a published repository from a project in IBS. It gets exported via NFS, which makes it simple to transfer the directory somewhere else. I copied the exported directory to my own webspace.

Adding and using the repository with zypper ar -cf $URL $name && zypper ref -r $name works as expected.

Adding it as a custom repository with rmt-cli repos custom add $URL $name && rmt-cli mirror repository $name fails, rmt-cli complains about checksum mismatch in the files referenced by repomd.xml.

It seems zypper does not pay attention to the headers in the HTTP response. But rmt-cli apparently does.

The HTTP headers for the ${sha256sum}-${tag}.xml.gz files look like this:

HTTP/1.1 200 OK
Date: Thu, 16 May 2024 15:56:07 GMT
Server: Apache/2.4.59 (Unix)
Vary: User-Agent
Last-Modified: Thu, 11 May 2023 14:38:30 GMT
ETag: "12c-5fb6bf1384d1c"
Accept-Ranges: bytes
Content-Length: 300
Content-Type: text/xml
Content-Encoding: x-gzip

<binary data>

A similar file served by the typical RMT server looks like this:

HTTP/1.1 200 OK
Server: nginx/1.21.5
Date: Thu, 16 May 2024 16:02:36 GMT
Content-Type: application/octet-stream
Content-Length: 3554
Last-Modified: Tue, 14 May 2024 17:30:08 GMT
Connection: keep-alive
ETag: "66439fa0-de2"
Accept-Ranges: bytes

<binary data>

The questions are:

• Is the response from that particular Apache server valid, should it provide these Content-Type and Content-Encoding tags?
• Is a client supposed to interpret and use the Content-Type and Content-Encoding tags, instead of doing its own probing?

[digitaltom]

I don't think this is related to HTTP headers. When RMT is mirroring a repo, it validates the checksums of primary.xml with the files it downloads, and that seems to fail in your case. Afaik zypper does not care about those checksums. Could you post the error message, along with the metadata files, and one of the packages it complains about? Then we could compare the checksums between the metadata and the files.

[olafhering]

I think it is the RMT server that is at fault here. The file is XML (Content-Type), but it is compressed with GZIP (Content-Encoding).

Is the underlying Ruby framework unable to handle that properly (hard to imagine)?

[olafhering]

https://www.rfc-editor.org/rfc/rfc2616#section-14.11

[digitaltom]

Can you share with us the $URL of that repo? You can post it in Slack if you don't want to show it up here.

[olafhering]

The URL is http://www.aepfle.de/rmt1157/