Open joey-grant opened 1 year ago
It looks to me like the remote is configured correctly. And since the trailing slash is part of the url there, I cannot see that the cli is to blame either. Would you be able to provide a full stacktrace of this failure?
You should get that either from pulp task show
or from the server logs.
Sure thing, thanks for looking at this with me.
[root@secondary ~]# pulp task show --href /pulp/api/v3/tasks/676d1531-ba40-43f6-9c76-48bb3f4c4f87/
{
"pulp_href": "/pulp/api/v3/tasks/676d1531-ba40-43f6-9c76-48bb3f4c4f87/",
"pulp_created": "2023-09-01T15:19:58.646463Z",
"state": "failed",
"name": "pulp_rpm.app.tasks.synchronizing.synchronize", "logging_cid": "6cc165caf2204c5b97bf55399a0b057b", "started_at": "2023-09-01T15:19:58.769975Z", "finished_at": "2023-09-01T15:19:59.018980Z",
"error": {
"traceback": " File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 444, in _perform_task\n result = func
(*args, **kwargs)\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 486, in synchronize\n remote_url = f
etch_remote_url(remote, url)\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 305, in fetch_remote_url\n
remote_url = fetch_mirror(remote)\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 254, in fetch_mirror\
n result = downloader.fetch()\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulpcore/download/base.py\", line 175, in fetch\n return done.pop
().result()\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/pulpcore/download/http.py\", line 273, in run\n return await download_wrapper()\n Fil
e \"/usr/local/lib/pulp/lib64/python3.8/site-packages/backoff/_async.py\", line 151, in retry\n ret = await target(*args, **kwargs)\n File \"/usr/local/lib/p
ulp/lib64/python3.8/site-packages/pulpcore/download/http.py\", line 258, in download_wrapper\n return await self._run(extra_data=extra_data)\n File \"/usr/lo
cal/lib/pulp/lib64/python3.8/site-packages/pulp_rpm/app/downloaders.py\", line 117, in _run\n self.raise_for_status(response)\n File \"/usr/local/lib/pulp/li
b64/python3.8/site-packages/pulp_rpm/app/downloaders.py\", line 102, in raise_for_status\n response.raise_for_status()\n File \"/usr/local/lib/pulp/lib64/python3.8/site-packages/aiohttp/client_reqrep.py\", line 1004, in raise_for_status\n raise ClientResponseError(\n",
"description": "404, message='Not Found', url=URL('https://primary.env.company.com/pulp/content/application-x86_64-eng')"
},
"worker": "/pulp/api/v3/workers/0cd785be-c0ad-4415-89e9-6a7c9d6161fb/",
"parent_task": null,
"child_tasks": [],
"task_group": null,
"progress_reports": [],
"created_resources": [],
"reserved_resources_record": [
"/pulp/api/v3/repositories/rpm/rpm/36a955a4-f5d6-4b48-947f-46edbe396201/",
"shared:/pulp/api/v3/remotes/rpm/rpm/8c4ee2f0-e095-4967-bc48-fa1a041d37e2/"
]
}
@dralley Does this look familiar to you? Would you agree reassigning this issue to pulp_rpm?
I'm fine with reassigning it to pulp_rpm. I agree that it probably isn't a CLI issue, at least.
It appears that this line of code in synchronizing.py
is the culprit. What is the purpose of stripping out the trailing forward slash deliberately?
downloader = remote.get_downloader(url=remote.url.rstrip("/"), urlencode=False)
What happens here is that your URL has been identified as mirrorlist, which is not true https://github.com/pulp/pulp_rpm/blob/748b3dde057bfc90623e186842b14538a0162049/pulp_rpm/app/tasks/synchronizing.py#L301
Can you share with us the contents of /repodata? Is there repomd.xml present?
Hello @munkey01,
@ipanova has a point, this error is only raised if there is an error finding repodata/repomd.xml
(see the context). It would be really helpful to know if repomd.xml
is there or not.
Another possible reason for get_repomd_file
not "finding" the file (aka raising ClientResponseError
) would be something related to the downloader configuration. It's a long shot, but I can look into that if the repomd.xml
file is confirmed to be available at repodata/repomd.xml
.
I think this is not related to slashes, although it may "look like" at first sight.
ps: just additional information, the 404 is expected when trying to get https://primary.env.company.com/pulp/content/application-x86_64-eng
(no slash) before pulpcore 3.40.0, but that not meaningfully because the sync task only tries to hit this because its thinking its a mirrorlist in the first place, as Ina already said.
Summary
I have 2 pulp servers, one of which serves as a primary (where I control package promotion, etc) and the other simply syncs repositories from the primary. I am using nginx as my reverse proxy and am also utilizing certgaurd as well (though I don't think this has impact here). The problem, is that my secondary server's rpm remote points to the primary's distribution and includes a trailing slash, but
pulp rpm sync
seems to be stripping that slash away, resulting in 404s.Steps to reproduce
Expected behavior
I expected the call to
pulp rpm repository sync
on the secondary to have retained the tailing forward slash as defined in the rpm remote.Stacktrace/Error log
Pulp and pulp-cli version info
Additonal context