search

saltstack / salt

Software to automate the management and configuration of any infrastructure or application at scale. Install Salt from the Salt package repositories here:
https://docs.saltproject.io/salt/install-guide/en/latest/
Apache License 2.0
14.21k stars 5.48k forks source link

Unable to create local mirror of saltstack repo, due to filelists.xml not being available #67032

Closed salman8328 closed 1 week ago

salman8328 commented 3 weeks ago

Description Unable to create local mirror of saltstack repo, due to filelists.xml not being available

Setup

sync the saltstack repo using pulp and this url: https://packages.broadcom.com/ui/repos/tree/General/saltproject-rpm

Steps to Reproduce the behavior sync that rpm repo, it will fail with error:

"description": "XML file(s): filelists not found"

Expected behavior local mirror created

Additional context When the repo url has been moved to public Jfrog url, this issue occured, so the steps to solve it would be to enable filelists.xml indexing in jfrog.

welcome[bot] commented 3 weeks ago

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey. Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar. If you have additional questions, email us at saltproject@vmware.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

ScriptAutomate commented 3 weeks ago

I'm taking a look at this to see if we can get that enabled for the rpm repo.

In the meantime, we have alternative directions for pulling down the repos on Artifactory:

ssoto2 commented 3 weeks ago

Just some detail we are also seeing issue with the repo not being accessable, it was noticed when yum/dnf updating 

SaltStack latest Release Channel for RHEL/CentOS 8                                                                                                                                                                            0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'saltstack':
  - Curl error (6): Couldn't resolve host name for https://repo.saltproject.io/salt/py3/redhat/8/x86_64/latest/repodata/repomd.xml [Could not resolve host: repo.saltproject.io]
Error: Failed to download metadata for repo 'saltstack': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
Ignoring repositories: saltstack
clayoster commented 3 weeks ago

@ssoto2 The repositories at repo.saltproject.io no longer exist. More details on the transition can be found here:

https://saltproject.io/blog/salt-project-package-repo-migration-and-guidance/

Ivca50V commented 2 weeks ago

Issuing the same problem when trying to mirror repo on Foreman: [E|bac|9161b7fb] XML file(s): filelists not found (Katello::Errors::Pulp3Error)

abdelhousni commented 2 weeks ago

Hello team, We currently use redhat satellite(foreman) for managing our rpm repositories. When creating the repos following your instructions : https://saltproject.io/blog/salt-project-package-repo-migration-and-guidance/

I encounter the following error :
XML file(s): filelists not found (Katello::Errors::Pulp3Error)

I found out that the reason is from the data type used returned by this file : https://packages.broadcom.com/artifactory/saltproject-rpm/repodata/repomd.xml should be : <data type="filelists"> instead of <data type="other"> or <data type="primary">

you will find the detailed fix from this article : https://community.theforeman.org/t/external-private-yum-repository-on-jfrog-io-not-synchronizing-in-foreman-katello/24938

Thank you in advance for the fix and for all your hard work Best regards Abdellatif

MartinEmrich commented 2 weeks ago

I read elsewhere that Artifactory can be configured to enable/disable creation of the filelists file. I also get an Error message on yum based OS. Maybe Salt Project/Broadcom can turn it on?

ssoto2 commented 2 weeks ago

@MartinEmrich We use Jfrog and it can be turned on, don't know if Salt has a specific reason for not turning it on but it is available in the manage repo settings of Jfrog

Screenshot from 2024-11-06 13-11-03

How to Enable in Jfrog

1) As administrator 2) Click Administration 3) Click Repositories on the left-hand bar 4) In main panel click on the repository in question 5) Under the Basic section find enable file list indexing and toggle on

Screenshot from 2024-11-07 13-59-48 Screenshot from 2024-11-07 14-00-09

mmacdo02-tufts commented 2 weeks ago

Hi, We also use Red Hat Satellite/Pulp in our organization to mirror Salt repos so we definitely would like to have this fixed. This should definitely be enabled in JFrog. I'm pretty there going to be a huge percentage of people that run into this exact issue, and I'd prefer not to have to manually mirror/download this repo.

Katello::Errors::Pulp3Error: XML file(s): filelists not found

ScriptAutomate commented 1 week ago

Enable File List Indexing has now been enabled on the saltproject-rpm repository, and should work with tools that seek a <data type="filelists"> entry in the repomd.xml file.

Can see the new updates here: