search

SUSE-Cloud / automation

Automation scripts for development, testing, and CI
Apache License 2.0
68 stars 139 forks source link

Switch Cloud8 to use legacy ISO server (SOC-11466) #3945

Closed rtamalin closed 3 years ago

rtamalin commented 3 years ago

Recent changes to the mirroring policies for ibs-mirror.prv.suse.net mean that it no longer mirrors the SLE 12 SP3 ISO that is needed by the SOC 8 CLM cobbler (PXE Boot) re-imaging process.

I have manually setup a mirror of the necessary ISO on the legacy CI artifacts server for Ardana, ardana.ci.prv.suse.net, with the same directory hierarchy, so we can just switch the hostname in the URL to fix this.

Also updated an occurrence of the old suse.provo.cloud version of the name for the legacy CI artifacts server.

rtamalin commented 3 years ago

https://ci.suse.de/job/openstack-ardana/849/ kicked off to validate that this patch fixes the issues downloading the ISO... If it gets past the Prepare Environment stage, e.g. gets to Deploy Cloud or later, then this patch is working as intended ;-)

JanZerebecki commented 3 years ago

Why introduce another server? Can't we mirror it on clouddata?

KeithMnemonic commented 3 years ago

Why introduce another server? Can't we mirror it on clouddata?

Jan, it was there but since SLES-12-SP3 is old they deleted it and this is an alternative from them deleting it once again in the future.

KeithMnemonic commented 3 years ago

Why introduce another server? Can't we mirror it on clouddata?

Jan, it was there but since SLES-12-SP3 is old they deleted it and this is an alternative from them deleting it once again in the future.

https://chat.suse.de/channel/cloud-cicd?msg=8iFBYMDHBgtANB4dM

JanZerebecki commented 3 years ago

clouddata != ibs-mirror (here variable name sles_media_server). clouddata is maintained by us, ibs-mirror isn't. an ibs-mirror location is removed in this version of the patch. That chat conversation might be wrong, clouddata might not have had the isos before. But we could put it there, if we have an existing url in NUE where we can mirror the iso from we can add it in https://github.com/SUSE-Cloud/automation/blob/master/hostscripts/clouddata/syncSLErepos . Otherwise we can manually copy it there.

rtamalin commented 3 years ago

clouddata != ibs-mirror (here variable name sles_media_server). clouddata is maintained by us, ibs-mirror isn't. an ibs-mirror location is removed in this version of the patch. That chat conversation might be wrong, clouddata might not have had the isos before. But we could put it there, if we have an existing url in NUE where we can mirror the iso from we can add it in https://github.com/SUSE-Cloud/automation/blob/master/hostscripts/clouddata/syncSLErepos . Otherwise we can manually copy it there.

So per the discussion about this last week, the clouddata mirroring process, as implemented in syncSLErepos, is mirroring ISO content rather than the ISOs themselves, and I think only Ardana requires the SLE ISOs to be available like this, so we only really need this to support Ardana. Changing syncSLErepos to mirror the repos is more than just a one liner change that had previously been suggested because we actually need the SLE ISO images, not the Cloud ISO images, and not just their content. And given what just happened there is no guarantee that the same situation won't arise at a future date if we rely on the mirroring process.

The ardana.ci.prv.suse.net virtual host is served by static.ci.prv.suse.net, which is a key part of the legacy CI that is required to exist to support HOS CI build process, which has to remain functional until August/September 2022, and is backed by a support contracted 3Par, and thus should be a pretty safe bet to keep running until we no longer need it... Additionally the legacy artifact server is setup such that all published files are immutable so no risk of accidental deletions. And, finally, it was already serving the required SLE 12 SP3 ISO anyway, as part of the initial Ardana rebranding/Cloud8 CLM development process, so serving the files to the ECP CI is not a significant extra burden.

And thus this approach represents a minimal change to SUSE-Cloud/automation to get the cobbler/PXEboot working again... Additionally I've already setup the SLE 12 SP4 ISOs to be served by ardana.ci.prv.suse.net as well, so that we can potentially switch Cloud9 settings to use this source as well if we decide too, or the need arises.

So in summary, yes we could change syncSLErepos to do what is needed, but it would entail more than just the one line change that this patch implements to fix the problem, which is fundamentally breaking this aspect of the ECP CI workflow for Ardana testing right now.

And whichever approach we take will require about the same amount of effort to address the impact of this breakage in ardana/ardana-dev-tools, which I will be doing as well, so from, my perspective, opting for the minimal overall change to the systems and code bases involved makes the most sense.

If you feel that it should be handled by syncSLErepos, or by manually setting up provo-clouddata.cloud.suse.de to serve the ISOs, feel free to implement the necessary changes to either the script or the system itself, and propose the necessary changes.

JanZerebecki commented 3 years ago

While it might be planned to switch HOS8 CI off after 2022, that is not true for SOC8 CI. But it can be changed again then.

I'm not against merging this.

rtamalin commented 3 years ago

@JanZerebecki IIUC, around the same time the HOS 8 CI contract ends, the Provo systems "move/decommission" will happen so any on-going SOC 8 CI will likely need TLC to get it working on the moved/replacement environment, wherever that may end up being.