search

BCDevOps / developer-experience

This repository is used to track all work for the BCGov Platform Services Team (This includes work for: 1. Platform Experience, 2. Developer Experience 3. Platform Operations/OCP 3)
Apache License 2.0
8 stars 17 forks source link

Documentation - Platform Operations making use of Artifactory for internal components/services in Openshift #4266

Closed wmhutchison closed 4 months ago

wmhutchison commented 1 year ago

Describe the issue Now that we are committing to make use of Artifactory for internal image distribution (essential short-term for the NSX ncp image), we want to document how to make use of Artifactory internally going forward.

Additional context Add any other context, attachments or screenshots

How does this benefit the users of our platform? Reduced amount of churn for re-used images such as Nagios monitoring as we work through maintenance upgrades.

Definition of done

wmhutchison commented 1 year ago

Rough process for uploading ncp image as downloaded from VMWare to Artifactory.

Login to Artifactory with Platform Ops account.

Goto top-right for your logged in user, select "Edit Profile" and then "Generate an Identity Token". What is generated will be your password for logging into Artifactory with your username the same as the username your'e currently logged into Artifactory web console with.

Import the "ubi" tar file into local podman storage on the UTIL server.

podman load -i nsx-ncp-ubi-xxx.tar

Login to the Artifactory registry with your local account. Paste the long token generated from Artifactory when prompted for the password.

podman login -u wmhutchison@github artifacts.developer.gov.bc.ca/plat-cluster-docker-local

Run podman images to see how the imported image is referenced.

Tag the desired image name:tag for injecting into Artifactory, and then push the new image into Artifactory.

podman tag registry.local/4.1.1.0.22071564/nsx-ncp-ubi artifacts.developer.gov.bc.ca/plat-cluster-docker-local/nsx-ncp-ubi:4.1.1.0
podman push artifacts.developer.gov.bc.ca/plat-cluster-docker-local/nsx-ncp-ubi:4.1.1.0
wmhutchison commented 1 year ago

Stored securely in Platform Ops own password management, the following tokens exist with Reader access to Artifactory. The goal will be to import each of these into the global pull secrets for all Openshift clusters (NSX-based clusters for now).

lab-cluster-access: used for all non-PROD Openshift clusters. prod-cluster-access: used for all PROD Openshift clusters.

wmhutchison commented 7 months ago

Manually reviewing the accuracy of this ticket for the brain-dump portion.

The mentioned global secrets section where there are only two is incorrect, the lab-cluster-access and prod-cluster-access tokens are not what's used, each cluster has been given its own token to put into the global pull secrets for each Openshift cluster.

wmhutchison commented 7 months ago

Moving this into Sprint Backlog for now as a means of aligning with my last Retrospective where I did state I'd try and give documentation some more TLC.

wmhutchison commented 6 months ago

starting to put some thought into this, documenting this will also double as the start of our work to migrate image builds we normally curate for installing/upgrading clusters to Artifactory as well. Ignoring that Trident already is done on its own, the following namespace/image combos are found:

openshift-bcgov-nagios/ansible
openshift-bcgov-nagios/dhcpcheck
openshift-bcgov-nagios/nginx
openshift-bcgov-nagios/proxy
openshift-bcgov-vrops/vrops
openshift-bcgov-encase/encase
openshift-bcgov-image-registry-notifier/irn
openshift-bcgov-pruner/tenant-scaler
openshift-bcgov-perfmon/perfmon

As to versioning/tagging, my thought is to adopt the following which will make it super-easy at a glance to see when the image being referenced was built approximately.

YYYYqN-vN

So building one of the above images for the first time this year would result in the tag of 2024q1-v1, and if new version is built in the same quarter, 2024q1-v2 and so on.

StevenBarre commented 6 months ago

Many of these images use the ocp-cli as a base, or are otherwise tied to the OCP version. Since we have mixes of versions between labs/prod/nsx, can we include the OCP version in there somewhere too?

wmhutchison commented 6 months ago

I'm fine with that, ocp413-v1 or ocp412-nsx-v3 would work then, incrementing version numbers as we refresh the image either to align with an ocp refresh, or modifying the image for another reason (new monitor/check, and so on).

wmhutchison commented 5 months ago

Some of the previous comments are out of scope for this particular ticket, we need to keep it focused on how in general we can make technical use of Artifactory for our own images. Specifics for how to use it during operations like OCP upgrades will be handled separately as a documentation change to the upgrade process when we get to that stage.

wmhutchison commented 5 months ago

This ticket unfortunately didn't receive much attention this past sprint due to the priority and time-sink of attempting to upgrade KLAB2 to OCP 4.14. That issue is now on hold for a little while, so will make getting tickets like this done early on in the next sprint.

wmhutchison commented 5 months ago

Creating a new ZenHub tracking ticket for both myself and Cailey, involving Artifactory and the repo(s) we are officially setup on for Platform Ops images such as ncp. If resolved in a timely fashion, then will be able to include relevant screenshots in this documentation PR.

An older repo setup named plat-cluster-docker-local was once created for use by the Ops team to upload images such as ncp. This was however deprecated via a newer/improved setup for a repo titled cluster-docker-local instead.

The latter repo is where all live use of the ncp image resides, but logging into Artifactory's web console, it does not show that repo, but only the plat-cluster-docker-local repo is shown. This has caused confusion the last few times ncp images were updated, since a look into Artifactory's console shows that repo, and then image update is loaded there, but Openshift cannot pull from that repo since it's not properly federated.

The informal take-away when discussing this with Cailey is that she needs to clean-up/remove the old unused repo, and take additional steps to ensure the official repo can be properly seen in Artifactory, thus eliminating all future confusion. Hence getting a proper ticket for this, since this issue has been forgotten for a while now. :)

wmhutchison commented 5 months ago

Cailey will be removing the old repo via separate ticket created and assigned to her. Confirmed that the repo we are officially using can in fact be seen in the Artifactory web console, but is visible using the filter of "all" instead of the platform-services filter. This will reduce future confusion massively going forward, and will ensure this is covered in future docs.

wmhutchison commented 5 months ago

Dumping into Notepad the raw text of what will be the basis of the new Markdown docs. While not a personal fan of too many screen-shots as part of documentation due to having to refresh the screenshots when the application itself is upgraded, a fair amount of what needs to be shown in Artifactory's web console is simply too ambiguous to describe accurately with just text.

Given how often Artifactory itself receives upgrades, good use of screenshots should not be a major issue. Screenshots will only be invaluable for the current team's onboarding to make use of Artifactory or training a new team member.

wmhutchison commented 5 months ago

Now actively putting digital pen to paper, expecting to have an initial published first run of this document online for peer review before OCP Talks on Friday, though aiming for sooner, operations issues permitting.

wmhutchison commented 5 months ago

https://github.com/bcgov-c/advsol-docs/pull/307 created in Draft. Doing some internal reviews before turning it loose for formal review/approval.