Error processing ImageSet configuration

[cadjai]

Version

$ oc-mirror version
v0.1.0-alpha.3

What happened?

I was trying to mirror operators using the snippet below as the imageset config file. Wh:q en I use the same structure without the version added to the file I got the mirror to run until I ran out of storage on the device but once I added the version I started getting the error below.

Attach the relevant portions of your .openshift_bundle.log. There was nothing in the log file but I got the error below on the command prompt

error: decode tmp-redhatgov.com/v1alpha1, Kind=ImageSetConfiguration: No Major.Minor.Patch elements found

What you expected to happen?

A bundle created with list of operators defined in the the imageset config file.

How to reproduce it (as minimally and precisely as possible)?

$ oc-mirror --config operator-imageset-channels-config.yaml file:///data/registry-oc-mirror

Anything else we need to know?

The reason the above file only contain operators is because our existing processes that have various ansible playbooks dedicated to mirroring a single type of content like operators, OCP images, binaries, helm charts .... The goal is to simplify the process by replacing multiple tasks with the single oc-mirror command to produce the same bundle and. So we will be defining a file for each type of content being mirrored.

Also it would be good to have sample config files with a couple of operators listed with versions in the readme file so people can use that as a guide or example to follow when creating their own imageset config.

Building the binary and running the list sub commands worked great. Those sub commands are very straight forward and easier to use that the current approach in the OCP docs. Great job on providing those to make our lives easier.

The other observation I have is that if you provide a list of operators without version, the mirror command does not seem to honor the list or if it does, it tries to pull all available images for each of the operators in the list. I was not able to finish the process because of no space left on the device error (after 265GB of content was mirrored into the workspace directory), I ran into while using a subset of the operators from the file above. I was expecting to get the latest version of the operand images like Arvin A. 's code does when a list of operators is provided. We will need a way to specify that we only need the latest version without having to figure out the version and then add it to the imageset config file.

I also saw some other errors related to images not found (e.g. servicemeshoperator.v2.0.4 had missing images and was causing the mirror to fail until I added --skip-missing option to ignore that).

References

Enter text here.

[jpower432]

:wave: @cadjai. Thank you for your feedback! Could you please post the imageset config you are working with? Also, we do have a config reference under docs, but I totally understand how having an examples directory that has configs with certain use cases would be helpful. I can submit a PR to add more example configurations today.

[cadjai]

operator-imageset-channels-config.yaml.txt

Loading the file as .txt because yaml does not seem to be supported

[afflom]

@cadjai This looks like an issue with our docs. We haven't done a good job of documenting the config (especially for operators) To demo redhat-operator-catalog:v4.7 / elasticsearch-operator from your config. For me to find the version of the operator, I would use the following workflow:

alex@penguin:~/dev/bundle$ ./bin/oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7 --package=elasticsearch-operator
PACKAGE                 CHANNEL     HEAD
elasticsearch-operator  5.0         elasticsearch-operator.5.0.9-50
elasticsearch-operator  stable      elasticsearch-operator.5.2.2-21
elasticsearch-operator  stable-5.1  elasticsearch-operator.5.1.4-9
elasticsearch-operator  stable-5.2  elasticsearch-operator.5.2.3-31
alex@penguin:~/dev/bundle$ ./bin/oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7 --package=elasticsearch-operator --channel=stable-5.2
VERSIONS
5.2.3-31

From what I can tell, your config conflates operator package channels and versions. A valid elasticsearch section from your config would be correct like this:

        - name: elasticsearch-operator
          versions: 
            - '5.2.3-31'

Operator selection behavior will be getting an overhaul shortly, so some of the logic processing of your config will change soon. Hopefully that makes it easier to author these configs too.

[afflom]

The behavior pertaining to operators without version specification will benefit from the overhaul that I referenced above.

[cadjai]

Hello Alex Thank you for addressing some of my comments. I will give it a try again. Also to clarify how I got the version I am using I ran something like the following commands

oc-mirror list operators --catalog=registry.redhat.io/redhat/redhat-operator-index:v4.7
oc-mirror list operators --catalog=registry.redhat.io/redhat/community-operator-index:v4.7

and that gave me a list of the operator with versions that I plugged in the imageset so that I didn't have to run the command one at a time but I will give that a try. I see the error I made. I skipped a step and used channel as version.

Got it working finally after using the right versions. I am uploading the working version of the imageset file in case it might help someone Thank you operator-imageset-version-config.yaml.txt