Only parse manifest folders with valid bundle files

[jeremy-wl]

We previously require the nested manifest directories to have a SemVer name. But this blocks builds from another team that uses courier in their pipeline.

So we need to relax that requirement and parse all directories even if it does not have a SemVer name.

The requirement is removed in https://github.com/operator-framework/operator-courier/pull/116 and https://github.com/operator-framework/operator-courier/pull/118 when we verify manifest directory and run flatten command. But we need to further refactor it to only parse valid manifest folders, and ignore the rest (instead of raising errors when invalid folders are parsed). In the meantime, we also need to properly identify whether the input source directory structure is flat or nested after parsing all directories and files, and perform different actions accordingly.

APIs affected:

• build_and_verify(source_dir, ...)
• build_verify_and_push(source_dir, ...)
• flatten

CLI subcommands affected

• verify
• push
• flatten

Valid Nested Layout

• Previously, all nested manifest directories will be parsed and the API does not check if they contain valid manifest files until invalid files are encountered and errors are raised.
• Now, only the manifest directories with valid manifest files (meaning at least 1 valid CSV file is present inside the directory) will be parsed, and only valid CSV and CRD files will be extracted from the directories.
• If the source directory satisfies both nested and flat structure, meaning both valid manifest directories and valid CSV+package file are found in source directory root, we parse the source directory as it is in nested layout, and ignores the CSV in source root.

Valid Flat Layout

• Previously, as long as there are subdirectories found in source_dir, we think of the structure as nested.
• Now, since all subdirectories are processed first. If no valid manifest directories are found in source_dir, but there are still valid CSV+package file found in source_dir, then we will consider it to be a valid flat structure.

[jeremy-wl]

/cc @kevinrizza @MartinBasti

[kevinrizza]

@jeremylinlin Your PR description describes the limitations of what is happening now in terms of how you wanted the code to be refactored, but it does not actually describe what has changed from the perspective of how the API is affected. Could you please explain in your description (and commit message) which edge cases this PR is attempting to resolve, and what is expected from this?

[jeremy-wl]

@kevinrizza Thanks for the comment! I've updated the description and commit message.

[jeremy-wl]

@kevinrizza Thanks for the comments! I have addressed them and please take a look for new changes.

[kevinrizza]

/lgtm

[jeremy-wl]

Hey @MartinBasti, could you please also review it when you get a chance? Thank you!

[openshift-ci-robot]

New changes are detected. LGTM label has been removed.

[jeremy-wl]

@MartinBasti Thank you very much for the comments! I have addressed all of them and please take a look when you are free.