enable http registries as ocm repositories

fabianburth commented 7 months ago

Description

So far, it was not possible for the ocm tooling to communicate with OCI registries with http, but only with https. The reason therefore was primarily that the scheme was not considered in the parsing and it defaulted to https. This PR extends the code, especially the parsing rules, to be able to also handle http.

Furthermore, it was not possible to specify localhost in OCI commands (e.g. ocm get artifact). This PR also extends the code to be able to handle localhost.

Consequently, the following specifications are now possible: For OCI commands (such as ocm get artifact): http://example.com/repo:version http://localhost:8080/repo:version localhost:8080/repo:version(The port, thus :8080 is required, here. So, localhost/repo:version is not possible, since e.g. localhost/example:1.0.0 is also a valid dockerhub artifact reference and would therefore be ambiguous.) localhost//repo:version (The scheme, thus http:// can be omitted, if the host is separated from the repo with an unambiguous double slash (//).) http://localhost:8080//repo:version (A combination of the previous two formats is also possible, although kind of unintuitive.)

For OCM commands (such as ocm transfer): OCIRegistry::http://localhost:80/test//github.com/example (The type, thus OCIRegistry:: is required) OCIRegistry::localhost:80/test//github.com/example (Works as well without the scheme, but will default to https) and the variations with an actual domain OCIRegistry::http://example.com:80/test//github.com/example http://example.com:80/test//github.com/example example.com:80/test//github.com/example (will default to https) example.com/test//github.com/example (will default to https) ...

Furthermore, since using http might potentially be insecure and should only be used in test scenarios, the user should get a warning. To enable this, a corresponding log message was introduced on log level warn and the general default log level for the cli tool was changed from error to warn.

Short types for repositories that were expected to work (e.g. oci) are now registered with corresponding repository and repository spec handlers.

make generate also checks whether the examples are running. This failed on mac, since the component of the 01-getting-started did not contain an arm64 darwin executable due to a mistake in its build. This is now fixed, so after the next release make generate will be callable on mac again. Until then, one can call e.g. go generate ./cmds/ocm/... manually.

What type of PR is this? (check all applicable)

[X] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📝 Documentation Update
[ ] 🎨 Style
[ ] 🧑‍💻 Code Refactor
[ ] 🔥 Performance Improvements
[ ] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI
[ ] 📦 Chore (Release)
[ ] ⏩ Revert

Added tests?

[X] 👍 yes
[ ] 🙅 no, because they aren't needed
[ ] 🙋 no, because I need help
[ ] Separate ticket for tests # (issue/pr)

Added to documentation?

Adjusted the existing ocm references and oci references cli documentations and added additional examples for the most common use cases.

Checklist:

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[X] I have added tests that prove my fix is effective or that my feature works
[X] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

Skarlso commented 7 months ago

I'm honestly a bit against this. There is a reason for not supporting insecure repositories. ocm-controller is using self-signed certificates to set up testing registries that use HTTPS. It's super easy to create a secure registry for testing.

In the wild you should never EVER use an insecure HTTP registry. So I don't see the point of this change.

fabianburth commented 7 months ago

I'm open for discussion. Currently, you will at least get a warning if you use http and if you do not specify a scheme, it will default to https. But on one hand, there are several people that just want to start a registry locally and fool around a bit to get to know ocm's functionality. On the other hand, I'm not sure whether it really is that easy for all application using ocm to get your application trust the self-signed certificate.

Skarlso commented 7 months ago

@fabianburth you're right, the lower the entry for getting started the better the adoption. :)

Skarlso commented 7 months ago

On the other hand, I'm not sure whether it really is that easy for all application using ocm to get your application trust the self-signed certificate.

You can use mkcert to generate a self-signed, trusted cert, than when you are launching the registry, just pass the cert in as an environment property and everything should work. :)

fabianburth commented 7 months ago

So, I'm not strictly talking about command line usage here. Let's consider integration tests and say you start your application and also your registry in cluster. Now, it might already be kind of a hassle to get your application to trust the certificate, mightn't it?

Skarlso commented 7 months ago

Now, it might already be kind of a hassle to get your application to trust the certificate, mightn't it?

Not really, no. In fact, we use an HTTPS registry in our e2e tests here:

https://github.com/open-component-model/ocm-controller/tree/main/e2e

You'll see that we deploy cert-manager, generate a certificate, download it, install in the local store using mkcert, and done. Everything is running with HTTPS.

This is all we needed to do to get things running https://github.com/open-component-model/ocm-controller/blob/main/hack/prime_test_cluster.sh.

We only test using HTTPS.

The same goes for github actions which is our CI.

open-component-model / ocm