search

crc-org / crc

CRC is a tool to help you run containers. It manages a local OpenShift 4.x cluster, Microshift or a Podman VM optimized for testing and development purposes
https://crc.dev
Apache License 2.0
1.25k stars 236 forks source link

How to resolve default-route-openshift-image-registry.apps-crc.testing for podman machine #3897

Open praveenkumar opened 11 months ago

praveenkumar commented 11 months ago 
✗ podman machine ssh
Connecting to vm podman-machine-default. To close connection, use `~.` or `exit`

Last login: Thu Oct 26 17:18:15 2023 from 192.168.127.1

[core@localhost ~]$ podman login --tls-verify=false -u kubeadmin -p sha256~ZXNKqd_ZD6H8QJunOilCXuekkObv0gqoWbhb68oruHc default-route-openshift-image-registry.apps-crc.testing
Error: authenticating creds for "default-route-openshift-image-registry.apps-crc.testing": pinging container registry default-route-openshift-image-registry.apps-crc.testing: Get "http://default-route-openshift-image-registry.apps-crc.testing/v2/": dial tcp 127.0.0.1:80: connect: connection refused

[core@localhost ~]$ dig +short default-route-openshift-image-registry.apps-crc.testing
127.0.0.1

Question is how we should able to solve it for the users who are building the image using podman-machine instance and try to push it to openshift internal registry which expose by CRC. Some of the options we discussed on the slack internally but we want to open this discussion to broader audience to figure out best way.

Workaround

Currently the work around for this situation is add 192.168.127.254 for default-route-openshift-image-registry.apps-crc.testing for podman machine instance.

$ podman machine ssh -- 'echo "192.168.127.254 default-route-openshift-image-registry.apps-crc.testing" | sudo tee -a  /etc/hosts'

observed by some of other folks https://github.com/crc-org/crc/issues/3246

cfergeau commented 10 months ago

Another option, but I don't know if it's doable would be to improve gvisor-tap-vsock DNS resolution code: if a DNS request is resolved by the host and it returns 127.0.0.1, then change it to 192.168.127.254 which is the host IP from the virtual networking stack perspective.

cfergeau commented 10 months ago

Currently the work around for this situation is to add 192.168.127.254 for default-route-openshift-image-registry.apps-crc.testing in /etc/hosts in the podman machine instance.

As a short term workaround, could "something" do this automatically when both crc and podman-machine are used? Maybe the crc extension?

cfergeau commented 10 months ago

Another option, but I don't know if it's doable would be to improve gvisor-tap-vsock DNS resolution code: if a DNS request is resolved by the host and it returns 127.0.0.1, then change it to 192.168.127.254 which is the host IP from the virtual networking stack perspective.

The relevant code in gvisor-tap-vsock is https://github.com/containers/gvisor-tap-vsock/blob/c25d478e99ce5f9518901a833752c98e49405919/pkg/services/dns/dns.go#L92-L111

dgolovin commented 10 months ago

@cfergeau @praveenkumar should we open issue for gvisor to fix it?

praveenkumar commented 10 months ago

I discussed with @cfergeau and we are thinking if it is possible for crc-extenstion to run the https://github.com/crc-org/crc/issues/3897#issuecomment-1814280133 workaround for time being until we figure out better option.

BTW @dgolovin did you try the work around?

dgolovin commented 10 months ago

@praveenkumar it works from podman machine, but how I do the same from host OS? I started CRC instance only and I want to push image to CRC internal registry.

praveenkumar commented 10 months ago

it works from podman machine, but how I do the same from host OS?

host OS already have dns mapping for internal registry so it should just works. Which platfrom are you trying?

dgolovin commented 9 months ago

@praveenkumar I am on latest macOS Sonoma

dgolovin commented 9 months ago

@praveenkumar I am on Mac M1 macOS Sonoma. I do see default-route-openshift-image-registry.apps-crc.testing in hosts file along with developer console and api hosts but I cannot login into it using developer or kubadmin users.

dgolovin commented 9 months ago

@praveenkumar correction. I cannot login from podman desktop registries settings, but I can using 'podman login' command.

dgolovin commented 9 months ago

@praveenkumar it works in latest release, so I can login using podman login command, but there seems to be an issue when I try to login form podman-desktop registries page. So DNS resolution works for me now on macOS.

praveenkumar commented 9 months ago

@praveenkumar it works in latest release, so I can login using podman login command, but there seems to be an issue when I try to login form podman-desktop registries page. So DNS resolution works for me now on macOS.

https://github.com/crc-org/crc/issues/3897#issue-1965318439 actually describe why it is not work for podman-desktop registry page but it also have following workaround and that should work

podman machine ssh -- 'echo "192.168.127.254 default-route-openshift-image-registry.apps-crc.testing" | sudo tee -a  /etc/hosts'
dgolovin commented 9 months ago

@praveenkumar I am not following here. Podman Desktop registry page does not use podman login it seems to work with registry through REST api directly and it does not work.

praveenkumar commented 8 months ago

@praveenkumar I am not following here. Podman Desktop registry page does not use podman login it seems to work with registry through REST api directly and it does not work.

@dgolovin yes it works with API so this is what I did to add this registry to podman desktop registry option.

And it is now added to PD registry window.

Screenshot 2024-01-09 at 2 13 36 PM

Let me know if still have some confusion around it.

marcelomrwin commented 7 months ago

thank you @praveenkumar Your guidance was essential in overcoming this problem.

dgolovin commented 5 months ago

@praveenkumar sorry for a loooong wait. It does work after applying workaround and I can push to crc internal image registry, but only using `podman push' command. Podman Desktop push does not work, but it is not crc issue.

Thank you @praveenkumar!

praveenkumar commented 5 months ago

@cfergeau should we again relook it again and finalize what should we do?

cfergeau commented 1 month ago

@praveenkumar it works in latest release, so I can login using podman login command, but there seems to be an issue when I try to login form podman-desktop registries page. So DNS resolution works for me now on macOS.

#3897 (comment) actually describe why it is not work for podman-desktop registry page but it also have following workaround and that should work

podman machine ssh -- 'echo "192.168.127.254 default-route-openshift-image-registry.apps-crc.testing" | sudo tee -a  /etc/hosts'

gvisor-tap-vsock also provides a DNS server/API, but gvproxy does not expose this API by default :-/ If the API was exposed, we could use 

podman machine ssh -- curl http://192.168.127.1/services/dns/add -X POST -d '{"Name":".apps-crc.testing.","DefaultIP":"192.168.127.254"}'

as an alternative workaround.

brianmolinaspring commented 3 weeks ago

@praveenkumar Hello guys, are you still using the alternative option? Could this be solved in another way, can you please tell me?

praveenkumar commented 2 weeks ago

@praveenkumar Hello guys, are you still using the alternative option? Could this be solved in another way, can you please tell me?

As of now we are trying to evaluate what would be the best option to solve this without user to use this workaround. We will soon have something to share.

brianmolinaspring commented 2 weeks ago

good morning, @praveenkumar , thanks for your response, I will be attentive