todaygood / openshift-lab

lab on openshift
0 stars 0 forks source link

从外部访问registry失败 #3

Open todaygood opened 6 years ago

todaygood commented 6 years ago

Issue

参见 https://docs.openshift.com/container-platform/3.3/install_config/registry/securing_and_exposing_registry.html#exposing-the-registry

[root@ose0 ~]# oc get route 
NAME               HOST/PORT                                              PATH      SERVICES           PORT      TERMINAT
ION   WILDCARDdocker-registry    docker-registry-default.apps.test.cloud.genomics.cn              docker-registry    <all>     passthro
ugh   Noneregistry-console   registry-console-default.apps.test.cloud.genomics.cn             registry-console   <all>     passthro
ugh   None[root@ose0 ~]# docker images
REPOSITORY                                   TAG                 IMAGE ID            CREATED             SIZE
docker.io/busybox                            latest              22c2dd5ee85d        12 days ago         1.16 MB
172.30.165.209:5000/openshift/busybox        latest              22c2dd5ee85d        12 days ago         1.16 MB
docker.io/openshift/origin-web-console       v3.9.0              aa12a2fc57f7        4 weeks ago         495 MB
docker.io/openshift/origin-service-catalog   v3.9.0              96cf7dd047cb        4 weeks ago         296 MB
docker.io/openshift/origin-pod               v3.9.0              6e08365fbba9        4 weeks ago         223 MB
[root@ose0 ~]# docker tag docker.io/busybox docker-registry-default.apps.test.cloud.genomics.cn/openshift/busybox 
[root@ose0 ~]# docker push docker-registry-default.apps.test.cloud.genomics.cn/openshift/busybox
The push refers to a repository [docker-registry-default.apps.test.cloud.genomics.cn/openshift/busybox]
Get https://docker-registry-default.apps.test.cloud.genomics.cn/v1/_ping: dial tcp: lookup docker-registry-default.apps.test.cloud.genomics.cn on 192.168.122.245:53: no such host

看要求是这个hostname 能被DNS解析。

todaygood commented 6 years ago

DNS解析的问题Solution

在host上配置libvirt network dns

[root@cloud-sz-kolla-b13-01 opt]# virsh net-dumpxml default 
<network>
  <name>default</name>
  <uuid>a4f99d86-1c21-4741-9820-2d0c63a2deb8</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:ed:8c:a6'/>
  <domain name='cloud.genomics.cn'/>
  <dns>
    <forwarder addr='192.168.60.236'/>
  </dns>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>
[root@cloud-sz-kolla-b13-01 opt]# egrep -v "^$|^#"  /etc/dnsmasq.conf 
server=/localnet/192.168.122.1
local=/localnet/
address=/.apps.test.cloud.genomics.cn/192.168.122.245
except-interface=virbr0 virbr1
listen-address=192.168.60.236
no-dhcp-interface=virbr0 virbr1
bind-interfaces
conf-dir=/etc/dnsmasq.d

使之生效

systemctl start dnsmasq ; systemctl enable dnsmasq 
virsh net-destroy default 
virsh net-edit default 
virsh net-start default 
add vm  vnet  into virbr0 

这样下来,在vm中就可以解析这个hostname了。

[root@ose0 ~]# dig docker-registry-default.apps.test.cloud.genomics.cn

; <<>> DiG 9.9.4-RedHat-9.9.4-61.el7 <<>> docker-registry-default.apps.test.cloud.genomics.cn
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 56508
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;docker-registry-default.apps.test.cloud.genomics.cn. IN        A

;; ANSWER SECTION:
docker-registry-default.apps.test.cloud.genomics.cn. 0 IN A 192.168.122.245

;; Query time: 1 msec
;; SERVER: 192.168.122.245#53(192.168.122.245)
;; WHEN: Sun Jul 29 16:08:00 CST 2018
;; MSG SIZE  rcvd: 85
todaygood commented 6 years ago
  1. route hostname 这个要map到router pod所在的node ip上去,如图,可能有多个node ip

image

用哪一个ip呢? 哪一个都可以,不过最好是配置 一个 vip 参见: https://docs.openshift.com/container-platform/3.4/dev_guide/expose_service/expose_internal_ip_router.html#expose-router-svc

这里,先用ose7 192.168.122.103; 将wildcard dns 条目中的Ip修改为 192.168.122.103 ,参见上面的/etc/dnsmasq.conf address=/.apps.test.cloud.genomics.cn/192.168.122.103 即可。

  1. 用route hostname去访问 image
[root@ose0 ~]# oc get svc docker-registry
NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
docker-registry   ClusterIP   172.30.165.209   <none>        5000/TCP   4d

即可用 docker-registry-default.apps.test.cloud.genomics.cn:80 代替 172.30.165.209:5000

  1. 验证
    
    [root@ose0 ~]# docker pull nginx 
    Using default tag: latest
    Trying to pull repository docker.io/library/nginx ... 
    latest: Pulling from docker.io/library/nginx
    be8881be8156: Pull complete 
    32d9726baeef: Pull complete 
    87e5e6f71297: Pull complete 
    Digest: sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
    Status: Downloaded newer image for docker.io/nginx:latest

[root@ose0 ~]# docker tag c82521676580 docker-registry-default.apps.test.cloud.genomics.cn/openshift/nginx

[root@ose0 ~]# docker push docker-registry-default.apps.test.cloud.genomics.cn/openshift/nginx The push refers to a repository [docker-registry-default.apps.test.cloud.genomics.cn/openshift/nginx] 08d25fa0442e: Pushed a8c4aeeaa045: Pushed cdb3f9544e4c: Pushed latest: digest: sha256:4ffd9758ea9ea360fd87d0cee7a2d1cf9dba630bb57ca36b3108dcd3708dc189 size: 948