search

minishift / minishift-centos-iso

CentOS based ISO as an alternative for boot2docker ISO
GNU Lesser General Public License v3.0
40 stars 33 forks source link

Handle user data issues for RHEL #200

Closed gbraad closed 6 years ago

gbraad commented 6 years ago

Refer: https://github.com/minishift/minishift-centos-iso/issues/200#issuecomment-361182659 There is inconsistency when using systemd's unit minishift-handle-user-data on RHEL.

# First start, working

[docker@minishift ~]$ mount | grep sda1 | wc -l
5
[docker@minishift ~]$ mount | grep sda1
/dev/sda1 on /mnt/sda1 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)

# on restart, it seems to be working

[docker@minishift ~]$ mount |grep sda1 | wc -l
14
[docker@minishift ~]$ mount |grep sda1
/dev/sda1 on /mnt/sda1 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /mnt/sda1/var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /mnt/sda1/var/lib/boot2docker/etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker/etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /mnt/sda1/var/lib/boot2docker/etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker/etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /mnt/sda1/var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)

# on restart again and failing

[docker@minishift ~]$ mount | grep sda1 | wc -l
7
[docker@minishift ~]$ mount | grep sda1
/dev/sda1 on /var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /etc/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /mnt/sda1 type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/boot2docker type ext4 (rw,relatime,seclabel,data=ordered)
/dev/sda1 on /var/lib/minishift type ext4 (rw,relatime,seclabel,data=ordered)
[docker@minishift ~]$
# on restart I see
[root@localhost ~]# journalctl -u minishift-handle-user-data
-- Logs begin at Fri 2018-01-12 05:09:42 EST, end at Fri 2018-01-12 05:17:42 EST. --
Jan 12 05:10:02 localhost systemd[1]: Starting Minishift handle user data...
Jan 12 05:10:03 localhost minishift-handle-user-data[699]: mount p:sda1 ...
Jan 12 05:10:06 localhost minishift-handle-user-data[699]: activate swap ...
Jan 12 05:10:06 localhost minishift-handle-user-data[699]: umount: /var/lib/docker: not mounted
Jan 12 05:10:08 localhost minishift-handle-user-data[699]: mount: special device /mnt/sda1/var/lib/boot2docker/etc/docker does not exist
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: total 28
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: drwxr-xr-x. 3 root root  4096 Jan 12 05:01 etc
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: drwx------. 2 root root 16384 Jan 12 05:01 lost+found
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: drwxr-xr-x. 2 root root  4096 Jan 12 05:01 opt
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: drwxr-xr-x. 3 root root  4096 Jan 12 05:01 var
Jan 12 05:10:09 localhost minishift-handle-user-data[699]: Unable to find userdata.tar
Jan 12 05:10:09 localhost systemd[1]: minishift-handle-user-data.service: main process exited, code=exited, status=1/FAILURE
Jan 12 05:10:09 localhost systemd[1]: Failed to start Minishift handle user data.
Jan 12 05:10:09 localhost systemd[1]: Unit minishift-handle-user-data.service entered failed state.
Jan 12 05:10:09 localhost systemd[1]: minishift-handle-user-data.service failed.
gbraad commented 6 years ago

https://github.com/minishift/minishift-centos-iso/blob/9e06665d912924c11f18007701d50b0288b6f446/scripts/handle-user-data#L130

gbraad commented 6 years ago

we accidentally trigger 'handle user data' twice in RHEL ISO.

After we moved to using systemd, we disabled running rc.local, however by accident we left the code for rc.local in the RHEL iso.

gbraad commented 6 years ago

Reboots and restarts now work as expected...

 gbraad  ~  minishift-1.9.0-linux-amd64/minishift start --iso-url file:///home/gbraad/minishift-rhel-1.0.0-1.iso
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ... 
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... 
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Starting profile 'minishift'
-- Starting local OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    1 GB
   vCPUs :    1
   Disk size: 5 GB
-- Starting Minishift VM ........................... OK
-- Registering machine using subscription-manager
   Red Hat Developers or Red Hat Subscription Management (RHSM) username: gbraad-1
   Red Hat Developers or Red Hat Subscription Management (RHSM) password: [HIDDEN]
   Registration in progress ............................... OK [1m35.1s]
-- Checking for IP address ... OK
-- Checking if external host is reachable from the Minishift VM ... 
   Pinging 10.0.75.1 ... OK
-- Checking HTTP connectivity from the VM ... 
   Retrieving http://10.0.75.1:5000/v2/ ... FAIL
   VM cannot connect to external URL with HTTP
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- OpenShift cluster will be configured with ...
   Version: v3.6.0
-- Checking `oc` support for startup flags ... 
   host-config-dir ... OK
   host-data-dir ... OK
   host-pv-dir ... OK
   host-volumes-dir ... OK
   routing-suffix ... OK
Starting OpenShift using openshift/origin:v3.6.0 ...
Pulling image openshift/origin:v3.6.0
...
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift ssh                                                                             130 
[docker@minishift ~]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[docker@minishift ~]$ sudo reboot                                                                                                           
Connection to 192.168.42.90 closed by remote host.
Cannot establish SSH connection to the VM: exit status 255
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift ssh                                                                               1 
[docker@localhost ~]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[docker@localhost ~]$  exit
logout
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift stop                                                                                  
Stopping local OpenShift cluster...
Unregistering machine
Cluster stopped.
 gbraad  ~  minishift-1.9.0-linux-amd64/minishift start --iso-url file:///home/gbraad/minishift-rhel-1.0.0-1.iso                          
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ... 
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... 
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Starting profile 'minishift'
-- Starting local OpenShift cluster using 'kvm' hypervisor ...
-- Starting Minishift VM .............................. OK
-- Registering machine using subscription-manager
   Red Hat Developers or Red Hat Subscription Management (RHSM) username: gbraad-1
   Red Hat Developers or Red Hat Subscription Management (RHSM) password: [HIDDEN]
   Registration in progress ................................. OK [1m45.1s]
-- Checking for IP address ... OK
-- Checking if external host is reachable from the Minishift VM ... 
   Pinging 10.0.75.1 ... OK
-- Checking HTTP connectivity from the VM ... 
   Retrieving http://10.0.75.1:5000/v2/ ... FAIL
   VM cannot connect to external URL with HTTP
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- OpenShift cluster will be configured with ...
   Version: v3.6.0
-- Checking `oc` support for startup flags ... 
   routing-suffix ... OK
   host-config-dir ... OK
   host-data-dir ... OK
   host-pv-dir ... OK
   host-volumes-dir ... OK
Starting OpenShift using openshift/origin:v3.6.0 ...
Pulling image openshift/origin:v3.6.0
^C
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift ssh                                                                             130 
[docker@minishift ~]$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[docker@minishift ~]$ exit
logout
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift ssh -- sudo reboot                                                                1 
Connection to 192.168.42.90 closed by remote host.
Cannot establish SSH connection to the VM: exit status 255
 gbraad  ~  ./minishift-1.9.0-linux-amd64/minishift ssh -- docker ps                                                                  1 
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
 gbraad  ~  
gbraad commented 6 years ago

Hyper-V and IP assignment works with this ISO:

PS C:\work\src\github.com\minishift\minishift\out\windows-amd64> .\minishift.exe start --network-ipaddress 10.0.75.127 --iso-url file://C:/Users/gbraad/Downloads/minishift-rhel-1.0.0-1.iso --network-gateway 10.0.75.1
-- Starting profile 'minishift'
-- Checking if requested hypervisor 'hyperv' is supported on this platform ... OK
-- Checking if Hyper-V driver is installed ... OK
-- Checking if Hyper-V driver is configured to use a Virtual Switch ...
   'DockerNAT' ... OK
-- Checking if user is a member of the Hyper-V Administrators group ... OK
-- Checking the ISO URL ... OK
-- Starting local OpenShift cluster using 'hyperv' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Determing netmask ...
-- Determing nameservers to use ... OK
-- Set the following network settings to VM ...
   Device:      eth0
   IP Address:  10.0.75.127/255.255.255.0
   Gateway:     10.0.75.1
   Nameservers: 10.0.21.1
-- Starting Minishift VM .................................................... OK
-- Registering machine using subscription-manager
   Red Hat Developers or Red Hat Subscription Management (RHSM) username: gbraad-1
   Red Hat Developers or Red Hat Subscription Management (RHSM) password: [HIDDEN]
   Registration in progress ................. OK [41s]
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ...
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
-- OpenShift cluster will be configured with ...
   Version: v3.6.1
-- Checking `oc` support for startup flags ...
   host-config-dir ... OK
   host-data-dir ... OK
   host-pv-dir ... OK
   host-volumes-dir ... OK
   routing-suffix ... OK
Starting OpenShift using openshift/origin:v3.6.1 ...
Pulling image openshift/origin:v3.6.1
Pulled 1/4 layers, 26% complete
Pulled 1/4 layers, 60% complete
Pulled 2/4 layers, 87% complete
Pulled 3/4 layers, 99% complete
Pulled 4/4 layers, 100% complete
Extracting
Image pull complete
OpenShift server started.

The server is accessible via web console at:
    https://10.0.75.127:8443

You are logged in as:
    User:     developer
    Password: <any value>

To login as administrator:
    oc login -u system:admin

-- Waiting for persistent volumes to be created ... OK

-- Applying addon 'xpaas':.Error during post cluster up configuration: Error executing command oc apply -f xpaas-streams/fis-image-streams.json -n openshift.

however, and issue with xpaas occurs (unrelated)