search

IBM-Cloud / ibmcloud-image-builder

ibmcloud-image-builder
Apache License 2.0
10 stars 8 forks source link

ISSUE-8: ubuntu 18.04 base packer template #15

Closed syyang-in-cloud closed 4 years ago

syyang-in-cloud commented 4 years ago

Summary

This PR is to resolve the issues : issue-8 and some more to prepare issue-16.

Description

  • shell
  • Ansible

Piggybacked:

Tests

Manual Test Results as below. Confirmed the Ansible provisioner with the ubuntu release number:

Distributor ID:  Ubuntu
Description:     Ubuntu 18.04.4 LTS
Release:         18.04
Codename:        bionic

Manual Test Environment:

Host: Ubuntu 18.04.3 LTS
Docker: Ubuntu 20.04    LTS
Cloud Img: Ubuntu 18.04.4 LTS

Base Image: Ansible Playbook

qemu: output will be in this color.

==> qemu: Retrieving ISO
==> qemu: Trying https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img
==> qemu: Trying https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img?checksum=sha256%3A4ce2fbfc8bcdd186760614811e52bb38cfa2a969bcb298fc6acc8bca2e5b4e8f
bionic-server-cloudimg-amd64.img 329.81 MiB / 329.81 MiB  100.00% 17s01ss
==> qemu: https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img?checksum=sha256%3A4ce2fbfc8bcdd186760614811e52bb38cfa2a969bcb298fc6acc8bca2e5b4e8f => /ibmcloud-image-builder/packer/ubuntu/bionic/base/packer_cache/4b297036cbd2c89f35d6dd0efc52ed70092f18f2.iso
==> qemu: Copying hard drive...
==> qemu: Resizing hard drive...
==> qemu: Starting HTTP server on port 8426
==> qemu: Found port for communicator (SSH, WinRM, etc): 3923.
==> qemu: Looking for available port between 5900 and 6000 on 127.0.0.1
==> qemu: Starting VM, booting disk image
==> qemu: Overriding defaults Qemu arguments with QemuArgs...
==> qemu: Waiting 10s for boot...
==> qemu: Connecting to VM via VNC (127.0.0.1:5916)
==> qemu: Typing the boot command over VNC...
==> qemu: Using ssh communicator to connect: 127.0.0.1
==> qemu: Waiting for SSH to become available...
==> qemu: Connected to SSH!
==> qemu: Provisioning with shell script: shell/install.sh
    qemu: Install something here: TBD
==> qemu: Provisioning with shell script: shell/cleanup.sh
    qemu: Cleanup here: TBD
==> qemu: Provisioning with Ansible...
    qemu: Setting up proxy adapter for Ansible....
==> qemu: Executing Ansible: ansible-playbook -e packer_build_name=qemu -e packer_builder_type=qemu -e packer_http_addr=10.0.2.2:8426 --ssh-extra-args '-o IdentitiesOnly=yes' -e ansible_ssh_private_key_file=/tmp/ansible-key798373852 -i /tmp/packer-provisioner-ansible497915787 /ibmcloud-image-builder/packer/ubuntu/bionic/base/ansible/playbook.yml
    qemu:
    qemu: PLAY [play Hello World] ********************************************************
    qemu:
    qemu: TASK [Gathering Facts] *********************************************************
    qemu: ok: [default]
    qemu:
    qemu: TASK [task Hello World] ********************************************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [debug] *******************************************************************
    qemu: ok: [default] => {
    qemu:     "msg": "Hello World!\nDistributor ID:\tUbuntu\nDescription:\tUbuntu 18.04.4 LTS\nRelease:\t18.04\nCodename:\tbionic"
    qemu: }
    qemu:
    qemu: PLAY RECAP *********************************************************************
    qemu: default                    : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
    qemu:
==> qemu: Gracefully halting virtual machine...
==> qemu: Converting hard drive...
Build 'qemu' finished.

Tested Extra Playbook for the next Issue issue-16 : playbook-docker.yml

qemu: output will be in this color.

==> qemu: Retrieving ISO
==> qemu: Trying https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img
==> qemu: Trying https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img?checksum=sha256%3A4ce2fbfc8bcdd186760614811e52bb38cfa2a969bcb298fc6acc8bca2e5b4e8f
==> qemu: https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-amd64.img?checksum=sha256%3A4ce2fbfc8bcdd186760614811e52bb38cfa2a969bcb298fc6acc8bca2e5b4e8f => /ibmcloud-image-builder/packer/ubuntu/bionic/base/packer_cache/4b297036cbd2c89f35d6dd0efc52ed70092f18f2.iso
==> qemu: Copying hard drive...
==> qemu: Resizing hard drive...
==> qemu: Starting HTTP server on port 8792
==> qemu: Found port for communicator (SSH, WinRM, etc): 2378.
==> qemu: Looking for available port between 5900 and 6000 on 127.0.0.1
==> qemu: Starting VM, booting disk image
==> qemu: Overriding defaults Qemu arguments with QemuArgs...
==> qemu: Waiting 10s for boot...
==> qemu: Connecting to VM via VNC (127.0.0.1:5926)
==> qemu: Typing the boot command over VNC...
==> qemu: Using ssh communicator to connect: 127.0.0.1
==> qemu: Waiting for SSH to become available...
==> qemu: Connected to SSH!
==> qemu: Provisioning with shell script: shell/install.sh
    qemu: Install something here: TBD
==> qemu: Provisioning with shell script: shell/cleanup.sh
    qemu: Cleanup here: TBD
==> qemu: Provisioning with Ansible...
    qemu: Setting up proxy adapter for Ansible....
==> qemu: Executing Ansible: ansible-playbook -e packer_build_name=qemu -e packer_builder_type=qemu -e packer_http_addr=10.0.2.2:8792 --ssh-extra-args '-o IdentitiesOnly=yes' -e ansible_ssh_private_key_file=/tmp/ansible-key604861440 -i /tmp/packer-provisioner-ansible783161695 /ibmcloud-image-builder/packer/ubuntu/bionic/base/ansible/playbook-docker.yml
    qemu:
    qemu: PLAY [play Install Geerlingguy Docker] *****************************************
    qemu:
    qemu: TASK [Gathering Facts] *********************************************************
    qemu: ok: [default]
    qemu:
    qemu: TASK [Add universe repository for bionic] **************************************
    qemu: ok: [default]
    qemu:
    qemu: TASK [Install gnupg2] **********************************************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : include_tasks] **************************************
    qemu: skipping: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : include_tasks] **************************************
    qemu: included: /root/.ansible/roles/geerlingguy.docker/tasks/setup-Debian.yml for default
    qemu:
    qemu: TASK [geerlingguy.docker : Ensure old versions of Docker are not installed.] ***
    qemu: ok: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Ensure dependencies are installed.] *****************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Add Docker apt key.] ********************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Ensure curl is present (on older systems without SNI).] ***
    qemu: skipping: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Add Docker apt key (alternative for older systems without SNI).] ***
    qemu: skipping: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Add Docker repository.] *****************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Install Docker.] ************************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Ensure Docker is started and enabled at boot.] ******
    qemu: ok: [default]
    qemu:
    qemu: RUNNING HANDLER [geerlingguy.docker : restart docker] **************************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : include_tasks] **************************************
    qemu: included: /root/.ansible/roles/geerlingguy.docker/tasks/docker-compose.yml for default
    qemu:
    qemu: TASK [geerlingguy.docker : Check current docker-compose version.] **************
    qemu: ok: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Delete existing docker-compose version if it's different.] ***
    qemu: skipping: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : Install Docker Compose (if configured).] ************
    qemu: changed: [default]
    qemu:
    qemu: TASK [geerlingguy.docker : include_tasks] **************************************
    qemu: included: /root/.ansible/roles/geerlingguy.docker/tasks/docker-users.yml for default
    qemu:
    qemu: TASK [geerlingguy.docker : Ensure docker users are added to the docker group.] ***
    qemu: changed: [default] => (item=ubuntu)
    qemu:
    qemu: PLAY RECAP *********************************************************************
    qemu: default                    : ok=16   changed=8    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0
    qemu:
==> qemu: Gracefully halting virtual machine...
==> qemu: Converting hard drive...
Build 'qemu' finished.
syyang-in-cloud commented 4 years ago

It's weird, the PR pipeline was successful but the status isn't updated here: https://travis-ci.org/github/IBM-Cloud/ibmcloud-image-builder/builds/703568289

syyang-in-cloud commented 4 years ago

I am going to merge, the test was successful:

    qemu: changed: [default]
7956    qemu:
7957    qemu: TASK [debug] *******************************************************************
7958    qemu: ok: [default] => {
7959    qemu:     "msg": "Hello World!\nDistributor ID:\tUbuntu\nDescription:\tUbuntu 18.04.4 LTS\nRelease:\t18.04\nCodename:\tbionic"
7960    qemu: }
7961    qemu:
7962    qemu: PLAY RECAP *********************************************************************
7963    qemu: default                    : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
7964    qemu:
79652020/06/30 15:11:38 packer-provisioner-ansible plugin: shutting down the SSH proxy
79662020/06/30 15:11:38 [INFO] (telemetry) ending ansible
7967==> qemu: Gracefully halting virtual machine...
79682020/06/30 15:11:38 packer-builder-qemu plugin: Executing shutdown command: rm -f /home/ubuntu/.ssh/authorized_keys && sudo rm -f /root/.ssh/authorized_keys && echo 'packer' | sudo -S shutdown -P now
79692020/06/30 15:11:38 packer-builder-qemu plugin: [DEBUG] Opening new ssh session
79702020/06/30 15:11:38 packer-builder-qemu plugin: [DEBUG] starting remote command: rm -f /home/ubuntu/.ssh/authorized_keys && sudo rm -f /root/.ssh/authorized_keys && echo 'packer' | sudo -S shutdown -P now
79712020/06/30 15:11:41 packer-builder-qemu plugin: [ERROR] Remote command exited without exit status or exit signal.
79722020/06/30 15:11:41 packer-builder-qemu plugin: Waiting max 5m0s for shutdown to complete
79732020/06/30 15:13:15 packer-builder-qemu plugin: VM shut down.
7974==> qemu: Converting hard drive...
79752020/06/30 15:13:15 packer-builder-qemu plugin: Executing qemu-img: []string{"convert", "-O", "qcow2", "output-qemu/ubuntu-bionic.qcow2", "output-qemu/ubuntu-bionic.qcow2.convert"}
79762020/06/30 15:13:17 packer-builder-qemu plugin: stdout:
79772020/06/30 15:13:17 packer-builder-qemu plugin: stderr:
79782020/06/30 15:13:21 packer-builder-qemu plugin: failed to unlock port lockfile: close tcp 127.0.0.1:5977: use of closed network connection
79792020/06/30 15:13:21 packer-builder-qemu plugin: failed to unlock port lockfile: close tcp 127.0.0.1:3344: use of closed network connection
7980Build 'qemu' finished.
7981
7982==> Builds finished. The artifacts of successful builds are:
7983--> qemu: VM files in directory: output-qemu
79842020/06/30 15:13:21 [INFO] (telemetry) ending qemu
7985==> Builds finished. The artifacts of successful builds are:
79862020/06/30 15:13:21 machine readable: qemu,artifact-count []string{"1"}
79872020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "builder-id", "transcend.qemu"}
79882020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "id", "VM"}
79892020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "string", "VM files in directory: output-qemu"}
79902020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "files-count", "1"}
79912020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "file", "0", "output-qemu/ubuntu-bionic.qcow2"}
79922020/06/30 15:13:21 machine readable: qemu,artifact []string{"0", "end"}
79932020/06/30 15:13:21 [INFO] (telemetry) Finalizing.
79942020/06/30 15:13:21 waiting for all plugin processes to complete...
79952020/06/30 15:13:21 /usr/local/bin/packer: plugin process exited
79962020/06/30 15:13:21 /usr/local/bin/packer: plugin process exited
79972020/06/30 15:13:21 /usr/local/bin/packer: plugin process exited
7998+ NEW_IMAGE=output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2
7999+ ENCRYPTED_IMAGE=output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G-encrypted.qcow2
8000+ qemu-img resize output-qemu/ubuntu-bionic.qcow2 100G
8001Image resized.
8002+ qemu-img convert -f qcow2 -O qcow2 output-qemu/ubuntu-bionic.qcow2 output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2
8003+ qemu-img info output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2
8004image: output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2
8005file format: qcow2
8006virtual size: 100 GiB (107374182400 bytes)
8007disk size: 1.03 GiB
8008cluster_size: 65536
8009Format specific information:
8010    compat: 1.1
8011    lazy refcounts: false
8012    refcount bits: 16
8013    corrupt: false
8014+ rm output-qemu/ubuntu-bionic.qcow2
8015+ '[' -n '' ']'
8016+ SECRET=JustMySimpleSecret
8017++ base64
8018++ echo -n JustMySimpleSecret
8019+ BASE64_ENCODED_SECRET=SnVzdE15U2ltcGxlU2VjcmV0
8020+ qemu-img convert -O qcow2 --object secret,id=sec0,format=base64,data=SnVzdE15U2ltcGxlU2VjcmV0 -o encrypt.format=luks,encrypt.key-secret=sec0 output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2 output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G-encrypted.qcow2
8021+ qemu-img compare --object secret,id=sec0,format=base64,data=SnVzdE15U2ltcGxlU2VjcmV0 --image-opts driver=qcow2,file.filename=output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G.qcow2 driver=qcow2,encrypt.key-secret=sec0,file.filename=output-qemu/ibmcloud-ubuntu-bionic-cloudimg-amd64-100G-encrypted.qcow2
8022Images are identical.
8023+ rm -rf output-qemu
8024+ rm -rf packer_cache
8025+ rm -rf ./ssh
8026+ rm -f user-data
8027+ rm -f disk-ssh-pub.img
8028The command "make" exited with 0.
8029
8030
8031Done. Your build exited with 0.