xtrabackup support

[janaurka]

Hey

I’ve tried to create a dev release out of that, there was an error with fetching xtrabackup from the blobstore:

jan@itzli:~/Documents/Repos$ git clone https://github.com/gogolok/shield-boshrelease.git
Cloning into 'shield-boshrelease'...
remote: Counting objects: 1660, done.
remote: Total 1660 (delta 0), reused 0 (delta 0), pack-reused 1660
Receiving objects: 100% (1660/1660), 317.23 KiB | 336.00 KiB/s, done.
Resolving deltas: 100% (847/847), done.
Checking connectivity... done.
jan@itzli:~/Documents/Repos$ cd shield-boshrelease/
jan@itzli:~/Documents/Repos/shield-boshrelease$ ls
ci  config  doc  jobs  LICENSE.md  packages  Rakefile  README.md  releases  src  templates
jan@itzli:~/Documents/Repos/shield-boshrelease$ git checkout xtrabackup 
Branch xtrabackup set up to track remote branch xtrabackup from origin.
Switched to a new branch 'xtrabackup'
jan@itzli:~/Documents/Repos/shield-boshrelease$ bosh create release --force
Syncing blobs...
postgres/postgresql-9.4.5.tar.gz downloaded
utils/jq downloaded
utils/yaml2json downloaded
tar/libarchive-3.1.2.tar.gz downloaded
elasticsearch-cloud-aws-plugin/elasticsea... downloaded
mysql/mysql-5.6.28.tar.gz downloaded
nginx/nginx-1.8.0.tar.gz downloaded
nginx/pcre-8.37.tar.gz downloaded
mariadb/mariadb-10.1.14.tar.gz downloaded
shield/shield-server-linux-amd64.tar.gz downloaded
Please enter development release name: |shield| shield-xtrabackup-dev

Building DEV release
---------------------------------
Release artifact cache: /home/jan/.bosh/cache

Building license
----------------
Building license...
  Using final version '79267db2e7516e504dfe773afdfd03a5ecfaf43a'
  Downloading from blobstore (id=a1b3d539-9fa0-4a85-812b-3a28bd559ca8)...

Building packages
-----------------
Building bsdtar...
  Using final version 'fe279cae5da0aa7b9cbf07238dad8afba6c8edc7'
  Downloading from blobstore (id=9c3a3f47-b329-4cf3-b638-a981aa55862d)...

Building common...
  Using final version 'e401816a4748292163679fafcbd8f818ed8154a5'
  Downloading from blobstore (id=ef29492b-7e97-4181-80a9-3f85d44af791)...

Building elasticsearch-cloud-aws-plugin...
  Using final version 'a8701cdd6895f248d4d51fd918e4d71daa678b4f'
  Downloading from blobstore (id=1ffec92c-86a4-42c6-988e-f33665a1c867)...

Building mariadb...
  Using final version '527563776d016f4aa93fa3e9b03f8cd65e2d825a'
  Downloading from blobstore (id=521ccf00-acea-4034-938a-013ea76050c1)...

Building nginx...
  Using final version '3e69a75504769646a78bc7464722f10c73489129'
  Downloading from blobstore (id=8a428eeb-b94b-482c-a37e-7378510ff53e)...

Building plugins...
  Using final version '47597e3ffd2161c2145d84b80f30ef996c14e60a'
  Downloading from blobstore (id=641e5fb7-09ee-4c60-8305-d9c467793b9e)...

Building postgres-9.4...
  Using final version '06a51985e0701707b27d45c7a5757171b5cefb8c'
  Downloading from blobstore (id=e59d3b19-5402-448c-a4c1-8a72cb9656e4)...

Building shield-mysql...
  Using final version 'ccf715e8091054ac22cdcbeb9ded4ab213adaef7'
  Downloading from blobstore (id=ec72431b-bd25-4f20-bf9a-9169cbad9bdb)...

Building shield...
  Using final version '9ef3344ee95cbae002d19798b0258047c929b1e2'
  Downloading from blobstore (id=8688963d-b748-4ec5-af40-f4b734df3205)...

Building utils...
  Using final version 'c33cc8c7e9c3927d3b5cd9f6f0e321dfde86bd9e'
  Downloading from blobstore (id=7dfd364e-61e1-4067-b665-fe77ca2c4c63)...

Building xtrabackup...
Package 'xtrabackup' has a glob that resolves to an empty file list: xtrabackup/**/*

Might it be that the file did not arrive at the s3 bucket blobstore configured in https://github.com/starkandwayne/shield-boshrelease/blob/master/config/final.yml ?

[gogolok]

The shield guys need to put it in the blob store. As long as they don't do it, you have to put all sources manually to src/.

[janaurka]

@gogolok: k, thx.

I’ve create a dev release with this PR and it did build fine and xtrabackup is available:

backup-agent/3fe41726-3434-4a61-aae7-02b43ee66c4d:/var/vcap/packages/xtrabackup/bin# pwd
/var/vcap/packages/xtrabackup/bin
backup-agent/3fe41726-3434-4a61-aae7-02b43ee66c4d:/var/vcap/packages/xtrabackup/bin# ./xtrabackup --version
./xtrabackup version 2.4.5 based on MySQL server 5.7.13 Linux (x86_64) (revision id: e41c0be)

This seems to work fine.

Source packages used to build the release: https://owncloud.r3pi.net/index.php/s/joWDJepZPqxJJtf The yml file of the release: https://gist.github.com/janaurka/0b028422236b0811da39515e88db232f The release: https://owncloud.r3pi.net/index.php/s/nlVv5bLIwJkWQv1

[mrferris]

Thank you @gogolok for the pull request and @janaurka for the extra testing! I put the xtrabackup blob in the blobstore.