[WIP] Move to Galaxy 18.01

[pcm32]

This PR moves both our galaxy container and the setup for using the community images-based setup to Galaxy 18.01, plus our ISA (datatype) and Kubernetes runner changes (better handling of jobs on instance restart). This should be tested with helm charts galaxy version >= 0.4.0 and galaxy-stable version >= 1.5.5 probably.

I would appreciate if people could test this locally and try some tools before we merge.

[djcomlab]

Taking a look now...

[sneumann]

I had several of these in IIRC galaxy-k8s-runtime:dev_v17.09-pheno-lr_cv1.6.190, where some version number quotes were not properly quoted, hence the init/install step failed before Galaxy was started, see https://github.com/phnmnl/container-galaxy-k8s-runtime/commit/43df9ead641e4ebaa9ab3f504fd83c2f1e96107c

[djcomlab]

@sneumann Thanks for the pointer... doesn't look like this is the issue. Actually some lines before have some warnings from the phnmnl/galaxy source install:

isa-rwval 0.10.2 has requirement six==1.10.0, but you'll have six 1.11.0 which is incompatible.

that may have halted the base installation. Will fix that and see if it progresses.

[djcomlab]

I've issued a PR to update phnmnl/galaxy's isa-rwval dependency version to latest, which in turn updates its own dependecy of six to be compatible with Galaxy 18.01's dependencies. Just waiting for those changes to get pulled in and can then test again incase that was the blocker.

[pcm32]

Thanks for trying this @djcomlab!

Maybe I should have been more explicit on how to run these, sorry about the trouble.

For the galaxy chart (the one you usually run so far):

helm install --set galaxy_image="pcm32/galaxy",galaxy_image_tag=":18.01-pheno-isa",galaxy_image_registry="",pv_minikube="yes",galaxy_pull_policy="IfNotPresent",log_file_to_track="galaxy.log" galaxy-helm-repo/galaxy

here please replace the galaxy_image and galaxy_image_tag with what you build from the Dockerfile (as there will be no official image yet). You might want to add the usual galaxy admin email, key and password so that workflows are added and user created.

For the galaxy-stable chart (to the one we are moving to; this is different to the 17.09 - 18.01 move):

https://github.com/galaxyproject/galaxy-kubernetes/blob/develop/README-galaxy-stable.md

using this config:

https://github.com/galaxyproject/galaxy-kubernetes/blob/develop/example_configs/galaxy-stable-phenomenal-18.01-minikube.yaml

[djcomlab]

Thanks @pcm32, that's got it running! The changes I described earlier need to happen anyway...

[pcm32]

Yeah, I agree is important to pin things down.

[pcm32]

Could you get all your tools running without issues @djcomlab?

[djcomlab]

@pcm32 Testing right now before committing some updates to the tools.

[djcomlab]

I've merged some quite a bit off stuff into here from the ISA hotfix PR, so another review would probably be useful.

[pcm32]

Hi @sneumann as I discussed with @djcomlab we want to untangle the 18.01 part from the ISA hotfix, so I will force push my local version of this PR and we will merge the ISA hotfix on its own... then I'll rebase these changes.

[pcm32]

Will close this one in favour of #195. Will rebase #195 once isa hotfix is merged to develop.