19.05 update conda + fix slurm not starting

[abretaud]

So here's a fix that repairs the bioblend tests locally with the standalone image Let's see if it gets greener!

[abretaud]

Any chance to get longer jobs on travis? Bioblend tests look better for standalone image, but it stops at 50' Compose images still look broken though

[bgruening]

We have now 90 min now

[abretaud]

Cool, thanks! I'm looking into the compose stuff, don't know if I'll find a way to repair it

[abretaud]

What's the aim of the compose changes: do we want to get rid of the galaxy-init image and put everything in galaxy-web? (which means galaxy-base would only be used by galaxy-web?)

[bgruening]

I'm fine with skipping compose for this release. Next release is already around the corner. I'm not sure when we find time to get the needed changes in in time.

@gmauro was also planning to rewrite the testing. So that we have more bash file for testing and less logic in travis. This would also make it much easier for local testing and we could easier migrate to CircleCI or Azure.

[bgruening]

What's the aim of the compose changes: do we want to get rid of the galaxy-init image and put everything in galaxy-web? (which means galaxy-base would only be used by galaxy-web?)

Yes, there are/where the goals:

• get rid of galaxy-init
• more, proper testing
• newer PostrgeSQL version
• separate out the handlers to its own containers -> spin up new handler-containers without restarting the containers
• slim down container size

[bgruening]

ping also @erasche that was working on this at some point

[abretaud]

Ok, I'm ok with skipping compose for now :) (and I agree, less logic in travis would be greaaat) So for the standalone container, is the checklist at the top of https://github.com/bgruening/docker-galaxy-stable/pull/486 still valid?

• [ ] use new handler assignments by default -> not sure what that means, is it the same as "separate out the handlers to its own containers" = compose version?
• [ ] write database migration guide PG9 to PG11
• [ ] remove autofs for CMVFS, we can mount it just in by default, this would remove the need for --priv mode

[bgruening]

The database migration guide is probably needed, as we did the database migration. The other items we can move over to 19.09 or 20.01 ;)

use new handler assignments by default

Newer Galaxy can make use of newer PG features to assign jobs to handlers. Nothing urgent, but good to use it to get it under testing for the main repo as well.

[abretaud]

So I'm testing upgrading a 19.01 container, and there are some things to fix :D First, ansible is not installed in the image (removed here) but is needed by the startup script (for example here) I'll add it back to the image unless there's a better way (the image should get ~60Mb bigger) I see some condor error too, looking into it

[abretaud]

I still have some db problem, is there anything to do to migrate an existing db to postgres 11?

[bgruening]

I know only pg_dmup (https://www.postgresql.org/docs/9.0/migration.html). So dump and read all in again psql -f backup.sql postgres

[bgruening]

First, ansible is not installed in the image (removed here) but is needed by the startup script (for example here)

Oh, so true ... for later in the year. Is the 60MB worth to find a new way to update the ini files on the fly?

[abretaud]

Oh, so true ... for later in the year. Is the 60MB worth to find a new way to update the ini files on the fly? I can live with it for now :)

Ok for postgres 9 -> 11, I thought there was some magic at startup, I'll look into it

[bgruening]

@abretaud I'm not sure we should do the automatic migration. I'm happy if we document how to migrate for the users. They should either way do a backup.

[abretaud]

Yep I agree, I was just thinking about writing the doc while doing it on real data

[abretaud]

Ok, added some docs, does it look good? Also added something in startup to clean compiled_templates, when updating galaxy, I sometimes need to do it manually (like from 19.01 to 19.05) I can make some tests with GGA tools this afternoon, unless I find new problems it looks good to me now

[abretaud]

oh, found another problem... galaxy-wait not found at startup, looking into it

[abretaud]

It should be good now @bgruening, ready for review/merge when you have time

[bgruening]

Top!