search

galaxyFlavorsGenerators / galaxyFlavorGenerator

Configure and generate Galaxy server instances
http://galaxyflavorsgenerators.github.io/galaxyFlavorGenerator/
7 stars 2 forks source link

Offer an Amazon build #4

Open bgruening opened 9 years ago

bgruening commented 9 years ago

Would be nice to also offer a Amazon Image with all tools. This should be implemented as additional backend.

bgruening commented 9 years ago

@afgane ping We can now generate arbitrary tools lists. I guess it's time to work on the AWS backend :) Can you guide us?

afgane commented 9 years ago

That'd be excellent! Conceptually, it's pretty simple - we just generate the tools_list file and run the galaxy-cloudman-playbook to create the galaxy file system, which is then used by cloud instances at launch time.

There are two problems though: reference data and not all tools install cleanly. I guess the problematic tools can be fixed as they're ID'd but I haven't found a reliable automated method for getting the reference data installed. Plus, it's quite large, hence takes a lot of time (i.e., days). Switching to Galaxy's ObjectStore for it is something that's been on the todo list for years. That would grab the required data from a public S3 bucket on demand and largely solve this problem but that also requires a fair bit of work/time. How do you deal with the ref data for containers? (btw, I'm still planning on exploring the option to just use the container as the Galaxy installation for the default cloud images).

bgruening commented 9 years ago

The installation of tools should be fixed no matter which Image we create. This are probably general issues. Reference data are currently not installed by default, but we install data managers to make it easy for the admins. We do have a few special scripts in this container to automatically build indices with the data managers: https://github.com/bgruening/galaxy-rna-workbench-extra to provide ready to go training instances. Not sure this is a good idea for general purpose images, since they get really big and DockerHub will probably not host it. @afgane let me know if I can assist you in basing Cloudman on the container.

bgruening commented 9 years ago

@afgane do you think we can offer a shell script that runs the playbook with the tool_list file, so we can demonstrate the concept and searching eagerly for someone that pays the bills for the backend ;)

afgane commented 9 years ago

Sure - I can give it a try. Is there a particular toolset you'd like to use or should we just use the default for now?

Also, is there a target date by when you'd like to be able to demo this? Perhaps we can work on it while you're visiting?

bgruening commented 9 years ago

The tool set will be given by the user, we can work on this during my visit ... sounds good! Looking forward to it! :)