generating a reverse FTP tool

[GuyReeves]

Hi

Would like to generate a reverse FTP tool, so users could download zip files of multiple datasets to a location outside galaxy. Downloading to the users hard drive is an increasingly poor option as files become larger.
The export location could be folder(s) specified by admins as a drop down menu.

There is a simple tool which handles the compression of multiple datasets though a data collection (handles BAMs and the BAI index correctly).

https://toolshed.g2.bx.psu.edu/repository?repository_id=d077871367f67b47 what is missing is the skill to place the location of the .zip file to be a location outside galaxy. Would anybody have any ides how to do this? Cheers

Guy

This idea is briefly outlined in. https://biostar.usegalaxy.org/p/22760/

[bgruening]

@GuyReeves this folder should be on the same server then Galaxy? Please join this discussion and vote for it: https://github.com/galaxyproject/galaxy/issues/2968

[GuyReeves]

Yes on the same server.

On 7. May 2017, at 09:02, Björn Grüning notifications@github.com wrote:

@GuyReeves https://github.com/GuyReeves this folder should be on the same server then Galaxy? Please join this discussion and vote for it: galaxyproject/galaxy#2968 https://github.com/galaxyproject/galaxy/issues/2968 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/galaxyproject/tools-iuc/issues/1295#issuecomment-299686786, or mute the thread https://github.com/notifications/unsubscribe-auth/ANMJrOpx7eJDsfpbU_YJm1K9fWN8VCdLks5r3WyOgaJpZM4NSQzw.

[yhoogstrate]

@GuyReeves to me this does feel more like a framework level type of tool/feature. What is your opinion about this?

[GuyReeves]

My instinct is to make the tool if it is easy. Then get views from users as to their experience and suggestions as to how or if to make it a galaxy framework thing.

I do wonder if an already existing tool might be simply adapted if it can be made to specific a path outside galaxy.

Thanks

Guy

On 18. May 2017, at 16:53, yhoogstrate notifications@github.com wrote:

@GuyReeves https://github.com/guyreeves to mee this does feel more like a framework level type of tool/feature. What is your opinion about this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/galaxyproject/tools-iuc/issues/1295#issuecomment-302428443, or mute the thread https://github.com/notifications/unsubscribe-auth/ANMJrCi1gUDjRQRQ-iS_IKk59vZVwNUdks5r7FtlgaJpZM4NSQzw.

[nsoranzo]

Some time ago I created this tool for my users: https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/

It just copies the file to shared partition on our cluster. The Galaxy admin can specify the initial path of the partition.

Hope that helps!

[GuyReeves]

Dear HTH

This sounds exactly what I was after I will try installing it tomorrow. It even as the admin specifying the path.

Thanks for alerting me to this

Guy

On 18. May 2017, at 20:25, Nicola Soranzo notifications@github.com wrote:

Some time ago I created this tool for my users: https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/ https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/ It just copies the file to shared partition on our cluster. The Galaxy admin can specify the initial path of the partition.

HTH

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/galaxyproject/tools-iuc/issues/1295#issuecomment-302501151, or mute the thread https://github.com/notifications/unsubscribe-auth/ANMJrEGIUrAKfFpQZpMF-RY7pkdMFoTYks5r7I0ZgaJpZM4NSQzw.

[GuyReeves]

@nsoranzo Do you have instruction as to how the the admin sets the initial path? Does the 'initial' bit mean that the user can specify the rest of the path? I cannot find instructions on the tool shed.

Cheers Guy

PS I suspect that the entry in the toolshed could be improved, if I wrote out more detailed instructions would you consider adding them?

[jennaj]

Pinging for update on this idea here and https://github.com/galaxyproject/galaxy/issues/2968.

I like the idea of making "this framework level type of tool/feature", ideally a reverse FTP. It seems more consistent with the existing functionality. If users could choose the datasets to include in the archive (via "operations on multiple datasets"?), that would be a huge plus.

But any solution that can be put on a public server would probably be good (new tool).

History archive/download/transfer has been problematic -- we need some way for people to download data more efficiently/faster, especially since data is growing larger and quotas exist. Download per-dataset is tedious.

adding @natefoo and @dannon to this in case they haven't seen it or the linked ticket

[abretaud]

I think this is addressed by the new external sources access built in Galaxy now