Open bgruening opened 1 year ago
How many tools like that are there ? These should probably just be type="file"
instead of type="data"
?
Do these tools that natively process s3 do ranged reads or similar that would provide an efficiency gain? If not, might as well let Galaxy download and manage the file?
How many tools like that are there ? These should probably just be
type="file"
instead oftype="data"
?
One that I know :) Got a request for it.
Do these tools that natively process s3 do ranged reads or similar that would provide an efficiency gain? If not, might as well let Galaxy download and manage the file?
I have no idea. But I guess we should assume so, we will probably see more and more tools supporting this.
Tools popping up that can natively understand S3.
It would be nice if we can annotate the tools type="data" with a flag that indicates Galaxy that a file does not need to be cached in a POSIX filesystem before being submitted to the cluster, since the tool can work with
s3://bucket/prefix/key.mzML.gz
like path.This setting would need to be overwritable by an admin in case the cluster does not allow network access.