DataBiosphere / toil

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.
http://toil.ucsc-cgl.org/.
Apache License 2.0
901 stars 240 forks source link

Support FTP over SSL #5167

Open stxue1 opened 5 days ago

stxue1 commented 5 days ago

See the previous attempt at #5142

ftplib includes functions for connecting to an FTP server securely, but we would need to enforce SSL on the actual download, which urlopen does not support. urlopen also will put the entire file in memory first, which we would not be able to do all the time. See some ideas to resolve that with our own pipe implementation here.

┆Issue is synchronized with this Jira Story ┆Issue Number: TOIL-1677