search

berman-lab / ymap

YMAP - Yeast Mapping Analysis Pipeline : An online pipeline for the analysis of yeast genomic datasets.
MIT License
6 stars 6 forks source link

FTP uploads #3

Closed vladimirg closed 9 years ago

vladimirg commented 9 years ago

Allow users to upload the datasets via FTP, using any old FTP client (such as FileZilla or Cyberduck). The uploads must be resumable after an interruption or a pause.

darrenabbey commented 9 years ago

The ability to resume uploads would be dependent on the FTP protocol itself. Keeping the uploaded files in the location where they're uploaded would be all that is required for this.

The difficulty is in setting up FTP accounts and access for arbitrary users of the system.

An intermediate solution for the problem this would solve (minimizing cumulative transfer times) would be to copy every file uploaded through the current interface into a common directory (per-user) that can then later be accessed by that user. The initial upload of a data file would still be the slow step, but for reanalysis or new comparisons, the reduction in time would be dramatic.

vladimirg commented 9 years ago

Resuming in FTP does require a particular server configuration, though I agree that setting up the user permissions is probably the trickiest step.

For 1.0 I'd really like this solved for first-time uploads, not just re-analysis, because this is the first thing the user experiences, and if it fails, the user might abandon Ymap completely.

darrenabbey commented 9 years ago

Setting this up in a secure way remains problematic. Lab directors have been consulted and agree that FTP drop access should be setup for an account only with direct negotiation with the lab.

It would be a simpler task on a dedicated server, but I don't think this is in the plans for the near future.

vladimirg commented 9 years ago

Why is FTP access so problematic?

I've also googled a little online, and found this: http://www.resumablejs.com . What do you think? On the face of it sounds like a better alternative. Most browsers have HTML5 support now (see http://caniuse.com/#feat=fileapi ).

darrenabbey commented 9 years ago 
Problematic because the server administrators don’t want to open the system up in that way.

On Apr 7, 2015, at 4:34 PM, Vladimir Gritsenko notifications@github.com wrote:

Why is FTP access so problematic?

I've also googled a little online, and found this: http://www.resumablejs.com http://www.resumablejs.com/ . What do you think? On the face of it sounds like a better alternative. Most browsers have HTML5 support now (see http://caniuse.com/#feat=fileapi http://caniuse.com/#feat=fileapi ).

— Reply to this email directly or view it on GitHub https://github.com/darrenabbey/ymap/issues/3#issuecomment-90738568.

vladimirg commented 9 years ago

Alright, closing this in favor of resumable.js.