Globus file handling - Githubissues

cianfrocco-lab / COSMIC-CryoEM-Gateway

Web platform for analyzing cryo-EM data on Comet cluster at San Diego Supercomputer Center

http://cosmic-cryoem.org

GNU General Public License v3.0

9 stars 1 forks source link

Globus file handling #228

Closed mcianfrocco closed 4 years ago

mcianfrocco commented 4 years ago

Data handling for globus uploads:

If single file selected via Globus; upload into data list
If more than one file; must have star file
If it doesn’t, then remove uploaded data and give user error message: “Error: We did not find any STAR files in your uploaded directory. No data have been uploaded.”

monaw commented 4 years ago

@mcianfrocco - do we allow a single file of any type or only one that ends in .mrc?

mcianfrocco commented 4 years ago

Any type for now.

On Mon, Feb 3, 2020 at 7:43 PM monaw notifications@github.com wrote:

@mcianfrocco https://github.com/mcianfrocco - do we allow a single file of any type or only one that ends in .mrc?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/228?email_source=notifications&email_token=AALFRFMYBP2TSK7KJOI65S3RBC223A5CNFSM4KKM6XCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKV54GA#issuecomment-581688856, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALFRFJEZBBZI2RJP4WYP7TRBC223ANCNFSM4KKM6XCA .

monaw commented 4 years ago

tech note: need to update Transfer Status page to show OUR status, not Globus status because Globus may have successfully transferred multiple files but we consider the transfer failed if it does't contain at least 1 *.star file

monaw commented 4 years ago

Tech note, adding informative user messages for upload requires some refactoring to keep track of files, directories and each status as well as reason why each failed. Additionally, previously the logic is to get the status from Globus, then update our database. But now since our status may be different than Globus' status, we need to get Globus status, update our database and then get the info from our database (not Globus). Otherwise, the Transfer Status page may not display the correct status...

monaw commented 4 years ago

@mcianfrocco - ok, I've updated our Globus handling and added displaying of user messages that's hopefully helpful. Here are some screenshot examples:

Screen Shot 2020-02-20 at 10 37 15 AM

Screen Shot 2020-02-20 at 9 10 17 PM

Please try different Globus transfers on our staging server and let me know if you'd like to show different info and/or different message...

monaw commented 4 years ago

update has been pushed to production

monaw commented 4 years ago

Update...per meeting today, slight change to the files saved user message to hopefully make it clear that we are focusing on saving *.star files:

Screen Shot 2020-03-04 at 2 56 40 PM