search

cianfrocco-lab / COSMIC-CryoEM-Gateway

Web platform for analyzing cryo-EM data on Comet cluster at San Diego Supercomputer Center
http://cosmic-cryoem.org
GNU General Public License v3.0
9 stars 1 forks source link

web upload limit unchecked #225

Closed monaw closed 3 years ago

monaw commented 4 years ago

SGCI security team found that our gateway's web upload says 200 mb max but allows bigger. Mona verified this:

Screen Shot 2020-01-15 at 12 10 50 PM

Screen Shot 2020-01-15 at 12 14 09 PM

mcianfrocco commented 4 years ago

@kenneth59715 - is there way to actually limit this on the upload page? Or how do we limit this?

kenneth59715 commented 4 years ago

I don't see a total web upload space limit check in CreateData.java, but I think that's where it should go. For that user, check to see what globus upload limit is, then enforce the same limit for web uploads?

mcianfrocco commented 4 years ago

Is there a rule of thumb for this? Like practically we should limit it to 500 MB to save frustrations of people trying to upload larger files?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 1:35 PM kenneth59715 notifications@github.com wrote:

I don't see a total web upload space limit check in CreateData.java, but I think that's where it should go. For that user, check to see what globus upload limit is, then enforce the same limit for web uploads?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680169481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALFRFLNID62X4MQLOOLIGLSCPY5TANCNFSM4KHJI7PQ .

kenneth59715 commented 4 years ago

I think the per-upload practical limit will vary a lot, depending on the connection from the user to SDSC. I would let them try whatever they want. If we want a total per-user limit, we could implement that. Is there a typical dataset size for cryo-em?

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, August 25, 2020 10:56 AM, Michael A. Cianfrocco notifications@github.com wrote:

Is there a rule of thumb for this? Like practically we should limit it to 500 MB to save frustrations of people trying to upload larger files?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 1:35 PM kenneth59715 notifications@github.com wrote:

I don't see a total web upload space limit check in CreateData.java, but I think that's where it should go. For that user, check to see what globus upload limit is, then enforce the same limit for web uploads?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680169481, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALFRFLNID62X4MQLOOLIGLSCPY5TANCNFSM4KHJI7PQ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

mcianfrocco commented 4 years ago

In this case, it's not a dataset, it would just be individual 3D reconstructions that they would use for analyzing datasets.

Perhaps, then, we just remove the limit warning from the website?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 3:25 PM kenneth59715 notifications@github.com wrote:

I think the per-upload practical limit will vary a lot, depending on the connection from the user to SDSC. I would let them try whatever they want. If we want a total per-user limit, we could implement that. Is there a typical dataset size for cryo-em?

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, August 25, 2020 10:56 AM, Michael A. Cianfrocco < notifications@github.com> wrote:

Is there a rule of thumb for this? Like practically we should limit it to 500 MB to save frustrations of people trying to upload larger files?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 1:35 PM kenneth59715 notifications@github.com wrote:

I don't see a total web upload space limit check in CreateData.java, but I think that's where it should go. For that user, check to see what globus upload limit is, then enforce the same limit for web uploads?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680169481 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AALFRFLNID62X4MQLOOLIGLSCPY5TANCNFSM4KHJI7PQ

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680223973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALFRFJVIO5USGFMAMAGD7LSCQFZFANCNFSM4KHJI7PQ .

kenneth59715 commented 4 years ago

Sounds reasonable to me. Do you want to assign removal of warning to me?

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, August 25, 2020 1:20 PM, Michael A. Cianfrocco notifications@github.com wrote:

In this case, it's not a dataset, it would just be individual 3D reconstructions that they would use for analyzing datasets.

Perhaps, then, we just remove the limit warning from the website?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 3:25 PM kenneth59715 notifications@github.com wrote:

I think the per-upload practical limit will vary a lot, depending on the connection from the user to SDSC. I would let them try whatever they want. If we want a total per-user limit, we could implement that. Is there a typical dataset size for cryo-em?

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Tuesday, August 25, 2020 10:56 AM, Michael A. Cianfrocco < notifications@github.com> wrote:

Is there a rule of thumb for this? Like practically we should limit it to 500 MB to save frustrations of people trying to upload larger files?

Michael A. Cianfrocco, Ph.D. Assistant Professor, Department of Biological Chemistry Research Assistant Professor, Life Sciences Institute University of Michigan Pronouns: he, him, his

On Tue, Aug 25, 2020 at 1:35 PM kenneth59715 notifications@github.com wrote:

I don't see a total web upload space limit check in CreateData.java, but I think that's where it should go. For that user, check to see what globus upload limit is, then enforce the same limit for web uploads?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680169481 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AALFRFLNID62X4MQLOOLIGLSCPY5TANCNFSM4KHJI7PQ

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cianfrocco-lab/COSMIC-CryoEM-Gateway/issues/225#issuecomment-680223973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALFRFJVIO5USGFMAMAGD7LSCQFZFANCNFSM4KHJI7PQ .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

mcianfrocco commented 4 years ago

will do

mcianfrocco commented 4 years ago

Hi @kenneth59715 can you remove this error message?

kenneth59715 commented 4 years ago

Okay, I'll dig into it.

Kenneth

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Friday, September 4, 2020 10:46 AM, Michael A. Cianfrocco notifications@github.com wrote:

Reopened #225.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

kenneth59715 commented 4 years ago

Warning on upload limit has been removed in staging. Not rolled into production, yet.