Support a total size limit for the upload directory

[orhun]

It would be cool to have a configuration option for the total size of the upload folder. This would help with public instances in terms of avoiding a storage attack where a single user uploads a bunch of big/small files.

This new option might look like (or something similar):

[server]
max_uploads = "1G"

[Narayanbhat166]

Hey @orhun, can I pick up and work on this issue?

[orhun]

Hey @Narayanbhat166, sure thing! Are you still interested?

[Narayanbhat166]

Hi @orhun, what I have understood by having a glance at the code is, you are talking about the ./uploads folder which holds all the posted data in files. In this issue, we are talking about setting limit on this folder size and if we receive a request with a file, that when stored in the ./uploads folder exceeds the max_uploads size, we need to take some action. The actions can be

• Reject the request.

• Implement an eviction policy to Delete the least recently accessed file / near to expiry file or some other policy ( we can discuss about this ).

  Can you please help me clear few of these doubts? Thanks. Also can you assign this issue to me?

[orhun]

@Narayanbhat166 yup, that's correct. As a first step we can just reject the request with a message.

[tessus]

It would be great to reject the file BEFORE it has been completely uploaded.

[orhun]

Yeah, I guess we had a problem with the current upload mechanism like that, right? I vaguely remember something like it.

[Narayanbhat166]

It would be great to reject the file BEFORE it has been completely uploaded

Are you suggesting that we add this logic in the middleware? I had also thought of the same, but I wanted to handle this at once place ( for remote file uploads via url too ), so store_file function is where I have added this. Let me know if you want me to move this to a middleware

[tessus]

Are you suggesting that we add this logic in the middleware?

That's the thing. IMO the middleware does not work, or at least not in my environment. I have set max_content_length = "50MB", but whenever I upload a file larger than 50MB, the entire file is uploaded and only then I get the error message that the limit is exceeded. (I just tried again and I uploaded a 250MB file, before I got upload limit exceeded (status code: 413))

[orhun]

That's weird, I just tried the same scenario and it worked fine for me. How do you tell that the entire file is uploaded? Is there are a delay between the upload and the message? (because for me it is instant)

[tessus]

[orhun]

Can you try curl?

[tessus]

[orhun]

Strange... we should look into it. Can you shoot me an issue?