Does Turbo support file uploads

[rjpcomputing]

Hi,

I don't know if this is the right place for this, but how do file uploads work in Turbo.lua? I was planning on using a form with a file select. How do I get at the details after the file is uploaded?

[kernelsauce]

Yes, it supports file uploads. If it is a unnamed upload, the payload of course is in self.request.body.

Otherwise you can use self:get_argument("name") where "name" would be the file name given in your upload form.

[kernelsauce]

Please keep in mind that the max body size allowed is usually pretty low. Set a higher one if you are uploading larger files than 18KB:

turbo.web.Application({
    {"^/upload$", UploadHandler}
}):listen(8888, "127.0.0.1", {max_body_size=1024*1024*100})

to allow for 100mb

[rjpcomputing]

What would happen if I needed to upload a 10GB file?

[kernelsauce]

You would need 10GB of memory since the simplified turbo.web.RequestHandler always puts this in memory before calling handler. Other than that it should be fine...

For 10GB files/uploads I would recommend using: http://turbolua.org/doc/httpserver.html#HTTPServer and that you set the kwarg: read_body to false and handle the reading yourself in a proper fashion for your application. Otherwise you are going to struggle ;).

[rjpcomputing]

Would I be able to mix this with turbo.web.Application?

[kernelsauce]

Application:listen(port, address, kwargs) takes the same kwargs. :)

[rjpcomputing]

@kernelsauce Thanks so much for the quick replies, but I am not familiar with web server technology. Could you make a quick example? I know that is work on your part, but maybe it could become an example?

To give you a bit more background on what I am trying to do. I am trying to make a file upload with a simple ticketing system. I am using AngularJS for the "view" portion and hosting it using Turbo. Then I am making a REST API using Turbo routes, So I was using turbo.web.Application to host all that. Plus I needed an "upload" POST REST method that takes mult-part/formdata. That is the only method that would need to turn off read_body. Does this make sense?

I really enjoy Turbo.lua and want to say thanks!

[kernelsauce]

Yes, I can create a example for you when I have the time...

[rjpcomputing]

@kernelsauce Thanks very much.

[EstebanSannin]

@kernelsauce I tried to upload a very small file (<18k) and everything works fine. Then I tried a jpeg image of 566816 bytes, but missing a few bytes (received 566790 byte).

Using self.request.body the content is correct.

Why?

The code:

function MyHandler:post()
    local file = self:get_argument("file", "ERROR")
    self:write("(POST Action) FILE ")
    local fd1 = io.open("output1","w+")
    io.output(fd1)
    io.write(self.request.body)
    io.close(fd1)

    local fd = io.open("output","w+")
    io.output(fd)
    io.write(file)
    io.close(fd)
end

[kernelsauce]

Maybe you need to use "wb" as option for io.open to make things work correct for binary data?

[EstebanSannin]

I have tried the "wb" for io.open with the same result ... :-(

If you can be helpful, here are the files used for testing. The differences between the two are really minimal ...

file_received

file_original

[kernelsauce]

I will review this soon.

[kernelsauce]

What version are you running on?

[EstebanSannin]

I use the version of turbo there on github master

[kernelsauce]

Ok, please try this stable version one: https://github.com/kernelsauce/turbo/archive/v.1.1.1.tar.gz

[EstebanSannin]

Same result...

[EstebanSannin]

I found the problem ... https://github.com/kernelsauce/turbo/blob/master/turbo/httputil.lua#L445

This escape.unescape breaks the data... if I change this line without escape.unescape, everything works perfectly, but I do not know if it is the ultimate solution :-)

[EstebanSannin]

In my opinion you do not need to escape ... If you want I can make a pull request ..

[kernelsauce]

No need, I already have one. I will sort this out so just keep developing with your modification in place. I need to run some tests as this code is used for a lot of things in forms, and I did not personally write it (Jeff Solinsky did I think)...

[kernelsauce]

Fix in master now for @EstebanSannin problem.

[EstebanSannin]

It was just a line after an afternoon of debugging because I did not know the code .. No problem, you did a nice job with Turbo, very great framework!

[rjpcomputing]

@kernelsauce Did you get a chance to make an example? I didn't expect this issue to be closed till that happened. Should I create a different issue? Thanks.

[kernelsauce]

Sorry, yeah. Havent done it yet.

[EstebanSannin]

Hi @kernelsauce, What is the best way to get the file name of the upload?

[nagadomi]

https://github.com/kernelsauce/turbo/issues/158#issuecomment-91530445 this commit has not been included in 1.1.3. 1.1.3 (from luarocks) has bug in attached file.

[kernelsauce]

Thank you for reporting. I will take a look at it.

[kernelsauce]

v1.1.4 fixes your problem @nagadomi. Sorry for delayed fix, but due to work I have not been able to finish sooner.

[kernelsauce]

@rjpcomputing it seems you need to use two servers running at different ports to solve your problem, as the read_body option is a Application specific option... To be honest I would suggest you use two different processes so that uploads are handled in another process than your other requests to make sure your website stays "snappy". I don't have the time to create a example for you at the moment and there are more important issues. As such I am closing the case.