Upload Binary file

[OcatrapseSmith]

Hello,

Thanks for this piece of code. It's perfect for me.

Is there an example of how to upload a binary file from the file system? (for example to jpg, mp4, etc.)

Any clues how to do it?

Thanks in advance.

[vincasmiliunas]

Hi, great to hear that the library is useful for you.

Binary and text files are handled the same way. Just don't forget to set the right mimetype for the uploaded data. Jpgs are image/jpeg, mp4s are video/mp4, miscellaneous data use application/octet-stream.

I've modified the example code to test it out:

local JSON = require 'json'
local GDRIVE = require 'lua-basic-google-drive'
local GDUTILS = require 'lua-basic-oauth2.utils'

local gdrive = GDRIVE.new{creds_file = 'creds.json', tokens_file = 'tokens.json'}

local utils = GDUTILS.new()
local jpeg, code = utils:httpRequest('https://static1.squarespace.com/static/54e8ba93e4b07c3f655b452e/t/56c2a04520c64707756f4267/1493764650017/', nil, nil, nil, nil)
print(string.format('Got image: #jpeg:%d code:%d', #jpeg, code))

print('-- initializing')
gdrive:init()

print('-- file upload')
local file = {title = 'cute kitten.jpg', mimeType = "image/jpeg"}
local ret = gdrive:upload({}, file, jpeg)
print(JSON.encode(ret))

print('-- file retrieval')
local content, meta = gdrive:get({}, ret.id)

if jpeg == content then
    print('Same image content.')
else
    print('Image changed!')
end

print('-- saving image')
utils:saveFile('cute kitten.jpg', content)

I also patched the lua-basic-oauth2 library, because the setopt_postfieldsize method was no longer present in lua-curl and was merged into setopt_postfields.

Cheers

[OcatrapseSmith]

Thanks so much.

I'll try it.

I also had to patch the oauth2 yesterday because I encountered the same problem with setopt_postfieldsize.

Regards.

[OcatrapseSmith]

It works!

I used the utils.loadFile instead of utils:httpRequest because the file is in my local file system.

But, now I need to find the way to upload files by parts, because when I try to upload, for example, a 15MB file or so, I get a out of memory from Lua. (It's running in a Raspberry pi).

I hope that gdrive rest api has a resumable upload or something like that.

Thanks a lot.

[vincasmiliunas]

REST is just HTTP. The data transfer in small fragments you need is called Transfer-Encoding: chunked.

I took this as an opportunity to explore the Lua's Coroutine API and added streaming support to the libraries.

The API still works in the same manner, but now you can pass a stream worker instead of a byte string. For download: call :get and pass a function or a coroutine that writes data to a file. Likewise for upload: call :upload and pass a function or a coroutine that reads a file (or another data source).

Follow the example code below:

local JSON = require 'json'
local GDRIVE = require 'lua-basic-google-drive'
local GDUTILS = require 'lua-basic-oauth2.utils'

local utils = GDUTILS.new()

do
    print('-- saving puppy clip')
    local f, err = io.open('puppy.mp4', 'wb')
    if not f then error(err) end
    local size = 0
    local _, code = utils:httpRequest('https://giant.gfycat.com/WhirlwindBlissfulAxolotl.mp4', nil, nil, nil, {
        write = function(data)
            print(string.format('Write puppy.mp4 chunk size:%d', #data))
            f:write(data)
            size = size + #data
        end
    })
    f:close()
    print(string.format('Got code:%d, size:%d', code, size))
end

do
    print('-- initializing gdrive')
    local gdrive = GDRIVE.new{creds_file = 'creds.json', tokens_file = 'tokens.json'}
    gdrive:init()

    local f, err = io.open('puppy.mp4', 'rb')
    if not f then error(err) end

    print('-- file upload')
    local file = {title = 'puppy.mp4', mimeType = "video/mp4"}
    local ret = gdrive:upload({}, file, function()
        local ret = f:read(16*1024)
        if ret then
            print(string.format('Read chunk size:%d', #ret))
        else
            print('Read all of the chunks')
        end
        return ret
    end)
    print(JSON.encode(ret))
    f:close()

    f, err = io.open('puppy2.mp4', 'wb')
    if not f then error(err) end

    print('-- file retrieval')
    local content, meta = gdrive:get({}, ret.id, function(data)
        print(string.format('Write puppy2.mp4 chunk size:%d', #data))
        f:write(data)
    end)
    print('File metadata: ' .. JSON.encode(meta))
    f:close()
end

[OcatrapseSmith]

Hello.

I think you have come up with a pretty elegant solution by taking advantage of Lua, and enough for the particular use I was looking for. Thank you very much. I'll try it.

I was investigating the API and found that it does have Resumable Upload, through the establishment of a load session. I think it is good both to split the load, and to relaunch them in case of transmission errors without needing to reload from the beginning.

But what I said, I think, with the modification you've made, is more than enough.

Thx.