imroc / req

Simple Go HTTP client with Black Magic
https://req.cool
MIT License
4.3k stars 350 forks source link

Bug set correct Content-Length #62

Closed n0v3xx closed 2 years ago

n0v3xx commented 4 years ago

Hi,

i want to upload an file to oboom com via API. And req dont set the correct content length header. I get the following message: "missing or invalid Content-Length header"

I debug it and content length it always 0.

Is it possible to set correct header?

imroc commented 4 years ago

How did you upload it? Please show your code snippet.

n0v3xx commented 4 years ago

Here is the code. I think my code is correct i do upload on other services which dont need a correct Content Length. But oboom need it.

func (o Oboom) Upload(filePath string) (UploadResponse, error) { _, err := o.Login() if err != nil { return nil, err }

file, _ := os.Open(filePath)
fileName := filepath.Base(file.Name())

fupload := req.FileUpload{
    FileName:  fileName,
    FieldName: "file",
    File:      file,     // File
}

params := req.QueryParam{
    "token": o.Token,
    "parent": "1",
}

//rsp, err := o.Client.Request.Post(usrsp.Result, o.Client.Header, params, fupload, req.UploadProgress(progress))
rsp, err := o.Client.Request.Post(URLUploadServer, o.Client.Header, params, fupload)
if err != nil {
    log.Println(err)
    return nil, err
}

ursp := &UploadResponse{}
err = rsp.ToJSON(ursp)
if err != nil {
    log.Println(err)
    return nil, err
}

return ursp, nil

}

It looks like your req lib dont handle it correctly.

imroc commented 4 years ago

File upload uses Content-Type multipart/form-data with random boundary to seperate different files, and no Content-Length will been set. If you don't need multipart/form-data, you could just post the os.File:

file, _ := os.Open(filePath)
params := req.QueryParam{
    "token": o.Token,
    "parent": "1",
}

rsp, err := o.Client.Request.Post(URLUploadServer, o.Client.Header, params, file)
if err != nil {
    log.Println(err)
    return nil, err
}

ursp := &UploadResponse{}
err = rsp.ToJSON(ursp)
if err != nil {
    log.Println(err)
    return nil, err
}

return ursp, nil
n0v3xx commented 4 years ago

Ok thanks i will try it. Is there a way to post the file with a minimum amount of memory like the "multipart/form-data" with io.Pipe? Or is io:pipe also used when i post the file directly? Thanks for you help.

imroc commented 4 years ago

yes, both supported.

n0v3xx commented 4 years ago

Crazy both ways dont work with that filehoster. It looks like he need a "multipart/form-data" request with full content length and posted as a normal file. I am not sure if this possbile at the moment.

imroc commented 4 years ago

Sorry, not possible at this moment. We cannot know the total Content-Length at the begining, because the content was not just the raw file, it was base64 encoded, and filling some of the other fields.

n0v3xx commented 4 years ago

I think its possible to calculate the content length before you send the request. DumpRequest from thte httputil can print out the correct content length from a request (r *http.Request). Can you evaluate this option? https://golang.org/pkg/net/http/httputil/#DumpRequest

imroc commented 4 years ago

DumpRequest(req, true) will just simply read all the request header and body in memory, it's dangerous if we read the big file

n0v3xx commented 4 years ago

Thats realy anoying in Go. With PHP (CURLFile) i can upload huge files with a minimal amount of RAM to this hoster and with Go it is not possible. Looks like that Go is not the best option for fileuploads at all.