Error on upload Files - no data sent to the handler.

[toni-moreno]

Hi @Unknwon . I need to do file uploading (import formated data to a config database) and I have been testing with your example here (https://go-macaron.com/docs/middlewares/binding)

My code is exactly this, and the export API is working fine.

package webui

import (
    "bytes"
    "encoding/json"
    "github.com/go-macaron/binding"
    "github.com/toni-moreno/snmpcollector/pkg/data/impexp"
    "gopkg.in/macaron.v1"
    "io/ioutil"
    "mime/multipart"
    "os"
)

type UploadForm struct {
    Title      string                `form:"Title"`
    TextUpload *multipart.FileHeader `form:"TextUpload"`
}

func NewImportExport(m *macaron.Macaron) error {

    bind := binding.Bind

    m.Group("/api/cfg/export", func() {
        m.Get("/:objtype/:id", reqSignedIn, ExportMeasurementGroup)
        m.Post("/:objtype/:id", reqSignedIn, bind(impexp.ExportInfo{}), ExportMeasurementGroupFile)

    })
    m.Group("/api/cfg/import", func() {
        m.Post("/", reqSignedIn, binding.MultipartForm(UploadForm{}), UploadHandler)
    })
    return nil
}

/****************/
/*IMPORT*/
/*****************/

func UploadHandler( /*ctx *Context,*/ uf UploadForm) {
    if (UploadForm{}) == uf {
        log.Error("Error no data in expected struct")
        return
    }
    log.Debugf("Uploaded file : %s", uf.Title)
    log.Debugf("Uploaded File : %+v", uf)
    file, err := uf.TextUpload.Open()
    if err != nil {
        log.Warningf("Error on Open Uploaded File: %s", err)
        //  ctx.JSON(404, err.Error())
    }
    buf := new(bytes.Buffer)
    buf.ReadFrom(file)
    s := buf.String()
    log.Debug("DATA: %s", s)
}

/****************/
/*EXPORT*/
/****************/

func Export(ctx *Context) {
....
 //more code working ok here
}

func ExportFile(ctx *Context, info impexp.ExportInfo) {
....
//more code working ok  here
}

This is the ouput when a upload have been done

ERRO[2017-02-18 08:03:40] Error no data in expected struct 

IT seems like no data is sent to the UploadForm struct.

I've got HTTP real data with wireshark and this i exactly what we are sending to the macaron handler.

POST /api/cfg/import HTTP/1.1
Host: localhost:8090
Connection: keep-alive
Content-Length: 7275
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
Origin: chrome-extension://apcedakaoficjlofohhcmkkljehnmebp
Content-Type: multipart/form-data
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: es,ca;q=0.8,en;q=0.6,en-US;q=0.4
Cookie: MacaronSession=127714bd60ec66e0; snmpcollector-sess-lan=9da867813c641552

------WebKitFormBoundaryfin1vPdBlN5ZQrnv
Content-Disposition: form-data; name="fileUpload1"; filename="blob"
Content-Type: text/plain

{"Info":{"FileName":"","Description":"","Author":"","Tags":""},"AgentVersion":"","ExportVersion":"1.0","CreationDate":"2017-02-18T06:37:28.159948966+01:00","Objects":[{"ObjectTypeID":"influxcfg","ObjectID":"default","ObjectPtr":{"ID":"default","Host":"127.0.0.1","Port":8086,"DB":"snmp","User":"snmpuser","Password":"snmppass","Retention":"autogen","Timeout":5,"UserAgent":"snmpcollector_agent_00","Description":""}},{"ObjectTypeID":"oidconditioncfg","ObjectID":"test_multiple","ObjectPtr":{"ID":"test_multiple","IsMultiple":true,"OIDCond":"filter_port_if_status_up \u0026\u0026 filter_port_if_name_match_eth0","CondType":"","CondValue":"","Description":"adf"}},{"ObjectTypeID":"measfiltercfg","ObjectID":"filter_multiple_test","ObjectPtr":{"ID":"filter_multiple_test","IDMeasurementCfg":"linux_ports","FType":"OIDCondition","FilterName":"test_multiple","EnableAlias":false,"Description":"test"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifOutErrors","ObjectPtr":{"ID":"ifOutErrors","FieldName":"ifOutErrors","Description":"","BaseOID":".1.3.6.1.2.1.2.2.1.20","DataSrcType":"COUNTER32","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifInErrors","ObjectPtr":{"ID":"ifInErrors","FieldName":"ifInErrors","Description":"","BaseOID":".1.3.6.1.2.1.2.2.1.14","DataSrcType":"COUNTER32","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"evaluated_prefix_for_ifname","ObjectPtr":{"ID":"evaluated_prefix_for_ifname","FieldName":"prefixed_ifname","Description":"Add Prefix for ifname","BaseOID":"","DataSrcType":"STRINGEVAL","GetRate":false,"Scale":0,"Shift":0,"IsTag":true,"ExtraData":"'myprefix_'+IfName"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifName","ObjectPtr":{"ID":"ifName","FieldName":"IfName","Description":"Tags","BaseOID":".1.3.6.1.2.1.31.1.1.1.1","DataSrcType":"OCTETSTRING","GetRate":false,"Scale":0,"Shift":0,"IsTag":true,"ExtraData":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"IfH_out_percent_utilization","ObjectPtr":{"ID":"IfH_out_percent_utilization","FieldName":"out_percent_utilization","Description":"% utlization out","BaseOID":"","DataSrcType":"STRINGEVAL","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"(out_octets * 8) / ( ifhspeed * 10000)"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifH_in_percent_utilization","ObjectPtr":{"ID":"ifH_in_percent_utilization","FieldName":"in_percent_utilization","Description":"% utlization in","BaseOID":"","DataSrcType":"STRINGEVAL","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"(in_octets * 8 ) /  (ifhspeed * 10000)"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifHighSpeed","ObjectPtr":{"ID":"ifHighSpeed","FieldName":"ifhspeed","Description":"Bytes Out - 64-bit Counters","BaseOID":".1.3.6.1.2.1.31.1.1.1.15","DataSrcType":"Gauge32","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"user + system + idle"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifHCOutOctets","ObjectPtr":{"ID":"ifHCOutOctets","FieldName":"out_octets","Description":"Bytes Out - 64-bit Counters","BaseOID":".1.3.6.1.2.1.31.1.1.1.10","DataSrcType":"COUNTER64","GetRate":true,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"user + system + idle"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"ifHCInOctets","ObjectPtr":{"ID":"ifHCInOctets","FieldName":"in_octets","Description":"Bytes In - 64-bit Counters","BaseOID":".1.3.6.1.2.1.31.1.1.1.6","DataSrcType":"COUNTER64","GetRate":true,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"user + system + idle"}},{"ObjectTypeID":"measurementcfg","ObjectID":"linux_ports","ObjectPtr":{"ID":"linux_ports","Name":"linux_ports","GetMode":"indexed","IndexOID":".1.3.6.1.2.1.31.1.1.1.1","TagOID":"","IndexTag":"portName","IndexTagFormat":"","IndexAsValue":false,"Fields":[{"ID":"ifHCInOctets","Report":1},{"ID":"ifHCOutOctets","Report":1},{"ID":"ifHighSpeed","Report":1},{"ID":"ifH_in_percent_utilization","Report":1},{"ID":"IfH_out_percent_utilization","Report":1},{"ID":"ifName","Report":1},{"ID":"evaluated_prefix_for_ifname","Report":1},{"ID":"ifInErrors","Report":2},{"ID":"ifOutErrors","Report":2}],"Description":""}},{"ObjectTypeID":"oidconditioncfg","ObjectID":"filter_port_if_status_up","ObjectPtr":{"ID":"filter_port_if_status_up","IsMultiple":false,"OIDCond":".1.3.6.1.2.1.2.2.1.8","CondType":"neq","CondValue":"1","Description":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"test_cond_oid","ObjectPtr":{"ID":"test_cond_oid","FieldName":"oidtest","Description":"afaf","BaseOID":"","DataSrcType":"CONDITIONEVAL","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"filter_port_if_status_up"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"Linux_user_CPU_total","ObjectPtr":{"ID":"Linux_user_CPU_total","FieldName":"cpu_total","Description":"calculo de CPU total","BaseOID":"","DataSrcType":"STRINGEVAL","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":"user + system + idle"}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"Linux_idle_CPU_percent","ObjectPtr":{"ID":"Linux_idle_CPU_percent","FieldName":"idle","Description":"percentage of Idle CPU time","BaseOID":".1.3.6.1.4.1.2021.11.11.0","DataSrcType":"INTEGER","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"Linux_system_CPU_percent","ObjectPtr":{"ID":"Linux_system_CPU_percent","FieldName":"system","Description":"percentage of system CPU time","BaseOID":".1.3.6.1.4.1.2021.11.10.0","DataSrcType":"INTEGER","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":""}},{"ObjectTypeID":"snmpmetriccfg","ObjectID":"Linux_user_CPU_percent","ObjectPtr":{"ID":"Linux_user_CPU_percent","FieldName":"user","Description":"percentage of user CPU time","BaseOID":".1.3.6.1.4.1.2021.11.9.0","DataSrcType":"INTEGER","GetRate":false,"Scale":0,"Shift":0,"IsTag":false,"ExtraData":""}},{"ObjectTypeID":"measurementcfg","ObjectID":"linux_cpu","ObjectPtr":{"ID":"linux_cpu","Name":"linux.cpu","GetMode":"value","IndexOID":"","TagOID":"","IndexTag":"","IndexTagFormat":"","IndexAsValue":false,"Fields":[{"ID":"Linux_user_CPU_percent","Report":1},{"ID":"Linux_system_CPU_percent","Report":1},{"ID":"Linux_idle_CPU_percent","Report":1},{"ID":"Linux_user_CPU_total","Report":1},{"ID":"test_cond_oid","Report":1}],"Description":""}},{"ObjectTypeID":"measgroupscfg","ObjectID":"Linux","ObjectPtr":{"ID":"Linux","Measurements":["linux_cpu","linux_ports"],"Description":""}},{"ObjectTypeID":"snmpdevicecfg","ObjectID":"hostsnmpv3a","ObjectPtr":{"ID":"hostsnmpv3a","Host":"hostsnmpv3a","Port":161,"Retries":5,"Timeout":20,"Repeat":0,"Active":true,"SnmpVersion":"3","Community":"public","V3SecLevel":"NoAuthNoPriv","V3AuthUser":"v3usernoauth","V3AuthPass":"v3passauth","V3AuthProt":"MD5","V3PrivPass":"","V3PrivProt":"","DisableBulk":false,"Freq":30,"UpdateFltFreq":1,"OutDB":"default","LogLevel":"debug","LogFile":"","SnmpDebug":false,"DeviceTagName":"router","DeviceTagValue":"id","ExtraTags":["tagA=4","tagB=5","tagC=6"],"Description":"","MeasurementGroups":["Linux"],"MeasFilters":["filter_multiple_test"]}}]}

There is no any warning error , and I've done exact as you did in the above example. I will be happy if you can help me.

Thank you very much

[unknwon]

Can you post corresponding HTML as well?

[toni-moreno]

I'm doing HTTP POST with a REST client, The frontend code is not ready yet.

[unknwon]

You expect name of field which contains file as TextUpload, but I don't see anywhere you set it.

[toni-moreno]

Sorry I don't understand which is the goal of the UploadForm struct. Title and TextUpload have to be in the HTTP body?

Can you show me a equivalent valid HTML which can upload files with these 2 fields? please?

[unknwon]

I don't know what you're doing. Just make a valid HTML.

Example:

• https://github.com/gogits/gogs/blob/master/templates/user/settings/avatar.tmpl#L37
• https://github.com/gogits/gogs/blob/master/modules/auth/user_form.go#L110

[toni-moreno]

Thank a lot @Unknwon

The problem was in the REST client.

After created a simple HTML form, we can recover data from file. Thank You very much and sorry for the inconvenience

<html>
<head>
</head>
<body>
<form enctype="multipart/form-data" action="http://localhost:8090/api/cfg/import/" method="POST">
Choose a file to upload: <input name="export_file" type="file" /><br />
<input type="submit" value="Upload File" />
</form>
</body>
</html>

[toni-moreno]

The struct used is now

type UploadForm struct {
    ExportFile *multipart.FileHeader
}