mongo insert NumberLong error

[Beatrice7]

|| { "insert": "server_info", "documents": [ { "_id" : "all_3", "extAddr" : "http://...", "idx" : 0, "mark_ts" : NumberLong("1502351930348"), "priority" : 1, "r_status" : 1, "stype" : "all" }, { "_id" : "recv_1", "extAddr" : "http://...", "idx" : 1, "mark_ts" : NumberLong("1502351910053"), "priority" : 1, "r_status" : 1, "stype" : "recv" } ] }

happen error but, "mark_ts" : NumberLong("1502351910053"), to "mark_ts" : 1502351910053, is ok

this is why?

[mattes]

cc @dimag-jfrog

[Beatrice7]

"gopkg.in/mgo.v2"
"gopkg.in/mgo.v2/bson"

[dimag-jfrog]

This seems more as a generic Mongo question and not a question specific to the Migrate project. What is the error that you're getting? And what is the exact Go code that you have used?

[Beatrice7]

error report： I2017-08-11 08:29:45.198469 10 main.go:130] failed to create database: %!(EXTRA *json.SyntaxError=invalid character 'N' looking for beginning of value)

golang code： // CreateDatabase execute a mongo script file, and execute the script in it. // The mongodb migrate in github.com/mattes/migrate need write migrate code and // build into dataloader binary. It means every service who need mongo // dataloader should write and build a dataloader binary. // So we can't use it in service dataload. func (md *mongoDriver) CreateDatabase(name, owner string) error { files, err := ioutil.ReadDir(name) if err != nil { return err }

reg, err := regexp.Compile("\\s")
if err != nil {
    return err
}

for _, file := range files {
    if file.IsDir() {
        continue
    }

    dbName := strings.Split(file.Name(), "__")[0]
    bts, err := ioutil.ReadFile(name + "/" + file.Name())
    fileContent := string(bts)
    cmds := strings.Split(fileContent, "||")

    info, err := mgo.ParseURL(md.GetURL("", "", ""))
    if err != nil {
        return err
    }
    sess, err := mgo.DialWithInfo(info)
    if err != nil {
        return err
    }
    defer sess.Close()
    db := sess.DB(dbName)

    for _, cmd := range cmds {
        cmdbs := reg.ReplaceAll([]byte(cmd), []byte(""))
        var jmap map[string]interface{}
        if err = json.Unmarshal(cmdbs, &jmap); err != nil {
            return err
        }

        bsond := make([]bson.DocElem, 1)
        bsond[0] = bson.DocElem{"", ""}
        for key, value := range jmap {
            if ok := commands[key]; ok {
                bsond[0] = bson.DocElem{key, value}
                glog.Infof("mongo command:%s", key)
                continue
            }
            bsond = append(bsond, bson.DocElem{key, value})
        }

        var resp interface{}
        if err = db.Run(bsond, &resp); err != nil {
            glog.Infof("run mongo command error, db: %s, run: %s, err: %+v", dbName, cmd, err)
            continue
        }

        glog.Infof("mongo command success, db: %s, run %s, response: %s", dbName, cmd, resp)
    }
}

return nil

}

[dimag-jfrog]

Probably since the error that you're getting is: json.SyntaxError, the failure is in the line that deals with json "json.Unmarshal" - probably the json is not formatted correctly. Add a print message there to see if this exact line is returning the error. Anyway, this is not a question specific to Migrate project - it is just a random Golang code.

[dimag-jfrog]

And if you meant that this method will run as a part of a migration - it's signature is wrong - it should have *mgo.Session parameter and this session should be used in the method, you can see usages examples here: https://github.com/mattes/migrate/blob/v1/driver/mongodb/example/sample_mongdb_migrator.go

[Beatrice7]

yes，it just json.Unmarshal error，but so i should to do？ { "insert": "server_info", "documents": [ { "_id" : "all_0", "extAddr" : "http://...", "idx" : 0, "mark_ts" : "NumberLong('1502351930348')", "priority" : 1, "r_status" : 1, "stype" : "all" }, { "_id" : "recv_1", "extAddr" : "http://...", "idx" : 1, "mark_ts" : "NumberLong('1502351910053')", "priority" : 1, "r_status" : 1, "stype" : "recv" } ] }

"mark_ts" : "NumberLong('1502351930348')" with a string is ok but is a string， not a NumberLong

i want is： { "_id" : "all_0", "extAddr" : "http://...", "idx" : 0, "mark_ts" : NumberLong("1502775322882"), "priority" : 1, "r_status" : 1, "stype" : "all" } { "_id" : "recv_1", "extAddr" : "http://...", "idx" : 1, "mark_ts" : NumberLong("1502775331612"), "priority" : 1, "r_status" : 1, "stype" : "recv" }

not： { "_id" : "all_0", "mark_ts" : "NumberLong('1502351930348')", "priority" : 1, "r_status" : 1, "stype" : "all", "extAddr" : "http://...", "idx" : 0 } { "_id" : "recv_1", "stype" : "recv", "extAddr" : "...", "idx" : 1, "mark_ts" : "NumberLong('1502351910053')", "priority" : 1, "r_status" : 1 }

[Beatrice7]

mongo has NumberInt and NumberLong， has “mattes/migrate” been tested? and new Date()

[dimag-jfrog]

NumberInt and NumberLong and Date are not part of the JSON specification, so json.Unmarshal will fail on those types. They are types specific to Mongo. Again this is not a question for mattes/migrate, this is a generic json/mongo question.

[dimag-jfrog]

If you want to use json marshal/unmarshal you should stick to json types (string, int, bool, etc), and then parse them yourself and convert them to appropriate mongo types (NumberInt, NumberLong, Date, etc) with your business logic specific to your application.

[Beatrice7]

i used with mattes/migrate, i just wanna putting into mongo with NumberLong . cant i realize it with mattes/migrate? I don't care about his implement. pls ignore marshal/unmarshal, or others.

[dimag-jfrog]

This question is not connected in any way to mattes/migrate. You can remove the mattes/migrate package import from your code and it should run - you are not using mattes/migrate at all in the code that you sent. Before using migration, you should see if your code works at all.

[Beatrice7]

i mean you ignore the code，the codes not wirtten by me

[Beatrice7]

var resp interface{} if err = db.Run(bsond, &resp); err != nil { glog.Infof("run mongo command error, db: %s, run: %s, err: %+v", dbName, cmd, err) continue }

is mattes/migrate

[dimag-jfrog]

This code does not use mattes/migrate at all. First remove mattes/migrate package at all and make sure that your code runs successfully, only after that you see that it runs succesfully try to convert it to a migration.

[Beatrice7]

i am so sorry, it really not mattes/migrate problem. i had fixed my issue. thanks very much for mattes/migrate team.

[mattes]

thanks @dimag-jfrog

[dimag-jfrog]

@Beatrice7 , @mattes You're welcome