unable to restore original data while using map[string]interface{}

[phith0n]

Hi, I have to use codec to receive JSON format data from clients, and save it into database in Msgpack format. Then restore the object to JSON format and send back to clients if they need it.

The data struct likes that:

type Sample struct {
    KV map[string]interface{} `json:"kv" codec:"kv"`
}

I use a simple code to demostrate what I have to do:

package model

import (
    "fmt"
    "github.com/stretchr/testify/assert"
    "github.com/ugorji/go/codec"
    "testing"
)

type Sample struct {
    KV map[string]interface{} `json:"kv" codec:"kv"`
}

func TestModel(t *testing.T) {
    var data = `{"kv": {"a": "b", "c": 1, "d": {"e": "f"}}}`
    var mh codec.MsgpackHandle
    var jh codec.JsonHandle

    // first step, decode data as JSON format
    var s Sample
    var dec = codec.NewDecoderString(data, &jh)
    assert.Nil(t, dec.Decode(&s))

    // second step, encode data as Msgpack format
    var data2 []byte
    var enc = codec.NewEncoderBytes(&data2, &mh)
    assert.Nil(t, enc.Encode(&s))

    // 3rd step, decode data as Msgpack format
    var s2 Sample
    dec = codec.NewDecoderBytes(data2, &mh)
    assert.Nil(t, dec.Decode(&s2))

    // 4th step, encode data as JSON format
    var data3 []byte
    enc = codec.NewEncoderBytes(&data3, &jh)
    assert.Nil(t, enc.Encode(&s2))

    // full flow is [json decode]->[msgpack encode]->[msgpack decode]->[json encode]
    fmt.Println(string(data3))
}

The full data flow is:

But the problem is that, my input JSON data is {"kv": {"a": "b", "c": 1, "d": {"e": "f"}}}, and the result JSON data is {"kv":{"d":{"e":"Zg=="},"a":"Yg==","c":1}}, which is not equal to the original content.

I'm not sure if it is a bug in codec. But is there any solution to satisfy my situation (let input data same as output data)? Thanks~

[ugorji]

The problem is that by default, MsgpackHandle honors the old msgpack spec. To honor the new spec that differentiates well between string and bytes, use

mh.WriteExt = true

I tested it, and it solves your issue.