ohler55 / ojg

Optimized JSON for Go
MIT License
857 stars 49 forks source link

looking for example of something like json.Marshal #25

Closed marcelloh closed 4 years ago

marcelloh commented 4 years ago

I'm looking for something like your alternative for: b, err := json.Marshal(data)

But I can't find it (perhaps its there, but I just didn't see it)

Can you point me to something like this?

ohler55 commented 4 years ago

Try

b, err := oj.JSON(data)

or if you wanted indented output

b, err := oj.JSON(data, 2)
marcelloh commented 4 years ago
b, err := oj.JSON(data)

is wrong because it doesn't pass any error that can happen :-( And the b suggest a []byte , but it's a string :-( So it is not really compatible, but "sort of".

Any other suggestions?

ohler55 commented 4 years ago

You are correct. I was rushing out the door this morning. My apologies. The string vs []bytes is simply fixed with a []byte(s). Oj attempts to alway generate JSON output so things that can not be converted to JSON such as a chan are encoded as a string representation of the value.

Is this more in line with what you are looking for?

b := []byte(oj.JSON(data))
marcelloh commented 4 years ago

Thanks for answering btw :-)

I came up with this myself, but I'm kind of: what happens if there's an error? the normal marshaller would tell me this. and the oj.JSON just swallows it :-(

ohler55 commented 4 years ago

The only errors that could occur are encountering a type that can not be converted. Oj uses several approaches to assure everything can be converted even if it is just to a string. On the flip side, the gen package enforces JSON generation using compile time type checking.

Do you see a need for some kind of more strict conversion to JSON that would fail when ever a non-convertible type is encountered? Basically fail instead of using a string representation of the value.

marcelloh commented 4 years ago

I never had a situation where it failed. But when it does, I will know ;-)

ohler55 commented 4 years ago

Under what condition would you expect a failure to occur?

ohler55 commented 4 years ago

I created a branch called 'marshal' that has an oj.Marshal() function. If that is what you are looking for let me know and I'll release.

marcelloh commented 4 years ago

This is my code:

    type testModel struct {
        Field1 string
        Field2 string
    }
    data := make(map[string]interface{}, 2)
    data["success"] = true
    model := testModel{}
    model.Field1 = "F1"
    model.Field2 = "F2"
    data["record"] = model

    b, err := oj.Marshal(data)

it gives me an error: testModel can not be encoded as a JSON element

I think it has something todo with Opt.strict?

ohler55 commented 4 years ago

That's why I asked the question about what would be expected. I took the approach that unless the type implemented the Simplifier interface or was primitive it would fail unless an optional create key was provided. Based on your response I take it you think reflection should be used even without a create key. Is that true?

marcelloh commented 4 years ago

When I do the same via tje encoding/json it returns everything.

{"success":true,"record":{"Field1":"F1","Field2":"F2"}}

So I kind of expected to have thew same. If this is done via reflection in your library, that's fine by me. Perhaps this could be optional, so other users don't have to use this. But I was wondering what speed is left over after doing so. I'm willing to test this, but have to eat soon :-) So don't hurry, there are more days in this year ;-)

ohler55 commented 4 years ago

I'll make a change to not require the create type option for reflection.

I would not expect a significant change is performance.

ohler55 commented 4 years ago

Ok, new version pushed.

marcelloh commented 4 years ago

The fields are there :-) but... lowercase in the json result :-( (Which means it's still not compatible 100%)

I did 5 benchmark tests and here's the average of those 5.

Benchmark_PresentJSON-12.       991640     1148 ns/op        696 B/op         11 allocs/op
Benchmark_PresentJSONoj-12.   12648212      978 ns/op        989 B/op          8 allocs/op

So it is faster for sure :-)

ohler55 commented 4 years ago

The current behaviour for Oj is to convert the names to camelcase starting with a lower case letter. That tests to be the most common style convention for JSON from what I have seen. It you would like some other option such as no doing the conversion that would be an easy addition. I would prefer not to use the field decorations or tags a that forced lookup for each does take a toll and Oj provides alternatives to that with the Simplifier interface.

marcelloh commented 4 years ago

I would keep the same output as the standard JSON would give you. So if the struct is like this:

type Todo struct {
    ID      string `json:"id"`
    Label   string `json:"label" validate:"required,min=3"`
    Comment string `json:"comment" validate:"min=2"`
    DueDate string `json:"duedate"`
}

it will use the name 'id' for the ID field, but if it is like this:

Type Todo struct {
    ID      string
    Label   string `json:"label" validate:"required,min=3"`
    Comment string `json:"comment" validate:"min=2"`
    DueDate string `json:"duedate"`
}

it will use ID for the ID field

ohler55 commented 4 years ago

Where Oj excels is on parsing and the use of JSON Path. Writing in either the golang json package or with Oj will not be that much difference. I can add a full compatible mode but it really isn't intended to be a drop in replace for the json package. Give me a day or two.

ohler55 commented 4 years ago

The marshal branch now includes an option to use the json annotation tags.

marcelloh commented 4 years ago

Ok, I somehow expected a library that was fully compatible and amazingly fast. In order to compare this again the standard way, is if it is compatible ;-)

ohler55 commented 4 years ago

Misspelled 'now' as 'not' above. Fixed. The marshal branch now includes an option to use the json annotation tags.

marcelloh commented 4 years ago

It is fuzzy to me how to get this to work. And perhaps unnecessary: Can I still get a result where the fieldname of a struct is part of the json result? Like: {"success":true,"record":{"Field1":"F1","Field2":"F2"}}

In the write test, I've tried to add 2 tests (but don't know if the options are meant for my purpose):

{value: &Dummy{Val: 3}, expect: `{"Val":3}`, options: &oj.Options{Sort: true, NoReflect: false}},
{value: &Dummy{Val: 3}, expect: `{"Val":3}`, options: &oj.Options{Sort: true, UseTags: true}},
ohler55 commented 4 years ago

So you would like to see exact field names without using the field tags, is that right?

marcelloh commented 4 years ago

I think that is true, unless there's a json annotation tag for a field, in that case it should be that tag's field. This is how the standard encoding works.

ohler55 commented 4 years ago

I'll add another field to Options.

ohler55 commented 4 years ago

Okay, KeyExact option added to oj.Options.

marcelloh commented 4 years ago

My findings.

type Dummy struct {
    Val int
}
// part of test set
{value: &Dummy{Val: 3}, expect: `{"Val":3}`, options: &oj.Options{KeyExact: true, UseTags: true}},

works as expected

type Dummy struct {
    Val int `json:"value"`
}

// part of test set
{value: &Dummy{Val: 3}, expect: `{"value":3}`, options: &oj.Options{KeyExact: true, UseTags: true}},

However if I compare it to the standard library and run my benchmarks, I see that you library is now slower than the standard solution. I think this has to do with the .JSON returning a string instead of a []byte, which I have to correct afterwards to have it fully compatible.

ohler55 commented 4 years ago

oj.Marshal() returns ([]byte, error) just like json.Marshal(). I would expect oj to be almost the same as the json package when using the field tags. For writing oj has a number of options no supported by the golang json package. Those options allow for faster writing when used. If you want the exact behaviour of the json package there really isn't a reason to use oj for writing. If you take advantage of the extra encoding and decoding features such as decoding interface fields then oj offers features not found in the json package.

marcelloh commented 4 years ago

Perhaps I misunderstood. Can you show an example where I can have options doing the oj.Marshal()?

ohler55 commented 4 years ago

Sure. I added it to the example_test.go file as well.

func ExampleMarshal() {
    type Valley struct {
        Val int `json:"value"`
    }

    b, err := oj.Marshal(&Valley{Val: 3})
    fmt.Printf("%v %s\n", err, b)
    // Output: <nil> {"value":3}
}
marcelloh commented 4 years ago
type vally struct {
        Val int
    }
    data := make(map[string]interface{}, 2)
    data["success"] = true
    model := vally{}
    model.Val = 1
    data["record"] = model

    b, err := oj.Marshal(data)

output as string: {"success":true,"record":{"val":1}} Notice that val is still in small

ohler55 commented 4 years ago

I suspect you need to pull the latest. While you can provide options to the Marshal function, the defaults are now what you would expect from json.Marshal().

marcelloh commented 4 years ago
Benchmark_OutputJSONnormal-12         958072          1202 ns/op         702 B/op         11 allocs/op
Benchmark_OutputJSONoj-12            1000000          1022 ns/op         933 B/op         10 allocs/op

You succeeded :-)

ohler55 commented 4 years ago

I'll make a release tonight then.

ohler55 commented 4 years ago

released v1.3.0

ohler55 commented 4 years ago

okay to close?