rpgmaker / NetJSON

Faster than Any Binary? Benchmark: http://theburningmonk.com/2014/08/json-serializers-benchmarks-updated-2/
MIT License
227 stars 29 forks source link

Type/class object in dictionary is not serialized. #63

Closed StefanBelo closed 9 years ago

StefanBelo commented 9 years ago

Thanks for your work in fixing problems with Dictionary serialization. You already fixed problems with enum and string values, but unfortunately real REST API are built quite complicated way requiring complicated parameter's structure.

In the following JSON:

{
  "jsonrpc":"2.0",
  "method":"SportsAPING/v1.0/listMarketBook",
  "params":{
     "marketIds":["1.119136312"],
     "priceProjection":{
         "priceData":["EX_BEST_OFFERS"],
         "exBestOffersOverrides":  {
             "bestPricesDepth":3,
             "rollupModel":"NONE",
             "rollupLimit":0,
             "rollupLiabilityThreshold":0.0,
             "rollupLiabilityFactor":0},
         "virtualise":true,
        "rolloverStakes":false},
     "orderProjection":"ALL",
     "matchProjection":"ROLLED_UP_BY_AVG_PRICE"},
   "id":1}

The params parameter is Dictionary<string, object> so as values you can find primitive types like string, int, double, enums.., for instance: orderProjection is

"orderProjection":"ALL"

enum value, but priceProjection is type/object value with nested properties of arrays of enums, another type/object value (exBestOffersOverrides), bool property (virtualise) and so on..

Your serialization routine is not able to serialize such type/object value of "params" dictionary.

I do not know how you implemented dictionary serialization, but you are able to serialize type/object already, so when serializing dictionary you check type of value and call its serialization routine, right?

rpgmaker commented 9 years ago

At the moment, the types are check at runtime to determine what to use to serialize/deserialize. If you want to deserialize to a complex dictionary, you can use NetJSON.Deserialize(json). That will return a complex Dictionary<string, object> that recursively represent your json object. So every class entries will become a dictionary and so on. And same goes with having a list of class, they will automatically become a List[Dictionary[string, object]]..

As for supporting different type for a Dictionary[string, object] for serialization, I can somewhat do that but the issue is that i currently need to know all the types used in the dictionary to properly generate to the correct logic for serializing them. Without that knowledge, It will quit difficult even at runtime to call the right method for serialize it since my code is based on emit all the information about the types ahead of time rather than using reflection to navigate the object as it is been serialized.

And as for deserialization, that would never work since i cannot infer the class type from the json object itself. But I can do the same thing which i current did when using NetJSON.Deserialize[object] and simply make every complex class a dictionary at the point of deserialization.

Let me know if that make sense.

Thanks,

StefanBelo commented 9 years ago

Actually I have no problems to deserialize from JSON using your library, well as you described it, because when deserializing the required types/classes are fully known.

type JsonRpcResponse<'T> =
    {
        result : 'T
        error : ErrorData
    }

When serializing to api requests I use:

type Parameters = Dictionary<string, obj>

type JsonRpcRequest =
    {
        jsonrpc : string
        ``method`` : string
        ``params`` : Parameters
        id : int
    }

This way I avoided to write exact request parameter class/type for any API method, so I could write for instance:

member this.GetMarketCatalogues(filter, maxResults, ?marketProjection, ?sort, ?locale) =
            let parameters = createParameters()
                             |> withParameter "filter" filter
                             |> withParameter "maxResults" maxResults
                             |> withOptionalParameter "marketProjection" marketProjection
                             |> withOptionalParameter "sort" sort
                             |> withOptionalParameter "locale" locale

            jsonClient.Execute<ResizeArray<MarketCatalogue>>(serviceUrl, "SportsAPING/v1.0/listMarketCatalogue", parameters)

Well, I will see if I rethink my approach, as making your serialization library to work I would have to write a lot of plumbing types/classes for each request object parameter.

What is your approach when working with REST APIs?

StefanBelo commented 9 years ago

In Dictionary<string, obj> you know element value type/class: objValue.GetType()

rpgmaker commented 9 years ago

For rest api, i typically define the specific types that I need. I usually only use dict[string, object] when dealing with an object that contains just primitive types and basic List[primitive]. The solution you want is still possible, it just requires a slightly different solution due to the way serialization is been done to known types.

As for Dictionary, I only get the element.Value.GetType() only when dealing with Dictonary[string, object] and then call the correct method based on the arrays of predefined methods.

For your solution to work, I can create a method where you can pass known type to the methods as an array, then I can easily generate the correct serialization and deserialization logic that will be applied to each object at runtime using just the objValue.GetType.

This approach is kinda of similar to what is done in WCF using KnownType attribute. So maybe a good solution would be to have an attribute that you tag on to the class that wraps your Dictionary[string, object] and that will be used to figure everything out at runtime.

As for other serialization library out there such as JSON.NET, Jil, and ServiceStack. I am guessing they are able to handle this situation because they are mostly reflection based and do not do any code generation, right?

Thanks,

StefanBelo commented 9 years ago

I used JSON.NET for de/serialization of JSON, and tested Jil maybe at start of my project, but it did not work for me. At that time I had been looking for F# libraries found some projects but none of them worked.

Now I switched to your library for deserialization and wanted to fully replaced JSON.NET by your one.

Yes, I defined all classes for JSON responses for REST API calls, and some of these classes are actually reused for requests as well. People designing REST API service used a lot of parameters just some of them mandatory so using Dictionary<string, obj> class to represent those requests looked as a good idea for me.

I do not want to take your time, maybe creating those plumbing classes for JSON request data will give me advantage to reuse some of requests data.

What is nice feature on F# is TypeProviders

http://fsharp.github.io/FSharp.Data/library/JsonProvider.html

If all works with type providers then you actually do not have to write your classes for REST Api models.

rpgmaker commented 9 years ago

It is not a waste of my time. Everything is all about learning i suppose. I think for your particular scenario, if you are ok with specifying the known type as an attribute or a collection of Types then i can easily support serializing anything. And I might just make it possible to inject your own methods, so it can be registered per types making it possible to handle types various different ways as you need be.

Thanks,

StefanBelo commented 9 years ago

I am hobby programmer and yes it is mainly about learning and problem solving. On the other hand my implementation of request object by using Dictionary for parameters was wrong. Just imaging that if app service must call api each 5 ms, then request parameters are build as list of Add instructions to the dictionary object.

I have done it now implementing:

[<CLIMutable; NoEquality; NoComparison>]
type JsonRpcRequest<'T> =
    {
        jsonrpc : string
        ``method`` : string
        ``params`` : 'T
        id : int
    }

    static member ToJson(apiMethod, parameters : 'T) =
        let request =
            {
                jsonrpc = "2.0"
                ``method`` = apiMethod
                ``params`` = parameters
                id = 1
            }

        NetJSON.Serialize<JsonRpcRequest<'T>>(request)

So now I can say that your library can be fully used from F# written app.

There is only one problem, your library does not allow primitive value de/serialization.

string json = NetJSON.NetJSON.Serialize<string>("text");

Thanks again for your great work.