Unable to deserialize JsonProvider types

[UnoSD]

Hi, thank you for your library.

I successfully serialized the type generated by the JsonProvider type provider, but when I try to deserialize I get an exception:

Could not create an instance of type FSharp.Data.Runtime.BaseTypes.IJsonDocument. Type is an interface or abstract class and cannot be instantiated. Path 'folders[0].JsonValue', line 1, position 25.

use db = new LiteDatabase("wunderlist.db", FSharpBsonMapper())

type F = JsonProvider<"FoldersJsonSample.json",  RootName = "Folder">
let folders = F.Load(....)
{ folders = folders } |> db.GetCollection<BackUp>().Insert |> ignore

let get = db.GetCollection<BackUp>().FindAll() // <- exception

would you be able to suggest a workaround or fix it?

Thank you.

[humhei]

{ folders = folders } Can you show me the record type definition of BackUp

[Zaid-Ajaj]

Hi @UnoSD,

This won't work for many reasons, the first reason is storing values that have interface as a type, this cannot be deserialized because (like any deserialization libraries) the deserializer doesn't know which concrete implementation it should use

The second reason this won't work, is because your document type doesn't have an id or Id property which is required, from the README:

The library requires that records have a primary key called Id or id. This field is then mapped to _id when converted to a bson document for indexing.

Solution:

Don't use the provided types directly but instead map them first to an intermediate proper document type:

type FolderStructure = (* ... *)

type BackUpDoc = {
   Id : int
   Folders : FolderStucture
}

let folders = F.Load(....)
let backup = { BackUpDoc.Empty with folders = makeFolderStructure  folders  }
db.GetCollection<BackUp>().Insert(backup)  |> ignore

[UnoSD]

@humhei it's just { folders : F.Folder list }

@Zaid-Ajaj thanks for the reply,

my document has the Id, I rushed to write the issue yesterday and forgot to add it.

My bad I didn't investigate at all before writing the issue, but couldn't the serializer figure out the concrete type through reflection and write it to the database for deserialization? maybe through an option (WriteOriginalType=true).

As a workaround, I can get a Folder back parsing some json string and write it as json to the db with a custom mapper I guess.

Tried with the following custom mapper with no luck:

let createMapper() = 
    let mapper = FSharpBsonMapper()

    let serialize (object : IJsonDocument) =
        BsonValue(object.JsonValue.AsString())

    let deserialize (parse : JsonValue -> 'a) (bson : BsonValue) = 
        parse(JsonValue.Parse(bson.AsString))

    let createDelegates parse =
        ( Func<'a, BsonValue>(serialize), Func<BsonValue, 'a>(deserialize parse) )

    let registerType parse =
        createDelegates parse |>
        mapper.RegisterType

    registerType File
    registerType Folder
    registerType Task
    registerType Subtask
    registerType Note
    registerType WList

    mapper

[Zaid-Ajaj]

@UnoSD I can probably make it a built-in feature but I would rather leave it out and keep the library simple, from experience I can tell you that if I add every use-case as a built-in feature, the library becomes a big mess of a kitchen sink.

There are two simple workarounds:

• Use an intermediate type like I explained in this comment
• Store the raw JSON like you suggested as follows:

Storing the raw JSON would mean that you have this document:

type BackUp = {
  Id : int
  Folders : string
}

and stringify the JSON folders from type F.Folder list to string to make the document

You might say: "But I don't want to do this every time I insert a document!" Yes, that is why you were registering the types, but a simpler solution is to make an extension method collection.InsertBackupDoc that does the conversion

I hope this solves your problem, if you need a more detailed explanation, let me know.

[UnoSD]

@Zaid-Ajaj thanks for the reply, I ended up writing a custom mapper that maps from raw JSON to BsonDocument (and therefore from FSharp.Data.JsonValue to BsonDocument and vice-versa). Happy to share it although I understand you may not want to add it to the library.

[Zaid-Ajaj]

@UnoSD I am glad you could figure it out, maybe you could share it for others who come across this issue :)

[UnoSD]

Here it is:

let rec toBsonValue value =
    let toBsonDictionary (record : (string * JsonValue)[]) =
        record |>
        Array.map (fun (name, value) -> 
                        KeyValuePair((if name = "id" then "_id" else name), toBsonValue value)) |>
        (fun x -> new Dictionary<string, BsonValue>(x))

    let toBsonArray (array : JsonValue[]) =
        array |>
        Array.map (fun value -> toBsonValue value)

    let (|Integer|Decimal|) input =
        match System.Int64.TryParse (input.ToString()) with
        | true, value -> Integer value
        | false, _ -> Decimal input

    match value with
        | JsonValue.String  value -> BsonValue    (value)
        | JsonValue.Number  value -> match value with
                                        | Integer i -> BsonValue(i)
                                        | Decimal d -> BsonValue(d)
        | JsonValue.Float   value -> BsonValue    (value)
        | JsonValue.Boolean value -> BsonValue    (value)
        | JsonValue.Record  value -> BsonDocument (toBsonDictionary value) :> BsonValue
        | JsonValue.Array   value -> BsonArray    (toBsonArray      value) :> BsonValue
        | JsonValue.Null          -> BsonValue    ()

from https://github.com/UnoSD/WunderlistBackup

and deserializing is easy... JsonValue.Parse(bson.ToString()) although you may have to add some converters for types like long that get serialized as objects.