Attempting serialize JSON with circular references

[chrisnurse]

I'm trying to move to TypedJSON and off of 'flatted' but when serializing my object graph, which has circular references I get the error: TypeError: Converting circular structure to JSON.

Is TypedJSON supposed to handle circular references?

A quick look at the code indicates a dependency on JSON.stringify. Perhaps we can convert to flatted then?

Cheers Chris

[JohnWeisz]

hi @chrisnurse

Sorry if this was misleading to you, what TypedJSON technically does is a 2-way conversion between (1) a raw object tree, and (2) a tree of instantiated classes. The raw object tree is converted to/from JSON using the native JSON stringify and parse methods.

Some time ago I made an attempt at supporting recursive object trees by embedding "reference objects" in the resulting raw object tree, but never succeeded at bringing it to a production-ready state, unfortunately.

[Neos3452]

Unfortunately, it may not be as easy as switching JSON to flatted because TypedJSON will infinitely descend into the circular references. One option would be to cut the serialization at the point where the circuit could occur and instead of stringify use toPlainJson which will return a plain json object that you can then pass to flatted. Example:

interface Circular {
    smt?: number;
    cir?: Circular;
}

@jsonObject
class FooClass {

    @jsonMember
    prop: string;

    // use the obj itself when serializing and deserializing
    // this is to circumvent typedjson lack of circular ref support
    @jsonMember({deserializer: json => json, serializer: json => json})
    cir: Circular;
}

/// test
const foo = new FooClass();
foo.prop = 'thing';
foo.cir = {
    smt: 20,
};
foo.cir.cir = {
    smt: 30,
    cir: foo.cir,
};
console.log(TypedJSON.toPlainJson(foo, FooClass));

const json: any = {prop: 'other', cir: {smt:55, cir: {smt: 66}}};
json.cir.cir.cir = json.cir;
console.log(TypedJSON.parse(json, FooClass));

/// using with flatted
const jsonWithCircularRef = TypedJSON.toPlainJson(obj, FooClass);
const jsonStr = flatted.stringfy(jsonWithCircularRef);

const jsonWithCircularRef = flatted.parse(jsonStr);
const obj = TypedJSON.parse(jsonWithCircularRef, FooClass);

Technically, it is possible to make TypedJSON support serialising to json object with circular references without custom serializer/deserializer, we would just need to use lib like https://github.com/pvorb/clone, but I'm not sure if the use case is so common to increase the lib size by 30%.

To support it properly we would need to use similar approach I think — cache input -> output and check each time if we already handled the object.

[chrisnurse]

First of all guys, thank you so much for the responses and suggestions. I did insert some calls to flatted and that worked really well actually (from my perspective). One thing you might try is putting wrapper methods around anything that touches JSON.stringify/parse and then a JSON serializer (flatted or JSON.*...) could be injected and called to serialize / deserialize when required.

My object model is what I would call extremely complex from a JSON perspective. I do a lot of very rapid prototyping and things that simply have to run standalone on your own machine in front of a customer. So I'm hacking object models together in what you'd think of as an in memory database, and running things in Docker and things like TypedJson make me highly productive. So great job.

Anyway I won't write War and Peace here. My use case is an in memory object graph with circular e.g. relationships customer <- -> order <- -> product, but the schema of these objects is defined in individual YAML files. So I have to load the YAML and then I discover these relationships between objects and so consequently I then end up injecting attributes on to object...

customer.orders (orders gets added dynamically when I discover the relationship) order.customer (customer gets added dynamically)

So this is what killed me in the end :( as the dynamically added fields were created on deserialisation but strangely the arrays (customer.orders) were not populated.

So I have had to go a different way for now.

Thanks, Chris

[DHager]

Some time ago I made an attempt at supporting recursive object trees by embedding "reference objects" in the resulting raw object tree, but never succeeded at bringing it to a production-ready state, unfortunately.

@JohnWeisz Is this still a limitation/worth-tackling? I've been reinventing a bit of the wheel trying to de/serialize a graph of objects recently.

[manticorp]

Might be worth noting that jackson-js supports this through their @JsonIdentityInfo decorator

https://itnext.io/jackson-js-powerful-javascript-decorators-to-serialize-deserialize-objects-into-json-and-vice-df952454cf#a667

It would be awesome to add this to TypedJSON!