How to serialize as nested json ?

[mastershaig]

Hi, Do you have any idea or already existing solution to serialize the result of with_tree_fields() using DRF serializers as below ?

{
    "id": 1051,
    "name": "Data",
    "children": [
        {
            "id": 121,
            "name": "DataObject1"
            "children":[]
        },
        {
            "id": 122,
            "name": "DataObject2"
            "children": []
        }
    ]
}

Thanks!

[matthiask]

Hi

I don't know exactly how the DRF serializers work, but here's some untested code to do this in memory, even without tree queries:

children = defaultdict(list)
for node in Node.objects.with_tree_fields(False).select_related("parent"):
    children[node.parent].append(node)

def serialize(parent):
    return {"id": parent.id, "name": parent.name, "children": [serialize(child) for child in children[child]]}

roots = [serialize(root) for root in children[None]]

[mastershaig]

thanks for your response! just out of curiosity, why don't you use with_tree_fields here somehow ? Because if it's not used it would be a normal sql query and I think it would be an expensive one, wouldn't it ? I installed this library to use with_tree_fields and make the queries efficient :)

[matthiask]

I'm fetching all objects and building up the structure in-memory; also, the code doesn't need the ancestry or the level of nodes etc so adding the CTE only causes overhead in this exact example.

What's expensive is fetching level by level (e.g. recursively calling self.children.all()) but we're not doing that here.

[mastershaig]

makes sense, thanks @matthiask! Do you think it's possible to get nested results by editing current SQL query ?

[matthiask]

Maybe PostgreSQL could do it with JSON aggregates etc. or even with PL/pgSQL. I'm not sure why you'd want this, though :)

[mastershaig]

Currently I use with_tree_fields(False) and get flat list of all nodes then parse them into nested dict by using your above function and I think parsing is a bit expensive. I think it would be more efficient to get nested result from sql so I wouldn't need to parse...

[matthiask]

It's possible that this is a performance bottleneck with very big trees. How many nodes does your tree have?

Did you measure the time/energy required to create this nested data structure?

I'd bet that the time needed for this is dwarfed by everything else which has to happen when rendering the site (except when the tree is really big!)

Maybe you could also look into caching the data structure. But first you have to measure.

[matthiask]

Closing for inactivity: Please reopen (or open a new issue!) if you have new findings. Thanks!