search

OpenPecha / Toolkit

🛠 Tools to create, edit and export texts and annotations
https://toolkit.openpecha.org
Apache License 2.0
7 stars 4 forks source link

don't produce an remove unneccesary null values in yaml serialization #211

Closed eroux closed 1 year ago

eroux commented 1 year ago

We now have a lot of null values in layers, like in this example. The yaml files are already quite a bottleneck in terms of performance, this certainly will make the problem worse. Before we do more transformations into opf we really need to find a way to fix this issue.

This is linked to https://github.com/OpenPecha/Toolkit/issues/186 (I think) and it's become more apparent in a recent change (I'm not sure which one).

I think a good solution would be to use yaml representers (or something similar) so that a pedantic class can be serialized to yaml and a yaml file can be read as a pydantic class directly (with no round trip with dictionaries)

10zinten commented 1 year ago

Yeah, It's definitely related to #186.

Once I am done with current ongoing project. I will ask NT to set aside time to work on pydantic refactor and hopefully we will be able to fix this issue all together.