This should be really easy, I'm just busy with other stuff this second and don't want to forget.
Reason being, currently if we want to build and serialize a large number of graphs, they'd all have to be stored in memory and then serialized one by one, and you lose progress if something goes wrong (such as, for example, some graphs couldn't be built and then the script borks when it tries to serialize a missing object). if we make this change, then we can broadcast over output files and serialize a graph as it's being created, and we won't run into the problem I'm describing.
Is this informed by very specific experience today? Maayyyybe...
This should be really easy, I'm just busy with other stuff this second and don't want to forget.
Reason being, currently if we want to build and serialize a large number of graphs, they'd all have to be stored in memory and then serialized one by one, and you lose progress if something goes wrong (such as, for example, some graphs couldn't be built and then the script borks when it tries to serialize a
missingobject). if we make this change, then we can broadcast over output files and serialize a graph as it's being created, and we won't run into the problem I'm describing.Is this informed by very specific experience today? Maayyyybe...