search

Shared-Reality-Lab / IMAGE-test-graphics

Scripts and graphics used for ongoing testing of IMAGE
1 stars 0 forks source link

Copy of graphic should not be stored in output #2

Closed jeffbl closed 2 years ago

jeffbl commented 2 years ago

Preprocessors (correctly) send back the full graphic along with all of the preprocessor output. This means if the test script stores the entire preprocessor output, we'll end up with another copy of the (potentially large) base64 graphic itself on each run. I can't think of a practical use for this, so unless we come up with a compelling reason, this should be removed.

Easiest is probably just a hard-coded hack that excludes this field.

Better way long term would be to have some sort of list of keys that are to be excluded, so that we can easily add others if we decide there are additional ones that should not be stored. In this case, it might be similar to a list of keys that we don't want compared when we diff output files to see if there are legit. changes to which we should pay attention. Similar mechanism might be a good approach, but I haven't thought it through.

rianadutta commented 2 years ago

For now, I've hard coded out the base 64 graphic. Not sure how it would work with the keys and the diff output as the jsons are created when testset.py is run and the excluded keys would be decided when testdiff is run.