I may be barking up the wrong tree here, but an idea occurred to me for some possible test optimization.
Right now the standard against which any output test is checked is stored as a pdf file. This is good because pdfs are multi-page and vector based so only one file is needed to store the standard. However, this also means that the standard must be converted to a png file on the fly each time the tests are run in order to make the comparison. It occurs to me that if we were to save these generated png files then we could save on that conversion time when re-running the test with the same resolution. The question would be how to indicate the resolution of these png files (perhaps a put them in a folder with the resolution indicated in the name) and how many resolutions of these files should be kept (just the most recent or some number of resolutions?).
Of course, I haven't done any checking to see just how much time goes into this conversion process, but given the ever increasing number of our tests, I expect that even a small gain on a single test might add up to a significant savings over the whole set.
I may be barking up the wrong tree here, but an idea occurred to me for some possible test optimization.
Right now the standard against which any output test is checked is stored as a pdf file. This is good because pdfs are multi-page and vector based so only one file is needed to store the standard. However, this also means that the standard must be converted to a png file on the fly each time the tests are run in order to make the comparison. It occurs to me that if we were to save these generated png files then we could save on that conversion time when re-running the test with the same resolution. The question would be how to indicate the resolution of these png files (perhaps a put them in a folder with the resolution indicated in the name) and how many resolutions of these files should be kept (just the most recent or some number of resolutions?).
Of course, I haven't done any checking to see just how much time goes into this conversion process, but given the ever increasing number of our tests, I expect that even a small gain on a single test might add up to a significant savings over the whole set.