search

trident-project / trident

A Synthetic Spectral Generation Suite
Other
21 stars 25 forks source link

Setting new gold standards to resolve failing CI tests #171

Closed chummels closed 3 years ago

chummels commented 3 years ago

Attempting to fix the circleci tests by bumping the cache keynames and clearing the cache.

OK, so that didn't work. But digging in further, I think I see what the problem is.

It appears that the failing tests occur when we try to generate the answer tests with the gold standards before even testing the tip of trident/yt. The gold standard for yt and trident are from over a year ago with version in between yt4 and yt4.0.1. I can run them locally no problem and generate the answer tests. However, a change was made to yt_astro_analysis setting a hard dependency for yt4.0.1, which has changes incompatible with the trident gold standard. So when circleCI tries to generate the answer tests, it tries to install the gold standards, but yt_astro_analysis bumps the version of yt up to a newer version that causes everything to fail. So the solution is just to bump the gold standard versions of the code to current. I'm just arbitrarily choosing yt4.0.0's release (yt4.0.1 was not in the yt commit tree?!) and the current trident tip as the gold standards since I've personally confirmed locally that these are consistent with the previous gold standard. This should finally resolve these stupid circleCI testing failures.

chummels commented 3 years ago

In retrospect, an alternative solution would have been to make a gold standard for yt_astro_analysis to use to help generate the gold standard answers, but that seems a little too deep for my liking. Hopefully now that yt4 is out, these sorts of breakages will be few and far between.