If for some reason one of the render operations fails, it'd be great to have TabCompare automatically retry the pair of render operations again, as sometimes requests just don't go through the first time particularly well. Another reason to retry would be if differences are found--sometimes we see differences that are attributable strictly to timing differences or caching. In these situations I think it'd also be good to auto-retry, as this is typically a small subset of all the vizzes compared, and shouldn't take long in the grand scheme of things.
My thinking is that we should record one line of output per attempt so that we get the entire history of retries in the same CSV. We'll need the Tableau workbook that analyzes the results to use LODs to determine how likely an actual issue is depending on the results.
Not sure if it's best to retry these immediately, or push them to the end to retry them again after the other vizzes have been scanned....especially in light of the "resume" ( #3 )feature we'd also like to see implemented.
If for some reason one of the render operations fails, it'd be great to have TabCompare automatically retry the pair of render operations again, as sometimes requests just don't go through the first time particularly well. Another reason to retry would be if differences are found--sometimes we see differences that are attributable strictly to timing differences or caching. In these situations I think it'd also be good to auto-retry, as this is typically a small subset of all the vizzes compared, and shouldn't take long in the grand scheme of things.
My thinking is that we should record one line of output per attempt so that we get the entire history of retries in the same CSV. We'll need the Tableau workbook that analyzes the results to use LODs to determine how likely an actual issue is depending on the results.
Not sure if it's best to retry these immediately, or push them to the end to retry them again after the other vizzes have been scanned....especially in light of the "resume" ( #3 )feature we'd also like to see implemented.