square / spoon

Distributing instrumentation tests to all your Androids.
https://square.github.io/spoon/
Apache License 2.0
2.7k stars 476 forks source link

Old test screenshots associated with new tests #280

Open jraska opened 9 years ago

jraska commented 9 years ago

Hi,

When no screenshots are taken during test suite, no old screenshots are deleted. Naturally because no Spoon method is called, but unfortunately these old screens are associated after that with current tests by runner.

This happens to me when I take screenshots only on failures and old screenshots appear in output after successful run, because there are collected by runner at the end of tests run.

I understand that you cannot do anything with that when no Spoon method is called, but I would appreciate some possibility to call clearing this dir, because now all methods handling that are private or runner itself could delete this old files during start of tests. Now I solved it with taking one screenshot at the very beginning of test suite and immediately deleting it, but I don't consider this as ideal solution.

Thanks for reply.

manishgo commented 9 years ago

I am also facing the similar issue. The biggest problem is that Spoon class has private methods that perform deletion of files which can be done only by invoking public method Spoon.takeScreenshot(). I suggest exposing utility methods in Spoon class for deleting of files.

JakeWharton commented 9 years ago

Wouldn't the biggest problem is that the old screenshots are associated with new tests, not that the method is private? Since, you know, that's the bug.

On Wed, Aug 12, 2015 at 4:37 AM Manish Goyal notifications@github.com wrote:

I am also facing the similar issue. The biggest problem is that Spoon class has private methods that perform deletion of files which can be done only by invoking public method Spoon.takeScreenshot(). I suggest exposing utility methods in Spoon class for deleting of files.

— Reply to this email directly or view it on GitHub https://github.com/square/spoon/issues/280#issuecomment-130217370.

manishgo commented 9 years ago

Let me explain where I am coming from when I say having private methods is a problem. I see some very useful utility methods in the Spoon class like obtainScreenshotDirectory(), findTestClassTraceElement(), deletePath() which if exposed as public methods could be used to delete the snapshots directory without calling Spoon.takeScreenshot() method. Also obtainScreenshotDirectory() can be used for the purpose of saving screenshots created using other methods like UiAutomation.takeScreenshot. However if spoon comes up with a solution of clearing the screenshots directory as part of test runner then I think it solves the problem in cleanest way.

manishgo commented 9 years ago

Any updates on this issue?

edenman commented 8 years ago

Related/dup of #284

phq commented 8 years ago

Any updates on this? Shouldn't Spoon create screenshot folders for each test execution? Screenshots from previous test runs are being pulled down by Spoon, and then either generating an error saying couldn't match up the image w/t testClass/testMethod, or attaching the image to the test report for the new test run, regardless current test run pass or fail --- not an expected behaviour.

Seems to be dup of #284.