Report page with all screenshots

[ligi]

First of all I have to say thanks for composer - I was in deep trouble with spoon - especially now with spoon 2 - was a really sad situation - now thanks to @jaredsburrows I discovered composer - and love it - makes me really happy!

Just one really minor thing - I would love a page with all screenshots - this can really help to make a quick check of layout problems in a change.

[christopherperry]

Do you mean other than the page for the individual test?

Also, hi @jaredsburrows!

[ligi]

yes as clicking through all tests takes more time than just visiting one site to get a glimpse if all screenshots look ok

[christopherperry]

That would be extremely slow to load the page if you have a lot of screenshots, that's why this library doesn't do it.

Are you using the screenshots to verify visuals? I only ever check the screenshots of failed tests, for a clue as to what the issue might be in that particular test. We don't even bother with the report at all if all the tests pass, it saves us from pulling all the screenshots to our build environment.

[artem-zinnatullin]

Correct, that was major pain point with Spoon on large test sets as it was displaying all screenshots on one page and browser dropped scroll position while they were loading.

(it's even worse with Firebase Test Lab)

However, I think @ligi raised a valid use case for viewing all screenshots on single page.

I'm not opposed to have a separate html that we can leave a link to which will show all screenshots.

Or, you can flatten them manually in a single folder after test run

(I did not check this script, just out of my head)

cd composer-out
mkdir screenshots
find . -type f -name '*.png' -exec cp {} composer-out \;

@yunikkk WDYT?

[artem-zinnatullin]

I would also recommend to look into screenshot-based testing (you can run them with Composer as well, they're just tests) if you often need to verify that design is not broken.

There is a project from Facebook, but I don't know how hard would it be to detach from their runner infra https://github.com/facebook/screenshot-tests-for-android

Idea is quite simple though: you store expected screenshots and version them with the project (you can upload them to remote storage, just version them via commits/etc) then you run tests that make same screenshots and compare with expected. There are caveats on comparing screenshots, look into FB project for that.

[trevjonez]

So pardon my lurking here. I wrote a plugin/library set that performs a similar task to the facebook screenshot library but uses concepts similar to the way composer works and also has a much more intimate integration with AGP so setup and running is simpler.

https://github.com/trevjonez/Kontrast

and on the topic of listing all screen shots on one page. my report uses https://github.com/tuupola/jquery_lazyload so that you only load the images when scrolling them into view. Also noteworthy is that since the report is generated staticly and the image size is known I am able to presize so that scrolling doesn't become super janky.

[artem-zinnatullin]

Noice!

[ligi]

Hey - thanks for all your input. I also think it should be a separate page to make the main path faster and jank-free. But as an option I think it can still be nice - I thought of not copying the images but make a page that just references them. Will have a look into screenshot testing with Kontrast - but think the page could still make sense as an option. Thanks @trevjonez for the hint and your composer gradle plugin - happy user here. Still think the screenshot page might be a good option - the screenshot testing requires more work and will be more fragile in my imagination (sometimes I want things to change but not the build to break - and it looks I cannot simply reuse my existing tests)

[artem-zinnatullin]

Yep, we don't need to copy images, HTML report can reference them.

The copy solution I've posted is for your local use, so you could do it locally and view them in single folder in your file manager.

If @yunikkk is onboard with separate HTML page with all screenshots, I think we're open to merge such PR :)

[yunikkk]

Hey, Im totally for such feature & PR, though IMO its hardly usable with lots of tests (hence tons of screenshots).

[ligi]

though IMO its hardly usable with lots of tests (hence tons of screenshots).

not every test needs a screenshot - so I think it is manageable - or perhaps a tag-filter is also an option here for some use-cases