Think about screenshot frequency in user testing add-on

[nbcl-maronoff]

We need to figure out how often screenshots will be captured. (I've got the code to create a screenshot programmatically, it's just a question of how often we invoke it). Things to consider:

• More than one thing may respond to a touch (in the case of the annotation button, both the button itself and the image underneath it actually get notified that the user tapped them; this isn't uncommon). Even if we throttle the screenshots so it doesn't take more than one every second or so, that's still a lot to do in terms of system resources, so I think that's probably not going to be doable.
• We could take screenshots on a timer (while the app is running, take a screenshot every, say, 15 seconds, and keep the last thousand screenshots... something like that). This also might affect performance a little while it's taking the screenshot, but maybe not enough to be noticeable during testing.
• Some other possibility I haven't thought of yet.

Anyway, to ponder.

[nbcl-kkgreene]

I'm more concerned with the touch log (timestamped X,Y coordinates at least—haven't thought too much beyond that yet, maybe touch duration/dwell?) than the screenshots for sure. Most touches won't change the system in a way that's really screen-shot-worthy-interesting. Reconstructing screenshots based on the usual Vanilla touch sequences won't be hard. For the rarer and more interesting Mango touch sequences, may be a little harder, but if there were a way to add system-state info interleaved with the touch data, you'd still easily have all the detail needed to reconstruct. In sum: we can make screenshots at any point, but we can only collect quantitative data from real users at the necessary fine level of granularity if we do it during testing.