opentrace-community / opentrace-calibration

OpenTrace Calibration. Device calibration data and Trial Methodologies for testing implementations of the BlueTrace protocol.
https://bluetrace.io
GNU General Public License v3.0
118 stars 62 forks source link

Battery saver whitelisting during tests? #1

Open tomekziel opened 4 years ago

tomekziel commented 4 years ago

Hi! I'm very interested in raw data of your tests. You mentioned dontkillmyapp.com and whitelisting, so the question is - did you whitelist your test app on lab devices? Or was it working in vanilla conditions, as every other userspace app just installed from store? Also: did you record Doze Mode bin your app was assigned to at given moment?

EDIT: also - did you use any kind of wakelock? What was the screen off timout setting?

hanyangtan commented 4 years ago

Hi there, As much as possible we would like to emulate our users behavior when testing the application - so our focus will be on vanilla conditions, no wakelock and screen off as much as possible (at least for Android).

We plot out the pair-wise connections in order to visualise the times where the manufacturer optimisations or doze mode kicks in to see which are the more problematic devices (See below image). The more dots in a row, the better and more consistent our data is. If our device fails to see other devices because of doze etc, then it compromises our contact tracing efforts.

tt_dots_visualisation_example

We have then come to realise from this data that tweaking the manufacturer's system configuration plays a part in keeping 1) the service alive, and 2) consistent communication to other devices. We have an FAQ page for recommendations to users to address this.

Generally, we notice that as long as the user is actively keeping the phone awake through normal usage and they do not explicitly kill the foreground service, the app will function properly. There are exceptions we are still trying to figure out, particularly for some Chinese manufacturers of lower end models. We are now trying to reach out to them to see what else can be done.

tomekziel commented 4 years ago

Thanks for your reply! How about raw data from your tests? Is it possible to have it published? @hanyangtan ?