Open gabrc52 opened 1 year ago
Cartesian product on ('iOS', 'Android')
, (mobile, web)
, (send encrypted message after first login, send encrypted message after second login)
.
What is kept consistent is:
For consistency these tests will all use Chrome on Linux and uplink.mit.edu as the homserver and hosted Element instance. They will all use regular password-based accounts.
https://docs.google.com/document/d/14N5Pg5NFpxz9jb-TdQntd4iy_5fGNk3cS7YQ42UqODM/edit
I tested a couple cases and mostly everything worked. Found other minor bugs.
Having 2 devices wasn't an issue, but having only one at a time may (I was able to reproduce the "room upgrade" bug which led to broken UI), but it was recoverable by setting up secure backup then verifying.
It appears that end-to-end encryption issue are somewhat common(!) So we need to test which setups result in end-to-end encryption woes. This could kill the entire effort.
For now let's assume 2-device setups of laptop and phone. Let's try doing some 6.102 partitioning:
Partition on whether the first device is online while logging in to the second device.
Partition on when the first end-to-end conversation is started:
Partition on whether the user joined via an email invite:
Partition on first device:
Partition on laptop OS (probably unimportant unless there are any bugs specific to Safari or Edge):
Partition on mobile OS:
Login method
Whether logouts are involved (I've been assuming no)
User accepts the verification prompt in the first place