search

phetsims / qa

Quality Assurance Task Tracking
MIT License
12 stars 8 forks source link

Android App Test: PhET For Schools #1088

Closed mattpen closed 2 months ago

mattpen commented 4 months ago

Android App QA Test

Links and dates

OS versions to test

For Android, test the earliest and latest versions of the OS that are in the QA arsenal, and make a note of the version you test if it is different from the one mentioned.

Form Factors to test

Only necessary if the test includes user interface features.

Specific Test Cases

  1. Upon install, the app should prompt for an Access Code. QA should use this access code: 23QA999Q. If testing after June 1, use the code 24QA999Q.
  2. There should be no way to evade the prompt in (1), e.g. by clicking the background, hitting esc on a keyboard, using the back button, or any other means.
  3. Once a valid code is entered, the submit button should be clickable. After submitting the code, the app should work normally.
  4. In the PhET Info screen (tap the PhET logo in the bottom right of the collection view), there is a new section for the PhET For Schools Program.
  5. The expiration date should be Aug 31, 2024 if your access code started with "23", or 2025 if your access code started with "24"
  6. The access code you entered should show up.
  7. Validate that the access code can be changed using the button. Change it to 23QA888Z and verify that the dialog has the correct info, then change it back to 23QA999Q.
  8. Verify statistics for app and sim runs are reported:
    1. You can access the analytics by logging in to https://phet-direct.colorado.edu/analytics (creds provided separately)
    2. After you launch the app or run any sim, you should see a log of the event in the "Most recently collected events" table.
    3. After you run any sim, the appropriate count should be updated in the "Sim runs per organization" table.
    4. After you enter a new access code, the appropriate count should be updated in the "Unique Users per Organization" table.
    5. If you run the app or sims offline, all of the events will be reported at once after you (a) bring the device online and (b) conduct another app launch or sim run

Simulation Update Test Cases

These should be tested for every update.

This test is for the PhET For Schools app, which is likely impossible for QA to install from the play store. Please just test by installing it from the Internal Test.

New releases are fetched.

 1. Follow the steps for Testing an initial Play Store installation 10. Open the app, making sure that internet access is enabled on the device. 11. Within 2 hours, the app should have the *Sim release for testing the online update system* in Links and dates.

Common Procedures

Clear the Play Store cache

 This should allow you to instantly switch to your selected track for your Play Store user. 1. On your Android device go to Settings > Apps > Google Play Store. (Shortcut: long tap on the play store and click App Info) 2. Click Force Stop. 3. Click Storage and click Clear Cache (no need to click Clear Data) 4. Open https://play.google.com/store/apps/details?id=edu.colorado.phet.androidApp in the Play Store on your device.

Testing an initial Play Store installation

 1. Uninstall the PhET Android App. 2. Visit https://play.google.com/apps/internaltest/4697762600441754675/ and switch to the Google account you use on the Play Store. If there is a button called "Accept Invite" then click it. 3. Clear the Play Store cache. 4. Visit the [Play Store](https://play.google.com/store/apps/details?id=edu.colorado.phet.androidApp) and verify that it is offering the test version, then click Install.

Testing a Play Store Update

 1. Uninstall the PhET Android App. 2. Visit https://play.google.com/apps/internaltest/4697762600441754675/ and switch to the google account you use on the Play Store. If there is a button called "Leave Program" then click it. 3. Clear the Play Store cache. 4. Verify the [Play Store](https://play.google.com/store/apps/details?id=edu.colorado.phet.androidApp) is offering the previous version of the app install it. 5. Open the app (important!). 6. Visit https://play.google.com/apps/internaltest/4697762600441754675/ and switch to the google account you use on the Play Store. Click Accept Invite. 7. Clear the Play Store cache. 8. Verify that the [Play Store](https://play.google.com/store/apps/details?id=edu.colorado.phet.androidApp) is offering an upgrade to the test version and click Upgrade. 13. Open the app again and conduct upgrade specific testing.
mattpen commented 4 months ago

Helpful info for internal testing:

Enable play store: https://www.google.com/chromebook/howto/enable-google-play-store/ (you probably need to be on a personal gmail account, not on a colorado.edu account)

Turn on developer settings and allow internal app sharing: https://support.google.com/googleplay/android-developer/answer/9844679?hl=en#on&zippy=%2Cadd-authorized-uploaders%2Cadd-authorized-testers%2Cupload-and-share-apps-for-testing

mattpen commented 4 months ago

There is a new build available. https://play.google.com/apps/test/RQWzILOtqlk/ahAO29uNRTJDC-vVMa8NXoycHa_uVIc_TYIGI5wvrSXoYyXKYiUJFLYamVx-Cparlj-SZtj6SZ9OHlIbK0L_jLlFal

This build should fix these issues:

kathy-phet commented 4 months ago

Sorry - Github was hung up, and I closed this by mistake.

mattpen commented 4 months ago

Issues fixed in the next build:

Latest build: https://play.google.com/apps/test/RQWzILOtqlk/ahAO29uNQ8KN3P7oLSADwFJZL6IxTfgevJlHXattdUQ5zfgkrttvdS8p06ekzd3MzDA5nTr4STTA25xZhRLwEbcw5r

Important notes:

KatieWoe commented 4 months ago

When I open the app (Android 14 tablet) I don't see the updated sims. Once I close and open it again they show up. Is this expected/acceptable?

mattpen commented 4 months ago

When I open the app (Android 14 tablet) I don't see the updated sims. Once I close and open it again they show up. Is this expected/acceptable?

That is correct, with the new update system the sims won't pop in once they are ready, instead they will be delayed until the next launch.

mattpen commented 4 months ago

@KatieWoe - there is a new build available: https://play.google.com/apps/test/RQWzILOtqlk/ahAO29uNSl-sUI8XJ2B-pFtRTAhTwGA1yJDqxOiAcRkDgpgcTtRGrFuKDw25Dr1tB0dDjxqEt8nNKOIoCFjfwlEH2i

This build includes the following updates:

Can you please verify that these issues are fixed? Thanks!

KatieWoe commented 4 months ago

Other than https://github.com/phetsims/phet-android-app/issues/187#issuecomment-2140511101, the issues in https://github.com/phetsims/qa/issues/1088#issuecomment-2138129044 look ok.