search

SMILEConsortium / smile_teacher_android

The SMILE Teacher Android App
http://www.smileconsortium.org
Apache License 2.0
1 stars 2 forks source link

Integration Test #2 SMILE Teacher + SMILE Plug Server #118

Closed truedat101 closed 10 years ago

truedat101 commented 10 years ago

We need to complete a full batch of testing against the current versions. The description below is a complete matrix of the testing needed. The task will be to run through a complete integration test using the test plan below. Copy the results into subsequent comments to this issue. It is expected that complete test runs may not be possible if there are P1 issues found. We may need to re-run the test plan. In these cases, provide a new cut-n-paste of the results below with details of the test run. We must only be testing tagged & released code at this point. @truedat101 will provide the tags to use for testing.

Successful completion will allow us to continue with a test against the firmware on a SMILE Plug: https://github.com/SMILEConsortium/smileplug-bsp/issues/19

Test Plan

DATE:, < INSERT DATE > Tester Github ID:, < INSERT TESTER ID > SMILE Android Version:,< INSERT VERSION > SMILE Server Version:, < INSERT VERSION > HOST OS:, < INSERT VERSION > Android Device Model:, < INSERT MODEL > Android OS Version:, < INSERT OS VERSION > Android Browser & Version:, < INSERT BROWSER and VERSION > *Desktop Browser & Version, < INSERT BROWSER and VERSION > Number of Student Web Clients:, 3

Scenario 1: Run session and let students create the questions

  1. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  2. ,_TEACHER_APP : Verify Login Panel appears well formated for screen, Pass :+1: [ ] Fail :-1: [ ]
  3. ,TEACHER APP : Verify IP address is default: 10.1.0.1, Pass :+1: [ ] Fail :-1: [ ]
  4. ,TEACHER APP : Verify invalid IP address is not possible, Pass :+1: [ ] Fail :-1: [ ]
  5. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  6. ,SERVER: Startup server as root on port 80, Pass :+1: [ ] Fail :-1: [ ]
  7. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  8. ,TEACHER APP : Verify cancel button, Pass :+1: [ ] Fail :-1: [ ]
  9. ,TEACHER APP : Verify connect (again), Pass :+1: [ ] Fail :-1: [ ]
  10. ,STUDENT WEB : Verify user can reach login page, Pass :+1: [ ] Fail :-1: [ ]
  11. ,STUDENT WEB : Verify user can login, Pass :+1: [ ] Fail :-1: [ ]
  12. ,STUDENT WEB : Verify user on 'Get Ready' phase, Pass :+1: [ ] Fail :-1: [ ]
  13. ,TEACHER APP : Input Teacher session info and hit create, Pass :+1: [ ] Fail :-1: [ ]
  14. ,TEACHER APP : Verify Teacher can select Start Making Questions, Pass :+1: [ ] Fail :-1: [ ]
  15. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  16. ,TEACHER APP : Verify at Students Screen with at least one student present, Pass :+1: [ ] Fail :-1: [ ]
  17. ,STUDENT WEB : Verify Student at Start Make Questions phase, Pass :+1: [ ] Fail :-1: [ ]
  18. ,STUDENT WEB : Verify there are no major layout or UI defects, Pass :+1: [ ] Fail :-1: [ ]
  19. ,STUDENT WEB : Verify Student can make a question without an image, Pass :+1: [ ] Fail :-1: [ ]
  20. ,STUDENT WEB : Verify Student can make a question with an image (must use image < 1MB), Pass :+1: [ ] Fail :-1: [ ]
  21. ,STUDENT WEB : Repeat previous two steps for each additional web client, Pass :+1: [ ] Fail :-1: [ ]
  22. ,TEACHER APP : Verify questions are received for each student, Pass :+1: [ ] Fail :-1: [ ]
  23. ,TEACHER APP : Verify teacher can view Question Detail (images should be available to view) for each question sent, Pass :+1: [ ] Fail :-1: [ ]
  24. ,TEACHER APP : Verify 'Start Solving Questions' button is available, and starts 'solving' phase when clicked, Pass :+1: [ ] Fail :-1: [ ]
  25. ,STUDENT WEB : Verify Student in 'Answer Questions' phase, Pass :+1: [ ] Fail :-1: [ ]
  26. ,STUDENT WEB : Verify Number of Questions listed is equal to number of questions teacher sees, Pass :+1: [ ] Fail :-1: [ ]
  27. ,STUDENT WEB : Verify Student can answer questions and select a positive or negative rating, Pass :+1: [ ] Fail :-1: [ ]
  28. ,STUDENT WEB : Repeat previous two steps for each additional web client, Pass :+1: [ ] Fail :-1: [ ]
  29. ,STUDENT WEB : Verify Student can submit answers when finished, should see 'waiting' message, Pass :+1: [ ] Fail :-1: [ ]
  30. ,TEACHER APP : Verify in Students tab that all students have answered all questions, Pass :+1: [ ] Fail :-1: [ ]
  31. ,TEACHER APP : Verify click on 'Finish Show Results' causes results to display, Pass :+1: [ ] Fail :-1: [ ]
  32. ,STUDENT WEB : Verify Student can review results, Pass :+1: [ ] Fail :-1: [ ]
  33. ,STUDENT WEB : Verify Student can view question details for each question: Pass :+1: [ ] Fail :-1: [ ]
  34. ,TEACHER APP : Verify in Students Tab the results are showing with correct Top Scorer Correct % completion Correct % right Pass :+1: [ ] Fail :-1: [ ]
  35. ,TEACHER APP : Verify Teacher can save all questions into new iqset "IT2 set 1", Pass :+1: [ ] Fail :-1: [ ]
  36. ,TEACHER APP : Verify "Restart" menu sends each student back to Login Screen , Pass :+1: [ ] Fail :-1: [ ]
  37. ,IQManager : Verify Session was created and is viewable in the sessions list , Pass :+1: [ ] Fail :-1: [ ]
  38. ,IQManager : Verify Details of Session , Pass :+1: [ ] Fail :-1: [ ]
  39. ,IQManager : Verify IQ Set exists called "IT2 set1" , Pass :+1: [ ] Fail :-1: [ ]
  40. ,IQManager : Verify IQ Set details are correct and images can be loaded, Pass :+1: [ ] Fail :-1: [ ]

Scenario 2: Run session and use prepared questions

  1. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  2. ,_TEACHER_APP : Verify Login Panel appears well formated for screen, Pass :+1: [ ] Fail :-1: [ ]
  3. ,TEACHER APP : Verify IP address is default: 10.1.0.1, Pass :+1: [ ] Fail :-1: [ ]
  4. ,TEACHER APP : Verify invalid IP address is not possible, Pass :+1: [ ] Fail :-1: [ ]
  5. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  6. ,SERVER: Startup server as root on port 80, Pass :+1: [ ] Fail :-1: [ ]
  7. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  8. ,TEACHER APP : Verify cancel button, Pass :+1: [ ] Fail :-1: [ ]
  9. ,TEACHER APP : Verify connect (again), Pass :+1: [ ] Fail :-1: [ ]
  10. ,STUDENT WEB : Verify user can reach login page, Pass :+1: [ ] Fail :-1: [ ]
  11. ,STUDENT WEB : Verify user can login, Pass :+1: [ ] Fail :-1: [ ]
  12. ,STUDENT WEB : Verify user on 'Get Ready' phase, Pass :+1: [ ] Fail :-1: [ ]
  13. ,TEACHER APP : Input Teacher session info and hit create, Pass :+1: [ ] Fail :-1: [ ]
  14. ,TEACHER APP : Verify Teacher can select Use Prepared Questions, Pass :+1: [ ] Fail :-1: [ ]
  15. ,TEACHER APP : Verify Teacher is presented a list of prepared questions, properly formatted and listing number of questions, Pass :+1: [ ] Fail :-1: [ ]
  16. ,TEACHER APP : Verify Teacher can select "IT2 set 1" , Pass :+1: [ ] Fail :-1: [ ]
  17. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  18. ,TEACHER APP : Verify in Questions Tab that correct number of questions are loaded, Pass :+1: [ ] Fail :-1: [ ]
  19. ,TEACHER APP : Verify at Students Screen with at least one student present, Pass :+1: [ ] Fail :-1: [ ]
  20. ,STUDENT WEB : Verify Student at Start Make Questions phase, Pass :+1: [ ] Fail :-1: [ ]
  21. ,STUDENT WEB : Verify there are no major layout or UI defects, Pass :+1: [ ] Fail :-1: [ ]
  22. ,TEACHER APP : Verify Teacher can click Start Solving Questions, Pass :+1: [ ] Fail :-1: [ ]
  23. ,TEACHER APP : Verify questions are properly loaded and that images are present, Pass :+1: [ ] Fail :-1: [ ]
  24. ,STUDENT WEB : Verify Student at Answer Questions phase, Pass :+1: [ ] Fail :-1: [ ]
  25. ,TEACHER APP : Verify questions are received for each student, Pass :+1: [ ] Fail :-1: [ ]
  26. ,TEACHER APP : Verify teacher can view Question Detail (images should be available to view) for each question sent, Pass :+1: [ ] Fail :-1: [ ]
  27. ,TEACHER APP : Verify 'Start Solving Questions' button is available, and starts 'solving' phase when clicked, Pass :+1: [ ] Fail :-1: [ ]
  28. ,STUDENT WEB : Verify Student in 'Answer Questions' phase, Pass :+1: [ ] Fail :-1: [ ]
  29. ,STUDENT WEB : Verify Number of Questions listed is equal to number of questions teacher sees, Pass :+1: [ ] Fail :-1: [ ]
  30. ,STUDENT WEB : Verify Student can answer questions and select a positive or negative rating, Pass :+1: [ ] Fail :-1: [ ]
  31. ,STUDENT WEB : Verify Student can submit answers when finished, should see 'waiting' message, Pass :+1: [ ] Fail :-1: [ ]
  32. ,STUDENT WEB : Repeat previous two steps for each additional web client, Pass :+1: [ ] Fail :-1: [ ]
  33. ,TEACHER APP : Verify in Students tab that all students have answered all questions, Pass :+1: [ ] Fail :-1: [ ]
  34. ,TEACHER APP : Verify click on 'Finish Show Results' causes results to display, Pass :+1: [ ] Fail :-1: [ ]
  35. ,STUDENT WEB : Verify Student can review results, Pass :+1: [ ] Fail :-1: [ ]
  36. ,STUDENT WEB : Verify Student can view question details for each question: Pass :+1: [ ] Fail :-1: [ ]
  37. ,TEACHER APP : Verify in Students Tab the results are showing with correct Top Scorer Correct % completion Correct % right, Pass :+1: [ ] Fail :-1: [ ]
  38. ,TEACHER APP : Verify "Restart" menu sends each student back to Login Screen , Pass :+1: [ ] Fail :-1: [ ]
  39. ,IQManager : Verify Session was created and is viewable in the sessions list , Pass :+1: [ ] Fail :-1: [ ]
  40. ,IQManager : Verify Details of Session, make sure all session data is there along with images , Pass :+1: [ ] Fail :-1: [ ]

Scenario 3: One-time things to verify

  1. ,TEACHER APP : Verify Teacher can view "About" menu and proper version # is displayed, :+1: [ ] Fail :-1: [ ]
  2. ,TEACHER APP : Verify Send Report button works and sends to server issue on github, Pass :+1: [ ] Fail :-1: [ ]
  3. ,TEACHER APP : Verify deletion of at least two questions in the same IQSet during the Make Questions phase choosing questions in the middle of the set, Pass :+1: [ ] Fail :-1: [ ]
  4. ,IQManager : Upload IQSet works with the three JAMsj csv files, Pass :+1: [ ] Fail :-1: [ ]
  5. ,IQManager : Verify upload succeeds for each IQSet and view them in the IQSet list, Pass :+1: [ ] Fail :-1: [ ]

Scenario 4: Repeat Scenario 2, but use 15 Web Clients (Use browser on server host, use http://localhost, with each client in a tab)

  1. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  2. ,_TEACHER_APP : Verify Login Panel appears well formated for screen, Pass :+1: [ ] Fail :-1: [ ]
  3. ,TEACHER APP : Verify IP address is default: 10.1.0.1, Pass :+1: [ ] Fail :-1: [ ]
  4. ,TEACHER APP : Verify invalid IP address is not possible, Pass :+1: [ ] Fail :-1: [ ]
  5. ,TEACHER APP : Start teacher app, Pass :+1: [ ] Fail :-1: [ ]
  6. ,SERVER: Startup server as root on port 80, Pass :+1: [ ] Fail :-1: [ ]
  7. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  8. ,TEACHER APP : Verify cancel button, Pass :+1: [ ] Fail :-1: [ ]
  9. ,TEACHER APP : Verify connect (again), Pass :+1: [ ] Fail :-1: [ ]
  10. ,STUDENT WEB : Verify user can reach login page, Pass :+1: [ ] Fail :-1: [ ]
  11. ,STUDENT WEB : Verify user can login, Pass :+1: [ ] Fail :-1: [ ]
  12. ,STUDENT WEB : Verify user on 'Get Ready' phase, Pass :+1: [ ] Fail :-1: [ ]
  13. ,TEACHER APP : Input Teacher session info and hit create, Pass :+1: [ ] Fail :-1: [ ]
  14. ,TEACHER APP : Verify Teacher can select Use Prepared Questions, Pass :+1: [ ] Fail :-1: [ ]
  15. ,TEACHER APP : Verify Teacher is presented a list of prepared questions, properly formatted and listing number of questions, Pass :+1: [ ] Fail :-1: [ ]
  16. ,TEACHER APP : Verify connect success, Pass :+1: [ ] Fail :-1: [ ]
  17. ,TEACHER APP : Verify in Questions Tab that correct number of questions are loaded, Pass :+1: [ ] Fail :-1: [ ]
  18. ,TEACHER APP : Verify at Students Screen with at least one student present, Pass :+1: [ ] Fail :-1: [ ]
  19. ,STUDENT WEB : Verify Student at Start Make Questions phase, Pass :+1: [ ] Fail :-1: [ ]
  20. ,STUDENT WEB : Verify there are no major layout or UI defects, Pass :+1: [ ] Fail :-1: [ ]
  21. ,TEACHER APP : Verify Teacher can click Start Solving Questions, Pass :+1: [ ] Fail :-1: [ ]
  22. ,STUDENT WEB : Verify Student at Answer Questions phase, Pass :+1: [ ] Fail :-1: [ ]
  23. ,TEACHER APP : Verify questions are received for each student, Pass :+1: [ ] Fail :-1: [ ]
  24. ,TEACHER APP : Verify teacher can view Question Detail (images should be available to view) for each question sent, Pass :+1: [ ] Fail :-1: [ ]
  25. ,TEACHER APP : Verify 'Start Solving Questions' button is available, and starts 'solving' phase when clicked, Pass :+1: [ ] Fail :-1: [ ]
  26. ,STUDENT WEB : Verify Student in 'Answer Questions' phase, Pass :+1: [ ] Fail :-1: [ ]
  27. ,STUDENT WEB : Verify Number of Questions listed is equal to number of questions teacher sees, Pass :+1: [ ] Fail :-1: [ ]
  28. ,STUDENT WEB : Verify Student can answer questions and select a positive or negative rating, Pass :+1: [ ] Fail :-1: [ ]
  29. ,STUDENT WEB : Verify Student can submit answers when finished, should see 'waiting' message, Pass :+1: [ ] Fail :-1: [ ]
  30. ,STUDENT WEB : Repeat previous two steps for each additional web client, Pass :+1: [ ] Fail :-1: [ ]
  31. ,TEACHER APP : Verify in Students tab that all students have answered all questions, Pass :+1: [ ] Fail :-1: [ ]
  32. ,TEACHER APP : Verify click on 'Finish Show Results' causes results to display, Pass :+1: [ ] Fail :-1: [ ]
  33. ,STUDENT WEB : Verify Student can review results, Pass :+1: [ ] Fail :-1: [ ]
  34. ,STUDENT WEB : Verify Student can view question details for each question: Pass :+1: [ ] Fail :-1: [ ]
  35. ,TEACHER APP : Verify in Students Tab the results are showing with correct Top Scorer Correct % completion Correct % right, Pass :+1: [ ] Fail :-1: [ ]
  36. ,TEACHER APP : Verify "Restart" menu sends each student back to Login Screen , Pass :+1: [ ] Fail :-1: [ ]
  37. ,IQManager : Verify Session was created and is viewable in the sessions list , Pass :+1: [ ] Fail :-1: [ ]
  38. ,IQManager : Verify Details of Session , Pass :+1: [ ] Fail :-1: [ ]
truedat101 commented 10 years ago

@chrqls I'll get some tags set up and we should retest everything.

truedat101 commented 10 years ago

Please use SMILE Teacher tag: 0.9.9b2-jamsj Please use SMILE Server tag: 1.0.2-jamsj

truedat101 commented 10 years ago

I'm pretty sure we've got a bug in https://github.com/SMILEConsortium/node-smile-server/issues/74 . Can you verify #74 is working properly before doing anything else. I hate to spend this time to prepare a release if an issue is left open and has introduced a regression. If we can't start testing this, please let me know, and we'll need to spin a new release once you've verified this other issue is fixed.

truedat101 commented 10 years ago

Please use SMILE Teacher tag: 0.9.9b3-jamsj Please use SMILE Server tag: 1.0.3-jamsj

chrqls commented 10 years ago

Ok

chrqls commented 10 years ago

Scenario 1: Run session and let students create the questions

Scenario 2: Run session and use prepared questions

Scenario 3: One-time things to verify

Scenario 4: Repeat Scenario 2, but use 15 Web Clients (Use browser on server host, use http://localhost, with each client in a tab)

truedat101 commented 10 years ago

Great. Will start building a firmware based on these tags. Thanks.