Closed pixelzoom closed 3 years ago
Let's discuss the plan to provide a dev version. Two concerns:
(1) The dev team has previously recommended against providing dev versions. A dev version cannot be patched via PhET processes and tools. An RC version has no additional cost, and provides all of the maintenance benefits.
(2) The dev team has previously expressed concerns about the impact of study traffic on phet-dev.colorado.edu, and has recommended against giving links to phet-dev.colorado.edu. This will be an issue for a dev or RC version. Do we know how many students are involved in this study? (I recall that @kathy-phet was going to ask.) Has @mattpen been consulted?
Re the QA plan:
QA will do some testing, as will @kathy-phet, @arouinfar, and the researcher.
PhET-iO instrumentation (#85) required big changes throughout the sim. There is a very good chance that something is broken. So I recommend a thorough test, definitely not a spot check. If we don't have time for a thorough test, then let's make sure that we are all aware of the risk to the study.
I'm unclear on what "QA will do some testing" means. I know what the process and expectations are for "Dev Test", "RC Test", and "RC-Lite Test". It sounds like PhET will be doing similar QA for more Ido studies, so I think its important that we all know how to communicate about this type of test. Can we define a name and process for this type of test, which I interpret to be less rigorous than "RC-Lite"?
Proposal, assuming that PhET will provide an RC version:
"RC-Research":
- The sim will be reviewed by QA, preferrably on the platform(s) that will be used by the researcher.
- A test martix will be created.
- A QA issue will be created and labeled "RC-Research".
- The QA issue will be specific about what to focus on. Focus will be tailored to the needs of the research study. (Specific features, specific screens,...)
- Test coverage will be somewhere below "RC-Lite" testing, as time permits.
- Additionally, the simulation team and the researcher may be asked to test.
3/4/21 phet-io meeting
We discussed the deliverable and the QA plan.
Consensus:
3/11/21 phet-io meeting:
The research collaborator was provided with 1.1.0-dev.2 for review. Feedback was that it looked good. There were no change requests.
Update: The researcher needs the link to the final version by 4/4/21.
NOTE: I can publish the RC version using grunt rc
. Work with @zepumph to create the links for QA and the researcher, so that Metacog recording will work.
Note that the qa issues will likely look much like https://github.com/phetsims/QA/issues/624.
1.1.0-rc.1 has been published and is ready for testing. @zepumph and I created the QA issue, with appropriate links: https://github.com/phetsims/QA/issues/627.
On 3/23/21, @arouinfar sent the research collaborator a link to https://phet-dev.colorado.edu/html/molecule-polarity/1.1.0-rc.1/phet-io/, and requested feedback by Monday 3/29/21 @ 9:00 AM MDT:
Molecule Polarity is currently undergoing QA testing for your upcoming study. Here is the latest version: https://phet-dev.colorado.edu/html/molecule-polarity/1.1.0-rc.1/phet-io/
Please review this version to ensure that it will meet your needs. If you have any feedback or find any bugs, please let us know by Monday 9:00 AM MDT so that we will have time to make changes before your study begins.
Best, Amy
QA complete testing of 1.1.0-rc.1 on 3/24/21. There were 2 issues sim-specific issues (#106, #107), see phetsims/QA#627.
Feedback from the research collaborator on 3/25/21:
Hi Amy and all, This looks terrific - no comments from us. Thanks.
I will proceed with the next RC (spot-check).
Progress is blocked. I can't create the next RC due to a problem with the build server, see https://github.com/phetsims/perennial/issues/221.
phetsims/perennial#221 is no longer blocking.
1.1.0-rc.2 was published and is being tested in https://github.com/phetsims/QA/issues/631.
1.1.0-rc.2 testing has been completed, see https://github.com/phetsims/QA/issues/631. All previous issues were successfully verified. There was one new issue #113 which does not need to be addressed for this research version.
@kathy-phet @arouinfar 1.1.0-rc.2 is ready to be delivered to the research collaborator. The link to the index wrapper is: https://phet-dev.colorado.edu/html/molecule-polarity/1.1.0-rc.2/phet-io
Please note in this issue when 1.1.0-rc.2 has been delivered, then close this issue.
Thanks - will work with MK when he gets back from vacation, to make sure I send correct links for data collection and testing. On hold until March 31.
Unassigning myself since @kathy-phet will be sending the final links to the collaborator.
I don't want phet-io links in public issues, so I made https://github.com/phetsims/special-ops/issues/195 for the delivery, please see there. Closing.
This issue was the motivation for #85 (Add basic PhET-iO instrumentation).
A research collaborator would like to use this sim for a study. @kathy-phet provided some info on Slack, here's my summary:
a dev versionan RC version.some testing"RC-Research" testing, as will @kathy-phet, @arouinfar, and the researcher.Unless I hear otherwise, I'll assume "end of March" means 3/31/21.