Ready FAMB for a new interaction-dashboard study

[zepumph]

From a meeting quite recently with @DianaTavares and @kathy-phet. We want to conduct another study that will be very similar to the CLB one last spring. Here are the requirements:

• We will use 2.1.9 as the latest patched and published FAMB version
• same encryption of student IDs
• screens=2.3 (notice the dot instead of the comma, #oldversionproblems)
• Last time the links were inlined in the qualtrics assignment, but now we want it to be a link open on another tab. We will need to create an IFrame wrapper of 10124X618 to support this. (probably where we just launch the login wrapper in iframe, rather than the login wrapper redirecting to a sim in iframe, since that will be more streamlined to what we have already done.
  • allowfullscreen was in the iframe before, but I don't think we will want this, since fullscreen mode would mess with scenery input coordinates.
  • add "scrolling=no" to the iframe
• There will be 4 groups, so we will need 4 links (with updated widget_ids)

Here is a link that can be used to see the current implementation of the FAMB sim (with json events and studio interface): https://phet-io.colorado.edu/sims/forces-and-motion-basics/2.1.9-phetio/wrappers/instance-proxies/instance-proxies.html?screens=2.3&phet-io.log=console

Likely many of the commits to the login and record wrapper to support CLB will be able to be cherry-picked easily into this release branch, but seeing as 2.1.0 was published Oct, 2016, perhaps I'm just being optimistic.

@kathy-phet has told me to put this as top priority for PhET-iO time, even perhaps pushing some a11y time to later next week to get this done. As a result I'm going to assign @chrisklus and @samreid too to comment and potentially assist as they see fit. Most of this will be worked on tomorrow, and then hopefully finished up early next week.

[zepumph]

@kathy-phet just mentioned, that because of the maintenance release hassle, and since we will only have the data internally (in PhET), the encryption is less important.

[kathy-phet]

@kathy-phet has told me to put this as top priority for PhET-iO time, even perhaps pushing some a11y time to later next week to get this done.

To be clear, only if swapping when a11y and PhET-iO time happens this week (not changing amount of either) doesn't cause issues for a11y deadlines. So check with Emily.

I went ahead and sent an email inquiry to Emily to check on timing.

[samreid]

allowfullscreen was in the iframe before, but I don't think we will want this, since fullscreen mode would mess with scenery input coordinates.

Playback supports changing frame size, but it would complicate the interaction dashboard significantly.

When do you want to run the study?

[DianaTavares]

@samreid we don't have the exact date yet. We are going to meet the teacher of that class Tuesday, but he mentioned "a couple weeks". It's one of the first topics in the program. Tuesday I will have a specific day!

[kathy-phet]

We are getting the Qualtrics (activity) side of the study ready by early next week.

[zepumph]

I'm going to try to do as much of this as possible today, I'm going to start by making the "iframe" wrapper.

[zepumph]

I only committed to the FAMB 2.1-phetio shas in the phet-io repo, creating a forces-and-motion-basics-2.1-phetio branch. In the above commits I:

• created the iframe wrapper, that we can fill in with any URL.
• updated the metacog recording to use the new metacog library (different api)
• updated the login wrapper substantially to support things like promptext, encryption, learner_token, learnerID in session storage, and loginURL redirect (changes in Recorder too).

Basically there has been much hackary in this branch to support our modern recording practices. That said I am currently able to record to metacog, and I know that sessions successfully recorded and saved to their database. I haven't yet gotten the logs back yet, but I'll comment on their intactfullness in this issue when I have investegated.

If these sessions are good, I feel like (if we absolutely had to), we could do a bit more testing, publish the maintenance release, and then just go for the study. Which is good news, here are some more improvements that I want to work on that either don't work, or I haven't tested at all:

• [x] emitStatesInterval does not exist in this branch at all, I'm not sure how hard it would be to add it
• [x] encryption: I think all I have to do is copy the CLB encryption method into the branch. I will want to test this thoroughly though.
• [x] prompt text in login wrapper: I think it was successfully added in the above commit (when I basically copy pasted master's login wrapper into the branch and then made tweaks for api support), but I want to test this too.

[zepumph]

encryption method and phet-io.emitStatesInterval added in the above commits.

EDIT: tagging https://github.com/phetsims/capacitor-lab-basics/issues/236 here because it is nice to have a link to that issue here, to see what we did for last study.

[zepumph]

I looked over the data logs that came back, and things are looking good, very much as good as I can hope. The database recording's full stack has seem to just work out of the box for us this time.

From here:

1. I would like a review from @samreid, him to make recommendations.
2. Then, when all seems good to him, we should publish a new maintenance release, do appropriate rc procedure for it (I'm unsure about specifics)
3. Then we should do more testing with the published version (testing (a) metacog stack, (b) the exact parameters and customizations desired for the study
4. Create the "links" for @DianaTavares to embed into qualtrics, and test that the redirecting is appropriate.
5. I want to note to @DianaTavares that I think it is very important to note in the assignment to NEVER close the sim, or press the back button (in order to keep the recording as a single session). Then when it is time to end the session, it is VITAL when prompted "you are about to leave, do you want to?" that they elect to STAY!!! and wait for the event queue progress bar to fully complete (to 100%) before they close. Otherwise it is likely that we could lose part of that participant's data log. @DianaTavares if any of that didn't make total sense to you, let's talk about it more in person.

Assigning to @samreid to review. Here are the steps I would recommend for the review:

• Look over the above commits, and note potential for dangerous code. Besides the addition of the phet-io.emitStatesInterval, all changes are in the wrapper side, so that is good news.
• Checkout shas for FAMB's 2.1-phetio branch
• Load and assess the following link: http://localhost/phet-io/wrappers/iframe/iframe.html?url=../login/login.html?console&screens=2.3&wrapper=record&validationRule=validateDigits&numberOfDigits=7&sim=forces-and-motion-basics&publisher_id=5d4c8ae1&key_name=phet-study_2018_q4&widget_id=forces-and-motion-basics-test&phet-io.emitInputEvents=true&phet-io.emitStates=true&promptText=This%20is%20a%20test%20prompt&encryption=encryptNumberV1&phet-io.emitStatesInterval=60
• I have been using 1000000 as a learner id
• Two obvious features to me that aren't in this older branch are (a) login doesn't support phettest, and (b) wrapper metadata is not passed to the simStarted event, only the sessionID is logged. I'm not sure if we want
• If you want to then test the metacog recording follow these steps:
  • swap or add metacog in exchange for console
  • record a session (I would use 1000000 again). Notice that the widget_id is a testing one, keep that for now.
  • After upwards of 20 minutes, sessions should be recorded, and you can use http://localhost/phet-io-website/root/metacog/request-data.html to get at those sessions. You will need to provide the application ID from metacog's website, let me know if you need help with that.
  • When you get the log, note that playback doesn't work with the current playback wrapper, but when inspecting the data log, things look good to me. If was helpful for me to search for mouseDown, you tend to get interesting parts of the log that way.
  • EDIT: there are already 2 sessions recorded to this widget_id from my recording earlier today.
• Let me know if you have any questions.

[samreid]

I skimmed the change sets, checked out 2.1-phetio, checked out shas, launched http://localhost/phet-io/wrappers/iframe/iframe.html?url=../login/login.html?console&screens=2.3&wrapper=record&validationRule=validateDigits&numberOfDigits=7&sim=forces-and-motion-basics&publisher_id=5d4c8ae1&key_name=phet-study_2018_q4&widget_id=forces-and-motion-basics-test&phet-io.emitInputEvents=true&phet-io.emitStates=true&promptText=This%20is%20a%20test%20prompt&encryption=encryptNumberV1&phet-io.emitStatesInterval=60 and entered 1000000. The sim launched in the iframe and I was able to use it and see input events streaming to the console.

So far, everything looks good. I did not try metacog recording. Have you tried loading the data into interaction-dashboard?

UPDATE: this will be our first interaction-dashboard simulation that has multiple screens. Is there a plan for supporting that? Or should we split up the activity into two one-screen sims?

[samreid]

I thought it was a bit odd that the login wrapper is also in the iframe, but perhaps that's acceptable given our short time frame.

[samreid]

I sent a session to metacog, with a few triangles, circles, then in the 2nd screen checking all checkboxes and tuning to 10 m/s and opening the about dialog.

[samreid]

It looks like we must use http://localhost/phet-io-website/root/metacog/request-data.html from master--not sure what to provide for URL.

[zepumph]

Have you tried loading the data into interaction-dashboard?

No I haven't, that could be a good test to see mouse events. It should be pretty easy to do that.

I thought it was a bit odd that the login wrapper is also in the iframe, but perhaps that's acceptable given our short time frame.

I also feel this way. It doesn't look bad, just a bit strange. It was the easiest way to implement the iframe wrapper, and then there is also no redirecting (which is nice to me).

UPDATE: this will be our first interaction-dashboard simulation that has multiple screens. Is there a plan for supporting that? Or should we split up the activity into two one-screen sims?

I was considering this a post processing step based on looking at the screenIndexProperty, but maybe it could be easier if we decided that you only used one screen per session. I can only assume that both screens are important to @DianaTavares' study, so I defer to her judgement.

[zepumph]

It looks like we must use http://localhost/phet-io-website/root/metacog/request-data.html from master--not sure what to provide for URL.

I used everything right of the ?url=

[samreid]

I tried the application id for UCB - PhET that I found on developer.metacog.com. For URL I tried everything to the right of the first url= in the link given here: https://github.com/phetsims/forces-and-motion-basics/issues/259#issuecomment-419668578

I was told

POST https://api.metacog.com/DataRequest 401 (Unauthorized)
request-data.html:241 {error: "Invalid authentication params."}

@zepumph can you please advise?

[zepumph]

Likely it is not the single application I'd (which matches our publisher_id in the query parameters, but instead any of the full Sha length hashes below it.

[samreid]

Thanks, I'm QUEUED for c1c556bf-9b90-4a0a-86d8-82bda6b5be17

[samreid]

When you get the log, note that playback doesn't work with the current playback wrapper,

I downloaded my data and confirmed that it doesn't play back properly in http://localhost/phet-io-wrappers/playback/playback.html?sim=forces-and-motion-basics&ea. Is this because the sim is too old? Doesn't @DianaTavares need a way to view the playbacks? If we cannot get playback events into the playback wrapper, will we be sure they can work in the interaction-dashboard?

[DianaTavares]

I don't need the playbacks. I only need the data to build a dashboard like the one that we did for CL:B (click map, elements used and the time vs clicks graphs)

[zepumph]

I agree that playbacks won't work; I'm not sure how hard it would be to get things working in that regard, perhaps we could check out an old version of the playback wrapper, or we could try to get the version on master backwards compatible.

Neither sound very appealing, and I'm not convinced that it is necessary. So instead I focused some time just now to see about plugging the temp FAMB data I had into the interaction dashboard. It took a bit to get the preprocessor to work, since we used to emit scenery input events a bit differently, but I parsed the old way and got the small event sessions I recorded on Saturday to overlay on the CLB screenshot that we had.

From here I am confident that we can do good work with the current version of the sim we have been outfitting.

[zepumph]

From here I will:

• [ ] Create a dev version in #260 for @kathy-phet and @DianaTavares to look at
• [ ] Create an RC for QA to look at before publishing the "maintenance release" to FAMB 2.1-phetio branch.

@samreid anything else to do here?

[samreid]

I have some hesitation/fear about moving forward with a legacy version--I don't know exactly what could go wrong, but just generally concerned that there may be a problem that's difficult to see an we may not run into for a while. But it seems like everything is OK so far, so maybe it is OK? What about recent maintenance releases like the iOS touch bug? Is that in the branch? On the other hand, since we know it would take a while to create and test a new version from master, maybe it's preferable to use the legacy version. I'm glad you could render the data in the interaction dashboard, that was a good test to do.

[zepumph]

What about recent maintenance releases like the iOS touch bug?

This version was patched

I'm glad you could render the data in the interaction dashboard, that was a good test to do.

+1

[zepumph]

Tagging @jonathanolson in the off chance that it is not good to be rc deploying off of FAMB 2.1-phetio right now.

[jonathanolson]

No objections, let me know when work is complete (in case I need to batch maintenance-release something).

[zepumph]

Work is done here, final link can be found in https://github.com/phetsims/forces-and-motion-basics/issues/268#issuecomment-422107586. Closing