Prepare usability evaluation report

[RLC-DCPPC]

This is to prepare nih-cfde/deliverables#108 based on #19, and to prepare for #20.

[RLC-DCPPC]

Here is the report from Interface Guru that should be useful in this effort: CFDE report_final_07132020.pdf

[RLC-DCPPC]

hi @ngaddis . As discussed the other day: "tag, you're it" :-)

I need to find you Cia's report and you can ask me for other things that you need in getting this done.

[RLC-DCPPC]

Cia's report: CFDE report_final_07132020.pdf

[dcfcjcheadle]

Hi @RLC-DCPPC,

Is there a template or format you want us to follow when drafting this usability report?

Thanks!

[RLC-DCPPC]

Hi @RLC-DCPPC,

Is there a template or format you want us to follow when drafting this usability report?

Thanks!

Oops. Just found this ;-)

No particular format. Suggest you touch base with @fridayek who has handled some previous docs for us. Google doc is fine and we'll put it in our static deliverables folder out in CFDE GoogleLand as the actualy delivery when the time comes.

Along those lines, @ngaddis , how is this coming along?

[ngaddis]

@RLC-DCPPC I believe not much progress has been made so far. @swandle06 has taken over leading this task.

[swandle06]

Yes, I was waiting on the answer to the previous query, but we'll be starting on this next week.

[swandle06]

Template for this report is here: https://docs.google.com/document/d/19savT0LUzg--B6JAmbHOusdpboGHMeqGLn5U90LMRsA/edit?usp=sharing

[swandle06]

@RLC-DCPPC Has the Interface Guru document been (or will it be) submitted as a separate deliverable, or will we be incorporating it into this report? Their report notes three cohorts to be studied (Program Officers, Data Scientists and Researchers) with the report focusing on the first cohort. Are they planning to perform a similar study with the other two cohorts?

[RLC-DCPPC]

@swandle06 : To my knowledge we are not planning to submit the IG report as a deliverable (@ACharbonneau, @owhite, @carlkesselman , do any of you know if this has been given to NIH?).

Yes, we expect IG will follow through with the 2nd and 3rd groups later this fall.

[ACharbonneau]

We were not planning to. It's sort of anonymized, but I feel like it would be weird to give NIH a report of us interviewing NIH people. Also, it's written for us, not for NIH. The audience is all wrong.

[dcfcjcheadle]

Hi @RLC-DCPPC

@swandle06 and I worked on the outline of the CFDE Portal & Dashboard Usability Report last week. The outline is built around User Roles (Researcher & Data Scientist) each with a handful of user activities that we will work through. Based on our experience wearing these user role hats, we highlight features of the portal as well as recommendations for the how the portal might be improved. I fleshed out a simple user activity so that you can see how we walk through the activity and how we arrive at our recommendations.

The report outline is here.

Additionally, we have some questions for you about how to construct this document - perhaps we can talk about this on our call later today. As a starting point for this document, we've been following the format of the IG report, however we realize that we should incorporate some more positive messaging.

• Recommendations Formatting - currently we have recommendations after each user activity, but we could easily pull these out and put them in a recommendations section at the end of each Role section. Alternatively we could leave the detailed recommendations after each user activity and summarize at a high level the recommendations for each user role. Do you have opinions of how you might want this formatted?

• As we reflected on Amanda's comment about the intended audience this morning, we realized that the report emphasizes recommendations over features. As an alternative, we can migrate the detailed recommendations/criticisms to a separate document for CFDE use. We could subsequently bulk up the features of the report and just have a short recommendations section in the NIH-facing report.

• Pulling out positives - We of course want to highlight what the portal does well, and we will increase the content in this regard, aligning it with the User Activities.

• We are of course also interested in any feedback on the general structure/content of the report.

Thank you, and looking forward to discussing this at the meeting later today.

[RLC-DCPPC]

Hi @dcfcjcheadle , @swandle06 . I like what you've done so far and suggest you go ahead with this format.

Yes, let's have some sort of summary of suggestions, maybe a table organized by role (persona) [that would simplify creating use cases based on the recommendations].

By all means, accentuate the positive(s) ;-)

Not sure I fully grok @ACharbonneau's comment. But I like the idea that this could be used by NIH as an outline of things they might ask us to implement. We might just exceed their expectations by going ahead without explicit direction (sort of a "great minds think alike" situation).

[dcfcjcheadle]

Hi @RLC-DCPPC,

@swandle06 and I have completed the draft of this report, located here: https://docs.google.com/document/d/1MjxZvnBKwZ7ohIhyrIEQFvWdS_AWtnRkn_9s9dUHUnc/edit#

We have a few outstanding comments for you to provide feedback on; specifically, the 'share and cite' links don't work, and we're wondering whether this is better information for a GitHub Issue rather than included in this report. We are happy to make any additional changes between now and Wednesday, but we wanted to get your feedback first.

We also discussed @fridayek looking it over from a technical writing perspective, if she has time.

Thanks and let us know what questions you have.

[fridayek]

I reviewed the report and made a few small changes. I thought it was clear and concise, so I have no questions or concerns.

Assigning to @RLC-DCPPC to resolve some existing comments and do a final pass before submitting to NIH.

[RLC-DCPPC]

Hi @RLC-DCPPC,

@swandle06 and I have completed the draft of this report, located here: https://docs.google.com/document/d/1MjxZvnBKwZ7ohIhyrIEQFvWdS_AWtnRkn_9s9dUHUnc/edit#

We have a few outstanding comments for you to provide feedback on; specifically, the 'share and cite' links don't work, and we're wondering whether this is better information for a GitHub Issue rather than included in this report. We are happy to make any additional changes between now and Wednesday, but we wanted to get your feedback first.

We also discussed @fridayek looking it over from a technical writing perspective, if she has time.

Thanks and let us know what questions you have.

Hi @dcfcjcheadle, @swandle06: it took me awhile to get to this. I'd like you to remove the reference to links that don't work, the URL for Cia's report, and the Glossary (I responded to all the comments (I Hope)). Do you have time today, so I can ship this today (or tomorrow, if necessary)? Thanks!

[dcfcjcheadle]

@RLC-DCPPC I'm starting work on it now

[dcfcjcheadle]

The comments on the broken links were split out into the following issue: #100. All other comments were addressed.

[RLC-DCPPC]

this is complete. Delivered as noted here.