search

NIAID-Data-Ecosystem / nde-portal

Discovery platform to find NIAID-related datasets and tools
Apache License 2.0
3 stars 1 forks source link

[UX Test]: Resource Catalog Display Test #224

Closed EmilyHaag closed 7 months ago

EmilyHaag commented 7 months ago

Issue Name

Resource Catalog Display Test

Test Purpose

As discussed in https://github.com/NIAID-Data-Ecosystem/nde-portal/issues/218, the test objectives are to answer:

  1. Does the current design support users' ability to understand and distinguish between the different types of resources (datasets and resource catalogs)?
  2. Is "Resource Catalog" the best terminology for this type of resource or is there a better term?
  3. Should other ResourceCatalog-specific fields be displayed?
  4. What is user preference for coloring of the header bar?

Test Approach

The test will be conducted with 25 participants on Lyssna. Participants will be shown screenshots displaying the different types of results and will be asked the following questions to determine interpretability and preference: 1.1. There are 4 results in the image above. On a scale of 1-5, how easy is it to distinguish between the different types of results? 1.2. What are the different types of results? 1.3. On a scale of 1-5 how much do you agree with the statement: "I understand the difference between results that are 'Datasets' and results that are 'Resource Catalogs'." 1.4. On a scale of 1-5 how much do you agree with the statement: "The coloring helped me distinguish between the different types of results." 2.1. Select the best term for the following definition: "a centralized hub for data, resources, or information." (Options: Resource Catalog, Resource, Source, Other) 3.1. If you were using a data portal, would you want additional information about these resources displayed in this view? 3.2. If so, what additional information or fields would you want displayed?

Test Cost

$75

Test Time

The test is expected to take 2 days to set up, <1 day to collect results, and 1 day to analyze results.

Writeup URL

No response

Related WBS task

https://github.com/NIAID-Data-Ecosystem/nde-roadmap/issues/53

For internal use only. Assignee, please select the status of this issue

Status Description

No response

Test status check list

gtsueng commented 7 months ago

From the https://github.com/NIAID-Data-Ecosystem/niaid-feedback/issues/109:

To paraphrase an email from earlier this week. We recognize Lyssna is a way to capture feedback on the UI, relevance of search returns, etc. NIAID had some concerns with Lyssna since it’s unclear where the feedback is coming from in the community and how reliable the responses are given the small sample sizes. However, this concern is mitigated by the potential benefit of Lyssna and it's ability to rapidly capture feedback on a lot of information. Please use your judgement to leverage Lyssna to capture the appropriate feedback beyond the three initial search term, and update NIAID on progress in our upcoming calls.

From the https://github.com/NIAID-Data-Ecosystem/niaid-feedback/issues/106:

The process moving forward for Lyssna tests should be:

For each proposed test, create an issue and seek NIAID approval to proceed with the Lyssna test.

Based on the above comments, we are creating this issue to notify the NIAID team of our intention to conduct this test to help inform design details for the metadata completeness application to search results test.

gtsueng commented 7 months ago

@EmilyHaag as discussed in our internal meeting dated 2024.02.21, please proceed with this test.

EmilyHaag commented 7 months ago

Resource Catalog Display Test results can be found here