[UX Test]: Investigate prominence and length of description

gtsueng commented 3 months ago

Issue Name

Investigate prominence and length of description

Test Purpose

To investigate the value of increasing the prominence of the description in the record view. ~To determine the ideal length of a description which would balance out the real estate of the description relative to its value~. Having a wall of text may be potentially be less useful to a user evaluating a record than having a short description. At what point is a description so long that it is no longer useful?

This test is to determine HOW to include the AI-generated summary in the record view. See https://github.com/NIAID-Data-Ecosystem/nde-crawlers/issues/159 for additional details on the purpose.

The Ask:

We would like to know which one the designs, NIAID team would like to see in Staging. The designs can be viewed in the Lyssna study here: https://app.lyssna.com/do/a484c01b55d1/7069, and the NIAID/ODSET team is welcome to provide feedback on the designs using the Lyssna platform.
Alternatively, if NIAID/ODSET prefers, we can also run the study on the Lyssna platform and collect results from users either via:
- convenience sampling (no cost, but takes longer to get participants),
- or via paid sampling (likely to cost $50-$100, but will take only a day to get enough participants).

Test Approach

See the test

Test Cost

Depends on sampling method. $0 (convenience sampling) - $100 (paid sampling)

Test Time

Depends on sampling method 1 day (paid sampling) - 1.5 weeks (convenience sampling)

Writeup URL

https://app.lyssna.com/do/a484c01b55d1/7069

Related WBS task

No response

For internal use only. Assignee, please select the status of this issue

[ ] Not yet started
[ ] In progress
[ ] Blocked
[ ] Will not address

Status Description

No response

Test status check list

[X] This test has yet to be discussed between NIAID, Scripps, Leidos
[x] This test has been discussed/reported between NIAID, Scripps, Leidos
[x] This test has been scoped out
[x] This test has been drafted in the platform in which it will be conducted
[x] This test has been conducted
[ ] The results of this test have been analyzed

leandrocollares commented 1 month ago

The Lyssna test to determine the best design for the AI-generated summary has been created: https://app.lyssna.com/do/a484c01b55d1/7069

gtsueng commented 1 month ago

Hi @harwickma, @lisa-mml, @rshabman, @sudvenk,

Per our discussion on 2024.09.24, we have generated a few mock-ups to display AI-generated summaries of datasets for records that have excessively lengthy descriptions.

The Ask:

We would like to know which one the designs, NIAID team would like to see in Staging. The designs can be viewed in the Lyssna study here: https://app.lyssna.com/do/a484c01b55d1/7069, and the NIAID/ODSET team is welcome to provide feedback on the designs using the Lyssna platform.
Alternatively, if NIAID/ODSET prefers, we can also run the study on the Lyssna platform and collect results from users either via:
- convenience sampling (no cost, but takes longer to get participants),
- or via paid sampling (likely to cost $50-$100, but will take only a day to get enough participants).

We are also in the process of augmenting this summary metadata to ClinEpiDB records. We selected ClinEpiDB for an example augmentation for the following reasons:

It will be easier to find and view a summary-augmented record if we augment by repository (allowing filtering by source)
ClinEpiDB is a small enough repository (<70 records), of which the majority have very long descriptions

We will move these augmented records to staging once the display of the record has been selected

gtsueng commented 1 month ago

We are running the Lyssna study ($50) to help inform the decision on the selection of the design to move forward

gtsueng commented 1 month ago

We have completed the Lyssna study and implemented the winning design to Staging

The results of the study will be posted to this issue next week.

gtsueng commented 4 weeks ago

Results of Study 1 (1 bottom design vs 3 top designs): https://docs.google.com/presentation/d/1J6qb1tHO5VP_zAXYTl8CXZvcgzUazWhbuI4uZ7GpMQU/edit#slide=id.g1f804f59111_0_3 14 votes for one of the 3 top designs, 11 votes for the bottom design. Some comments indicating confusion as to what the summary actually represents

Results of study 2 (1 top design, 1 bottom design): https://docs.google.com/presentation/d/15wZNVZQyl0ehNJxvkbF3JqBX7r4-ARWffC7xRmO-OGQ/edit#slide=id.g30a76808019_0_72

hartwickma commented 3 weeks ago

Hi @gtsueng, following up on discussion from 15Oct2024, please draft an email for the ClinEpiDB to provide an overview of the process/motivation to develop AI generated summaries, and ask for their input feedback on the results. NIAID will provide inut and the inquiry will be sent from the Ecosystem email. Thank you.

gtsueng commented 3 weeks ago

Hi @hartwickma , @sudvenk,

We are working on the draft email and will create a separate issue for it once it has been drafted.

NIAID-Data-Ecosystem / nde-portal