johnwdubois / rezonator

Rezonator: Dynamics of human engagement
35 stars 2 forks source link

Rezonator Experiment web app (with Firebase) #1402

Open johnwdubois opened 1 year ago

johnwdubois commented 1 year ago
  1. Create a "Rezonator Experiment" web app, designed to be used by:
    • Experimenters (= experiment designers)
    • Subjects/workers
  2. The "Experiment design" button pops up a screen (or a dialogue) which allows the experimenter to design/create/edit an experiment.
  3. The screen or dialogue box allows the experimenter to specify:
    • Firebase Project Id
    • what crowd-worker data to collect (e.g. tracks & trails, or intonation unit boundary splits - see examples below)
    • timestamps for all crowd-worker actions
  4. Allow the experimenter to show a series of "Trials", and show each one on its own screen, in a specified order (see examples below).
    • 1 screen = 1 Firebase document
  5. Once the set of tasks (HIT) is complete, give the crowd-worker a secure payment token, which they can use to get paid (e.g. by Amazon Turk, Prolific, etc.).
  6. Let the experimenter download from Firebase the data that were created by crowd-workers, based on the Project ID they are using.
  7. Let the experimenter manage the data collection process on the web, by integrating the Rezonator Experiment web app with resources such as:
    • Firebase (database capture and storage)
    • Prolific, Mechanical Turk, PsycoPi, etc. (for managing tasks and workers)
    • Itch.io (for server functions)
    • (or other web-based solutions)

Examples

  1. Experiment design should allow the experimenter to show a series of screens to workers, such as:
    • experiment instructions (at the beginning)
    • trial 1 (e.g. .rez file with 10 lines of text, plus the corresponding audio excerpt)
    • trial 2 (a different small .rez file, plus audio)
    • trial 3 (a different small .rez file, plus audio)
    • follow-up survey questions (e.g. with sliders, and/or radio buttons)
    • screen to collect secure payment token for payment (at the end)
  2. Co-reference experiment:
    • Show a brief section of conversation
    • Allow workers to mark tracks, trails, chunks, and zeroes
    • Collect this data, with timestamps for all user actions
  3. Intonation unit segmentation experiment:
    • Show one conversational turn at a time, with the corresponding audio
    • Allow workers to split and merge turns
    • Iterate over several trials (1 trial = 1 turn)
    • Collect this data, with timestamps for all user actions
johnwdubois commented 1 year ago

Some important tweaks:

  1. Stop using the word "List" in names that the researcher sees -- it's too vague. Instead:
    • UnitList > UnitSequence
    • TrialList > TrialOrder
    • etc.
  2. Simplify:
    • "Unit Splits" > "Split"
    • "ParticipantGroup" > "Condition"
  3. Let the Experimenter sort (query?) their results in at least a few ways:
    • Results > Participant > Trial
    • Results > Condition > Participant > Trial
    • Results > Trial > Participant
johnwdubois commented 1 year ago