Check behavioral labeling

[KirstensGitHub]

Check the compilation and labeling of all behavioral data

[KirstensGitHub]

WHAT WE'RE CHECKING:

These tests check to make sure that the behavioral data is compiled correctly, and that when the memory images are assigned an attention level from viewing, the attention levels are assigned accurately.

INITIAL CHECKS (complete):

This involved reviewing the two functions used to compile and organize / label the data.

• Check A: check_behavioral_data.pdf

• • First, we load the raw files from the participants' directories and compile them into dataframes. We ind that for both experiments, after looping over all of the participant directories, we loaded data from 30 unique subjects per experiment. Additionally, we find that 8 unique runs were loaded for each subject, with 8 unique memory runs and 8 unique presentation runs per subject.
    
    

• Check B1: check_behavioral_data.pdf

• • The second function labels the images seen during the memory runs by the attention they received during initial viewing. This function had an error and was using the first cued category from each presentation block as the cued category for each trial in the block (true for experiment 1, but not for experiment 2). We updated it to dynamically pull the cued category from each trial in the presentation block.
• • After compiling and labeling the memory images by attention level (compile_behavioral_data.pdf) with our new function (see comments, below), we checked the proportion of attention labels to verify that half (20) of the images in each memory run were labeled Novel, 1/8th (5) were labeled Full, 1/8th (5) were labeled Side, 1/8th (5) were labeled Category, 1/8th (5) were labeled None (Check B1a). We also checked that half of the novel images were faces and half were places (Check B1b), and that, for experiment 1, Full and Category images from each trial were of the same image category, and Side and None images from each trial were of the same image category (see the hand annotated data from Experiment 1 in the comments below).
    
    

• Check B2: check_behavioral_data.pdf

• • We additionally randomly selected single experiment runs from two random participants in experiment 1, and from two random participants in experiment 2, and matched each of the memory images to its presentation trial manually, to verify the attention levels

EXP 1:

EXP 2:

SUMMARY OF CHANGES (complete):

• updated attention level labeling function to select trial-wise category cues
• commented each line in attention level labeling fucntion
• re-compiled and re-labeled behavioral data from both experiments

FINAL CHECKS (in progress):

• verified correct proportions of each image type shown after labeling (Novel, Full, Side, Category, None, etc) (complete)

• selected a run at random from each experiment and hand checked the attention level labels of the stimuli (complete)

[KirstensGitHub]

BEHAVIORAL DATA ANALYSIS FUNCTIONS

Functions to Aggregate Subject Data and Verify Correct Stimuli were Presented

def sum_pd(subdir): ''' input: subject directory (string) output: full experiment info (dataframe) '''

# list all files in the subject's directory that contain pres or mem run data
files = [ x for x in os.listdir(subdir) if 'pres' in x or 'mem' in x ]

# read in the data from each of these files
df_list = [ pd.read_csv(subdir+'/'+x) for x in files ]

# concatenate into a single dataframe
df = pd.concat(df_list, ignore_index=True)

return(df)

def add_level(df): ''' input: subject dataframe output: subject dataframe w/ Attention Level string for each Memory trial row '''

for each run

for x in df.Run.unique():

    # select the data only from that run
    mask = df['Run']==x

    # pass the data for that run through run_level to add attention levels to memory images
    df[mask] = run_level(df[mask])

return(df)

def run_level(df): ''' input: df containing pres and mem from single run output: df with string in 'Attention Level' column in each Memory trial row '''

# loop over the trials in this run
for index,row in df[df['Trial Type']=='Memory'].iterrows():

    # obtain the image presented in the memory run
    mem_image = row['Memory Image']

    # obtain the category of the image from its filename
    # (SUN database images -- places -- contain the string 'sun')
    if 'sun' in mem_image:
        mem_image_category = 'Place'
    else:
        mem_image_category = 'Face'

    # add the image category to the memory trial row
    df['Category'][index] = mem_image_category

    # look in the columns for previously presented composites
    for composite in ['Cued Composite', 'Uncued Composite']:

        # if one of the previously seen composites contains the memory image file name (minus the last 4 chars: '.jpg')
        if df[df[composite].str.contains(mem_image[:-4], na=False)].shape[0]!=0:

            # pull the cued category from that row/presentation trial
            cued_cat = df[df[composite].str.contains(mem_image[:-4], na=False)]['Cued Category'].item()

            # if the image category matches the cued category for the presentation trial
            if mem_image_category == cued_cat:

                # AND the image was presented on the Cued Side
                if composite == 'Cued Composite':
                    # label "Full" attention
                    attention = "Full"

                # AND it was presented on the Uncued Side
                elif composite == 'Uncued Composite':
                    # label "Category" attention
                    attention = "Category"

            # else, if the image category does NOT match the cued category for the trial
            else:
                # AND it was presented in the Cued location
                if composite == 'Cued Composite':
                    # labbel "Cued" attention
                    attention = "Side"

                # AND the image was presented in the Uncued location
                elif composite == 'Uncued Composite':
                    # label "None" attention
                    attention = "None"

            # add the attention level to the memory trial row
            df['Attention Level'][index] = attention

mem_mask = df['Trial Type']=='Memory'
df.loc[mem_mask,'Attention Level'] = df.loc[mem_mask,'Attention Level'].fillna('Novel')

return(df)