initial analysis notes

[andrewsu]

Notes

• Sample data were originally provided to us as 2023 Scripps DEI Survey_dataset to share with Scripps 7.10.23.xlsx, which has been converted to a plain TSV file in /data/sample_data.txt
• The three columns Department/Org Level 1, Division/Org Level 2, and Strategic Unit/Org Level 3 describe the levels of aggregation as we move up the org chart
• The columns between Q1:Gender Identity - Selected Choice and Q5:Citizenship status define the demographic groups. Reports for each one of these groups will be created when we meet the threshold to ensure anonymity only.
• The columns starting at Q6:How long have you been with Scripps Research? represent the survey data to be summarized in a report
• Columns with discrete, ordered values (e.g., Strongly agree, Agree, ... , Strongly Disagree) should be shown as stacked bars. The vast majority of data should be in this format.
• Some columns contain a comma-delimited list of discrete values (e.g., for Q13B:What methods do you use to communicate with your mentor? (Check all that apply) - Selected Choice, example value is In-person, one-on-one meetings,Group meetings,Email.) These data should be shown as a bar chart showing the percentage of respondents who selected each response.
• Some columns contain free-text answers (e.g., Q54_1_TEXT:Is there something that you experienced working with previous mentors that you wish was also done with your current mentor? - Yes (please explain): - Text). All answers should be presented in a simple text box.

To-Do

This section will be broken out into individual tickets.

• Ask The Mark to adjust the Org Chart columns for GRAD PROG - STUDENTS
• Confirm that The Mark will pre-screen all free-text responses to remove identifying information
• Andrew to adjust sample_data.txt to include more realistic counts for demographic groups to test inclusion/exclusion of reports as outlined in this analysis spreadsheet

[rjawesome]

I've set up the code to generate bar charts for single/multi select questions: example single select-

example multi select-

Let me know if I should make any changes to this format, or switch some from bar charts to other format like a pie chart. Also, I currently plan to add the percentages for each option in the legend.

Also, I was unsure how exactly to display the answers to the free text questions so please let me know on that.

Here is a PDF containing figures for all single/multi select questions in the sample data: All.pdf

[andrewsu]

This is looking really good!

Let me know if I should make any changes to this format, or switch some from bar charts to other format like a pie chart. Also, I currently plan to add the percentages for each option in the legend.

The bar charts I think are good. Just to make the visualization more concise, perhaps it could be adjusted to be something like this? I agree with adding exact numbers somewhere, either in the legend or as a label on the bar itself

Also, I was unsure how exactly to display the answers to the free text questions so please let me know on that.

Maybe something like this?

Here is a PDF containing figures for all single/multi select questions in the sample data: All.pdf

excellent start. Minor points I'm sure you've thought of...

• try to paginate and format to standard 8.5 x 11 sized pages
• preserve color scales across questions when the allowed responses are the same
• when appropriate, keep the order the same -- e.g., Q44m is an example where ordering by frequency ended up being not ideal

[rjawesome]

I have an updated PDF now:

• The items in legend are sorted alphabetically (therefore, if questions have same options the colors should remain consistent)
  • Certain sets of options (such as agree/disagree) are ordered from most agreement -> most disagreement
• The legend contains the count/percentage for each item
• Free text questions are included in the bottom of the PDF
• paginated to 8.5 x 11 pages

File: All.pdf

Also, @andrewsu when do you plan to update the sample data?

[andrewsu]

Looking good, thanks @rjawesome! A few notes:

• I'm working on better sample data today -- should have it complete tomorrow sometime
• Looks like the agree/disagree options changed -- compare Q10 to Q18
• please also order the "importance" answers eg Q32
• change numeric ordering eg Q59
• fix ordering in Q61, Q62, Q63

[rjawesome]

Looks like the agree/disagree options changed -- compare Q10 to Q18

Seems to be because Somewhat Agree changes to Agree -- I guess I could treat these as equal?

Also some of the other issues seem to be due to different casing, so I will see how to make my method case insenstive

[andrewsu]

Looks like the agree/disagree options changed -- compare Q10 to Q18

Seems to be because Somewhat Agree changes to Agree -- I guess I could treat these as equal?

Yes, I think that would be fine...

[andrewsu]

Okay, I've generated new sample survey data in sample_survey_data_20230814.xlsx. A few notes:

• I'm providing it in xlsx format because The Mark confirmed that will be the format in which the script will access that data.
• In this sample data file, the sample data is in the first sheet Data.
• The subsequent sheets L1 Reports, L2 Reports, L3 Reports, and Institute show what reports should be generated (and which should not) based on the sample data and our threshold to maintain anonymity. (These sheets will not be provided in the actual data file, and the analysis script should ignore these sheets.)
• There will be another update to the sample data file that shows which reports should be generated based on the Race/Ethnicity demographic data. But because that demographic data allowed for multiple selections, there are some details we first need to square away in terms of desired analyses. So stay tuned on this...

[rjawesome]

Here, it says a report should be generated for "ACADEMIC RESEARCH" specified to each gender. However, one of the respondents for academic research in this sample data responded "prefer not to say". Therefore, my program interpreted this as "prefer not to say" being below the cutoff, and not generating gender specific reports.

Is this logic correct, or should "prefer not to say" be treated differently.

[andrewsu]

Great question. Yes, I was treating "prefer not to say" and blank values as special cases that can be ignored for the purposes of setting cutoffs for sub-groups. If a respondent says "prefer not to say", then their response just doesn't factor into any report based on gender identity.

[rjawesome]

Updated PDF: All.pdf This is based on the new sample data.

(I haven't changed it to use excel sheets yet, so currently I manually converted to TSV)

[andrewsu]

enough of the initial requirements are done that I'm going to close this issue as complete.