Psy2R: Developing an R package for better inference in multivariate statistical analysis

[garner-code]

Title

Psy2R: Developing an R package for better inference in multivariate statistical analysis

Leaders

Kelly Garner Github: @kel-github Mattermost: @kels

Collaborators

Kevin Bird,
Melanie Gleitzman,
Sonny Li,
Christopher Nolan Github: @crnolan Mattermost: @cnolan

Brainhack Global 2023 Event

Brainhack Australasia

Project Description

We consistently use massive data sets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics.

Multivariate statistical analysis typically follows a two stage procedure, an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances this leads to a drastically inflated rate of type 1 error. It is even less well known that this procedure can also lead to an even lessor known type IV error (incorrect interpretation of a correctly rejected hypothesis)!

It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interest. This approach provides interval inferences on effect sizes and it also ensures that the familywise type 1 error rate associated with directional inferences (the inferences usually derived from tests of null hypotheses) is controlled at alpha. One piece of software (PSY) can produce SCIs appropriate for planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows.

Our goal is to build an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses!

Link to project repository/sources

TBC

Goals for Brainhack Global

Milestones

1. Implement analyses that can be done in PSY in R [intermediate]
2. Convert R implementation into generalisable functions [intermediate/advanced]
3. Outline the math involved in computing critical values for STP tests under the null hypothesis [advanced]
4. Interpret the PASCAL code that underpins the computation of critical values in PSY [advanced]
5. Investigate the R world for a comprehensive list of packages that share some functionality with PSY [beginner]
6. Test R functions and implement own analysis [beginner]
7. Interpret what you think our R code does, to help us write more readable code [beginner]
8. Document the PSY software functions and write documentation for the new R functions [beginner]
9. Create project management board (e.g. trello) to keep us on track! [beginner]

Good first issues

1. Implement analyses that can be done in PSY in R

2. Document existing PSY software functions

3. Investigate the R world for a comprehensive list of packages that share some functionality with PSY

4. Create project management board (e.g. trello) to keep us on track!

5. Create github repo for the project

Communication channels

https://mattermost.brainhack.org/brainhack/channels/psy2r

Skills

• R: all levels
• stats/math: all levels
• github: all levels
• comp sci: intermediate/advanced

Onboarding documentation

No response

What will participants learn?

You'll learn more about statistical analysis of big datasets, collaborative coding using R and github, and hopefully a bit about package development and coding for other users.

Data to use

No response

Number of collaborators

more

Credit to collaborators

Project contributors will be listed on the github repository's README.md

Image

Type

coding_methods, data_management, documentation, method_development

Development status

0_concept_no_content

Topic

reproducible_scientific_methods, statistical_modelling

Tools

other

Programming language

C++, documentation, R, other

Modalities

behavioral, DWI, ECOG, EEG, eye_tracking, fMRI, MEG, MRI, PET

Git skills

1_commit_push

Anything else?

No response

Things to do after the project is submitted and ready to review.

• [X] Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
• [X] Twitter-sized summary of your project pitch.

[garner-code]

Hi @brainhackorg/project-monitors my project is ready!

[crnolan]

Looks good.

[garner-code]

Hi @brainhackorg/project-monitors - I have edited my project, please can you push it to the website?

[Remi-Gau]

Reopening to be able to "list it" in the brainhack jupyter book Planning to update this page: http://brainhack.org/brainhack_jupyter_book/projects.html