PsyR: An R package for better inference in multivariate statistical analysis

garner-code commented 3 weeks ago

Title

Leaders

Kelly Garner (Mattermost: kels)

Collaborators

Christopher Nolan (Mattermost: cnolan)

Brainhack Global 2024 Event

Brainhack Aus

Project Description

We consistently use massive data sets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics.

Multivariate statistical analysis typically follows a two stage procedure, an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances this leads to a drastically inflated rate of type 1 error. It is even less well known that this procedure can also lead to an even lessor known type IV error (incorrect interpretation of a correctly rejected hypothesis)!

It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interest. This approach provides interval inferences on effect sizes and it also ensures that the familywise type 1 error rate associated with directional inferences (the inferences usually derived from tests of null hypotheses) is controlled at alpha. One piece of software (PSY) can produce SCIs appropriate for planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows.

We have been building an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses! We are ready for you to play with some of the functions to analyse your own data, and create Quarto documents that document your workflow, to help future users. If you are feeling hacky enough, we could also use help developing further the R package, which is around 80% finished right now.

Link to project repository/sources

https://github.com/kel-github/PSY2R - this has all the code we used to get started. I'll share the package code at the event https://doi.org/10.1177/0013164402062002001 - this is the paper that defines the functionality of Psy. https://r-pkgs.org/ - Hadley Wickham's bible for building R packages

Goals for Brainhack Global

Bring your own data and use the functions we have already written. This will help us learn how user friendly they are and what can be done to improve interaction with the function. Document own data analysis in quarto documents to help future users Contribute to the R package, particularly with higher level functions that handle data wrangling and standardisation Build a GitHub website for the package

Good first issues

issue one: read [this paper](https://doi.org/10.1177/0013164402062002001 to get an overview of what Psy does)
issue two: find a dataset that requires an ANOVA analysis, share it with the team and we'll talk you through how to use Psy to analyse it
issue three: If not already, get comfortable with quarto documents - https://quarto.org/
issue four: Go through the exercises that show you how to create an R package in chapter 1 of this book.

Communication channels

https://mattermost.brainhack.org/brainhack/channels/psy2r

Skills

R: basic and advanced
Stats knowledge: can run an ANOVA and know what a confidence interval is
Github: can clone and make pull requests (if needed, we'll show you how)

Onboarding documentation

No response

What will participants learn?

Participants will learn about multivariate data analysis:

how to compute confidence intervals for between x within effects, ideally on their own data
what goes under the hood of an R package

Data to use

No response

Number of collaborators

4

Credit to collaborators

Project contributors will be listed in the project ReadMe, and as authors on any help documents

Image

Leave this text if you don't have an image yet.

Type

coding_methods, documentation, pipeline_development

Development status

1_basic structure

Topic

statistical_modelling

Tools

other

Programming language

R

Modalities

behavioral, eye_tracking

Git skills

1_commit_push, 2_branches_PRs

Anything else?

No response

Things to do after the project is submitted and ready to review.

[X] Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
[X] Twitter-sized summary of your project pitch.

crnolan commented 3 weeks ago

Your Mattermost handles are being interpreted as github handles on here...

garner-code commented 3 weeks ago

I changed it :)

crnolan commented 3 weeks ago

You ticked the boxes without making the post :) I assume this is ready?

garner-code commented 3 weeks ago

Sorry! Yes, it's ready. Thanks!

On Fri, Nov 1, 2024, 17:45 Christopher Nolan @.***> wrote:

You ticked the boxes without making the post :) I assume this is ready?

— Reply to this email directly, view it on GitHub https://github.com/brainhackorg/global2024/issues/12#issuecomment-2451389926, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXC34YG4MDFWQVRUNXLYMDZ6MPP3AVCNFSM6AAAAABQ7MQ66WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJRGM4DSOJSGY . You are receiving this because you authored the thread.Message ID: @.***>

brainhackorg / global2024