Agenda 12/3/2021

[F-said]

Updates

• Pre-release + progressions on first release complete. Good to pre-release?
• Named tuples as default param constants
• Removed redundant test for template_params
• Input list as baseline for segment \
• can select subjects using int
• Pipeline skips data properly (before set_ref) if specified
• Removed error message constants that don't repeat
• metrics for tasks extracted from 48 participants, unsure of how many participants exactly

Blocks/Challenges

• sme creates empty df

Next Actions

• 394

• 359

• 406

• for running data through made, and other preprocessing tools, I think it should be done through another repo that provides the framework for testing multiple pipelines. Otherwise, pepper gets too large in scope and in definition
• Is pepper supposed to be used for data quality metrics? Maybe?

[F-said]

@georgebuzzell copying and pasting from 11/5

discuss idea of upcoming Pepper retreat/hackathon to get us to pre-release. And, a second one to get us to v.01 release. George is paying for pizza and Mountain Dew!

Review/discuss the updated scope of PEPPER: Not a "pipeline" but a "framework for community-driven, reproducible EEG analysis across age/context/system" -modular, reproducible, "living pipeline" at center -EEG4p to facilitate rapid benchmarking and identification of parameter sets for different ages/contexts/systems -Intentionally designed infrastructure for collaborative development and use. To include standards for how to ensure reproducibility (container-based tests), and checks on code development (required prs, automatic tests run at commit), as well as consideration given to governance, credit/authorship, licensing.

Discussion of timeline for pre/full release(s) and what to include.

1. We NEED to get a prerelease out SOON (November), and the minimal requirements are: -fully tested docker, singularity -multiple runs of data, with sensible results that at least look "OK" -proofread of existing documentation -white paper that describes the project. validation results are not needed for the white paper, and EEG4p does not need to be fully implemented/tested and/or part of the pre-release, but the design needs to be fully fleshed out. *Unless we include an alpha version of EEG4p and some decent initial validation results, I would suggest the pre-release is not advertised. It can be sent to a few collaborators, but ideally, it would be kept mainly underground for now.

-Timeline for release .01 and minimal requirements: -all of pre-release, but also need to demonstrate that validation metrics produce at least comparable results to Matlab-MADE. -ideally, an alpha version of EEG4p is also included, and this should be a minimal additional requirement, given that a primary requirement of the .01 release is demonstrating validation results via metrics. I.e., since we have to calculate these, might as well put in a slight bit more work to set up in a sustainable/scalable fashion. Focus should be on implementing structure well for a limited set of metrics, as opposed to quick and dirty computation of many metrics -White paper must be revised/updated, and crucially, must now include results of validation metrics and description of eeg4p as well -revise/update documentation on github, and, initial standalone website for EEG4p that is linked to from NDCLab wiki (BUT, pepper is its own thing and independent from the NDCLab).

Organization for bEEG4p -file positions/names of reliability.py, postprocessing.py and run_metrics.py -discuss whether it is viable to have standard input/output for "measures" and "metrics/stats" -where to place benchmarking datasets

Sonya comments on what is/is not intuitive about use of github, etc.

Discussion of results of 100-sub run of pipeline (errors, compute time, etc) Discussion of ERPs computed on 100-sub run Discussion of reliability metrics computed on 100-sub run; SME too??

Next steps and reliable timeline.

Recruitment needs: -coding -eeg knowledge -documentation overhaul -usability/design -writing/marketing -website design