Create Masterclass *Managing Open & Reproducible Computational Projects* (second masterclass in the series) - Githubissues

alan-turing-institute / data-training-for-bioscience

Introduction to Data Science Project Management for Project Leaders.

Other

9 stars 1 forks source link

Create Masterclass Managing Open & Reproducible Computational Projects (second masterclass in the series) #18

Open fedenanni opened 2 years ago

fedenanni commented 2 years ago

Part 1: From raw data to computational analysis draft slides

[ ] decide big headings
[ ] fill in content
[ ] polish figures

LydiaFrance commented 2 years ago

Slideshow for this master class: https://docs.google.com/presentation/d/1FUPWPa8-j3HvA3djZpxe57lRkaJTxEhpx9UVZSU26gk/edit?usp=sharing

Proposal Slideshow: https://docs.google.com/presentation/d/1vwRLcj01DiPJ_OWqQzMb8PQxbQgv9dNwyj8jku02UUM/edit#slide=id.gf948759eaa_0_0

LydiaFrance commented 2 years ago

Trying out some big headings:

Raw Data

What does Raw Data mean?

Just a functional definition.

Directly from the instrument/measurement tool(?)
The first step/s of your data (handwritten to a spreadsheet? the output of your proprietary software? images?)
The input of your data pipeline

Did the data come from...

A microscope in your lab, or other instrument?
A collaborator?
An open data set?

If you're doing the data collection, what is your responsibility?

It can be easily lost how you went from raw data to processed data, especially if it is one PhD student.

Sometimes storage space can be an issue, especially with video/images.

Keeping track about...

Are there different versions of the data files?
Have the data files been updated and who by?
Are they backed up?
Meta information how the data files were created?
Can everyone access the data who needs it?
Should everyone be able to edit the raw files? Applying read only?

Is cloud computing an option?

Processing Code

Analysis Code

fedenanni commented 2 years ago

I spotted this which could be an interesting starting point for version control of data: https://towardsdatascience.com/are-you-still-not-using-version-control-for-data-d79a8b004c18

fedenanni commented 2 years ago

Overview on VM and Azure, which we could copy some snippets for the cloud computing part: https://docs.google.com/presentation/d/1kB5rNFfpaEyruWUfT4A4iJ0he5mbSRNrM49RQlRXSBo/edit?usp=sharing

malvikasharan commented 2 years ago

Content structure for the masterclass:

Title: Managing and Supervising Computational Projects

Motivation: Starting with Scientific Process
Setting up a computational project
Method selection
Method Implementation
Data Analysis and Results
Publication and Release

See details: https://docs.google.com/document/d/11r3iwxt3TocH8zGutx7bwxdASnpnY62_Mi8wkHdBTQ0/edit#heading=h.voi27s2sey73