Version 2 - Itemized plan / to-do list

[mathieuboudreau]

branch: v2

This issue is to centralize the action plan for revamping this book (version 2), which we are doing from scratch.

It's mostly a list for myself, which I'll update as I go along, but writing it out here for transparency, feedback, and to follow progress.

Create new branch

• [x] Branch out from master to "v2" branch
• [x] Delete old material
• [x] Create empty template Jupyter Book
• [x] Update Github actions to build gh-pages book when pushes happen on this branch

Upgrade all frameworks and dependencies to latest versions

• [x] Jupyter Book
• [x] Plotly
• [x] Numerical tools

Create simple data plot that's generalizable when subjects are added

(Use brain WM T1 dataset)

• [x] Write function that wrap around a Plotly scatter plot.
• [ ] Refactor function for a variable number of subjects/datapoints.
• [x] Centralize plot labels outside of function (to pass it to function).
• [x] Create option or function/method that configures default plot qualities (font size, line size, etc)
• [x] Load data.
• [ ] ...
• [ ] ?

(more items to be added)

[mathieuboudreau]

@jcohenadad I've made substantial progress, but there is still work to do, in particular with extending some of my new features to be compatible with the spine datasets.

Here are the major updates so far:

• Created a new book from template from the latest version of Jupyter Book, so all their current documentation are compatible with out book now.
• Created a Data class to handle downloading the datasets, selecting versions of datasets, and extracting values from the dataset that are needed/wanted to plot (this last point only works for brain data at the moment).
• Created a Plot class that takes in a Data object that wraps around the less appealing Plotly code and give an interface to set title names and other properties, display the Plotly plot, select tissues you want to plot, and change the display mode easily (i.e. displaying a Plotly graph in a Jupyter Book requires a different call than what is needed to view in a Jupyter Notebook).
  • The code in this was significantly refactored relative to @zelenkastiot's original implementation to remove redundancies (still more can be done) and hardcoded "magic numbers" (mostly done except for lists containing number of subjects).
  • Currently only implemented for the brain datasets, but now that it's been refactored significantly I hope to be able to reuse most of the code with some minor additions/modifications for the spine data.

Note that I've not documented my classes yet, as it's still a WIP and might continue to change significantly until it's compatible with the spinal cord datasets.

Also note that the implementation I've done so far only aims to reproduce @zelenkastiot's original figures but with the new code. Once the implementation is complete and reproducible for both datasets, then it should be much easier to change some plot options or how we display the data.

The user-end experience should be much better now too. To compare the old vs new UX, see the book here and compare the old implementation code ("Legacy" section, brain data) vs. my [new implementation] (https://courtois-neuromod.github.io/anat-processing-book/1-brain-data-new.html) ("New pages" section, brain data). Note: if you want to run this in a Jupyter Notebok and see the plots, you'll need to change fig_gm.display('jupyter-book', tissue = 'GM') to fig_gm.display('notebook', tissue = 'GM').

p.s. One last note: MyBinder's been giving me a lot of trouble this year - today, I wasn't able to spin up this notebook on there. I'm not sure if NeuroLibre's servers are more stable at the moment (we lost our server support person a few weeks ago), so just an FYI if you wanted to try out any of this code above, for the moment, you might need to clone locally and install dependencies.

[jcohenadad]

@mathieuboudreau Thank you very much for this push. This is exactly what I had in mind in terms of refactoring (ie, limiting duplication, cleaner 'code' display on the book).

As you said, we can tweak the plots a bit more later (eg: the 'dot' and 'dashed lines' on the figure legend are wrong, because it should say 'Individual' and 'Mean').

[jcohenadad]

I tried loading the notebook into MyBinder and got this:

[mathieuboudreau]

I tried loading the notebook into MyBinder and got this:

Yes I'm aware – I wrote about it in the p.s. at the end of my post above. MyBinder's been very flaky these recent months, I'm going to look into if I can use NeuroLibre's servers prior to submission there and if they're currently more stable, I don't recall.

[mathieuboudreau]

Update: I've integrated and refactored a bit the spinal cord cross sectional WM T1w/T2w plots. The only plots missing to be reproduced are one of the spinal GM T1w/T2w plots, and the spinal cord qMRI plot (MTR, T1, etc). This should be straightforward though.

After those are made, I'll remove the old "Kiril" section in the book and will merge it as a first version.

Note however, that because of how Kiril had originally customized the brain vs spinal plots, there remains some code duplication in my refactoring (some if brain else spinal sections), which I think could be simplified with some planning, but this leads me to my next point.

At this point @jcohenadad , I think it would be good to know what is the highest priority to do next. Would it be 1) simplify the code even further to that it is more flexible and easier to create plots with new/different data, or 2) to start polishing the front-end and leaving some of the "spaghetti code" (if brain, else spine) to a lower priority? It might be that while working on 2), I can incrementally work on some parts of 1), but my feeling is that 2) is higher priority now right? Are there any deadlines/goals we're aiming at? Maybe now could be a good time to have a meeting if you want to discuss further.

[mathieuboudreau]

Resolved in #43