Chapter 11 Cleanup

[adowling2]

Fix headers (if not done already):

/Users/adowling/DowlingLab/Teaching/data-and-computing/notebooks/11-publish/02-Standard-Normal-Distribution.ipynb:80002: WARNING: Non-consecutive header level increase; 2 to 4 [myst.header]
/Users/adowling/DowlingLab/Teaching/data-and-computing/notebooks/11-publish/02-Standard-Normal-Distribution.ipynb:100002: WARNING: Non-consecutive header level increase; 2 to 4 [myst.header]

[E-Ream]

Chapter-11-Statistical-Inference

Things to Cleanup:

• [x] remove CBE Spring 2020 line
• [x] move load libraries cell below learning objectives
• [x] check header leveling
• [x] fix LaTeX formatting
• [x] check referencing other notebooks
• [x] replace scanned graphs
• [x] add remove-output tags
• [x] run notebook all the way through

Notebooks to Cleanup:

• [x] 01-Central-Limit-Theorem
• [x] 02-Standard-Normal-Distribution
• [x] 03-Confidence-Intervals
• [x] 04-Students-t-Distribution
• [x] 05-First-Hypothesis-Testing
• [x] 06-Hypothesis-Testing-Review-and-Type-I-Type-II-Errors
• [x] 07-Flavors-of-Hypothesis-Testing
• [x] 08-Statistical-Power-Basics
• [x] 09-Statistical-Power-in-Python
• [x] 10-Statistical-Power-Practice-Problems

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/01-Central-Limit-Theorem.html

• [x] Replace "Central Limit Theorem" header with "Introduction"
• [x] Promote headers one level
• [x] Move reference to the top of the page.
• [x] For links to the documentation, replace the URL with the word "document" hyperlinked.
• [x] Remove "CLT" from "Illustrative Example: Membrane Defects"

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/02-Standard-Normal-Distribution.html

• [x] Remove the Membrane example from the beginning and link to https://ndcbe.github.io/data-and-computing/notebooks/11-publish/01-Central-Limit-Theorem.html
• [x] Repalce the header "Standardized Normal Distribution" with "Definite"
• [x] Add subsection for "Exam Practice"
• [x] Add reference to section in Navidi (need to look up)

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/03-Confidence-Intervals.html

• [x] Replace "Confidence Interval" with "Definition and Formula" in header
• [x] Replace "Motivating Example: Catalyst Life" with "Motivation: Catalyst Life Example"
• [x] Remove "68-95-99.7 Rule" and "Confidence Interval Formula" subheaders, add a sentence to introduce/explain the content.
• [x] Replace "Catalyst Life Example" with "Example Calculation", promote to level 2 header

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/04-Students-t-Distribution.html

• [x] Remove "Student’s t-Distribution" level 2 header
• [x] Promote level 3 subheaders to level 2
• [x] Shorten subheader to "Important Assumptions"
• [x] Replace the Process 1 and Process 2 images with Python code. Use an auto-regressive 1 model.
• [x] Link to other notebooks that consider the catalyst example

For an AR1 model, assume for process 1:

r = 0.2
sigma = 0.1
yield[i] = (1-r)*a*sin(t*b) + normal(0,sigma) + r*yield[i-1]

You can adjust a and b to get reasonable values for yield between 0 and 1

For AR1 model, assume for process 2:

r = 0.2
sigma = 0.1
yield[i] = (1-r)*0.5 + normal(0,sigma) + r*yield[i-1]

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/05-First-Hypothesis-Testing.html#hypothesis-testing

• [x] Change title to "Hypothesis Testing Basics"
• [x] Move references to top of the notebook.
• [x] Replace "Hypothesis Testing" with "Main Idea", promote to level 2 header, remove "Main Idea:" from the paragraph, add link to flavors of hypothesis testing
• [x] Promote all headers one level. Want mainly level 2 headers. (Okay for steps to be level 3.)

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/06-Hypothesis-Testing-Review-and-Type-I-Type-II-Errors.html

• [x] Move review of hypothesis testing section to the prior notebook
• [x] Rename notebook "Type I and Type II Errors"
• [x] Remove "Type I and Type II Errors for Statistical Inference" subheader, move text into the next section
• [x] Promote level 3 headers one level

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/07-Flavors-of-Hypothesis-Testing.html

• [x] Move this to before Type I and Type II errors in the TOC
• [x] In the introduction, clarify the section references in the table are for Navidi (2015)

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/08-Statistical-Power-Basics.html

• [x] Add reference to Navidi section to the top of the page
• [x] Replace "Statistical Power" subheader with "Definition"
• [x] Promote "Motivating Example" to level 2 header
• [x] Promote steps to level 2 header
• [ ] Draw step 2 and step 3 plots in Python, hide code in webpage using cell tags
• [x] Promote Question 1 and 2 to level 2 header

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/09-Statistical-Power-in-Python.html

• [x] Split the "meat" of the notebook into "Documentation and References" (with the links) and "Example Revisited" (with the code)
• [ ] Include the plot in the website. (Just need to commit the executed notebook?)

[adowling2]

https://ndcbe.github.io/data-and-computing/notebooks/11-publish/10-Statistical-Power-Practice-Problems.html

• [x] Promote all the subheaders (except learning objectives) one level. This means each practice problem will be level 2
• [x] Promote given data and parts 1 - 6 in problem C to level 3 subheaders
• [x] Give the practice problems meaningful names. Such as "Conceptual Understanding" and "Statistical Power Calculations"

[E-Ream]

Moved comments from adowling2 for Chap 12 to the Chap 12 Issue.

[jkeane14]

@adowling2 Hi Prof Dowling, just wanted to confirm that I should be working on further edits for this chapter from the Chapter-11-Cleanup branch from this point on

[adowling2]

Yes, just make sure you pull updates from the sender onto your local computer. Before starting, confirm the notebooks look good (e.g., contain your prior edits). Post here is you have any questions or encounter anything strange.

[jkeane14]

Yes, just make sure you pull updates from the sender onto your local computer. Before starting, confirm the notebooks look good (e.g., contain your prior edits). Post here is you have any questions or encounter anything strange.

It seems that all of the chapter 11 notebooks no longer have my prior edits

[adowling2]

@jkeane14 I'll investigate this more. I thought the script I ran merged them. Standby. I might not get to this until after 3 pm today.

[adowling2]

Posting here mainly for my future reference:

• I originally resolved the merge conflicts with nbdev. That overwrote changes from @jkeane14
• I am now trying nbdime to compare/manually merge the notebooks.
• This post was helpful when installing the dependency node.js.

[adowling2]

@E-Ream Do not change any of the Chapter 11 notebooks (besides creating the bootstrap one). I am in the process of recovering edits from @jkeane14. I'll update here when I am done.

[E-Ream]

I'm tracking on that, just created the bootstrap notebook today but it shouldn't have edited any of the other notebooks.

[adowling2]

Great! We also have two live branches for chapter 11 edits. Not a big issue, just something we need to be careful about.

[adowling2]

@jkeane14 I recovered your notebooks and added them to this branch:

https://github.com/ndcbe/data-and-computing/tree/Chapter-11-Statistical-Inference/notebooks/11

You'll see the 11 folder is a little cluttered with notebooks. I am differencing them now. I'll report what I find.

[jkeane14]

@adowling2 Great! Will I still be making edits to Chapter-11-Cleanup or will I now be working on the Chapter-11-Statistical-Inference branch?

[adowling2]

Here are the results in the comparison across versions:

• 01, 02, 03, 05 only metadata changed
• 04 looks like changes were made to versions with and without ' in the file name. This will likely need to be merged manually.
• 06-Flavors, 07-Type-I, 08, 09, 10 only meta-data changed

[adowling2]

@jkeane14 Which of your changes did you not see propagated in the merge from Friday evening? After restoring the prior versions of your notebooks, I am not seeing any changes except for 04 (see prior post). I think this means the merge on Friday went okay.

Please use the Chapter-11-Statistical-Inference branch. That has the restored notebooks with -JK added to the file name. You'll need to pull from origin to see these files on your computer.

[adowling2]

After you are done with the edits, let's remove the files with -JK in the name.

[adowling2]

Just to be clear, @jkeane14 is using the Chapter-11-Statistical-Inference branch which is linked to PR #69. Meanwhile, @E-Ream is using the Chapter-11-Cleanup branch which is linked to PR #73. The latter adds one notebook only. These two branches should not have any conflicts.

[adowling2]

I think we got this sorted out!

[jkeane14]

Looks good on my end. I will finish up edits on Chapter-11-Statistical-Inference branch. Thanks for figuring this all out.