Created HW3 and T3

[lucieyang1]

I uploaded the student-facing version of HW3 and the T3 tutorial slides. I also updated the README with links to both the homework and the tutorial (but I didn't know how to make it launch directly in jupyterhub). There's also a bunch of images I used in the homework/tutorial.

Hopefully I am doing this pull request correctly?

[mistryrohan]

Will be reviewing and this and adding comments. Can you also add in the tester code please.

[mistryrohan]

The homework and tutorial were very well formatted! On the student facing versions I made some changes to the comments, capitalization, and fixed some grammatical issues. The tester solutions work, but for Q0 and Q4 I suggest not giving away the answer as the hint, rather something that points to the solution.

I will send you the files I edited on Slack since it does not like me attach notebooks here and then I believe we should close this pull request and then you can create a new one with the edits and then Professor Schwartz can approve and merge it (will @ him here to see if he wants it done differently).

@pointOfive

[lucieyang1]

I've made all the changes and I think it should be good for review! I accidentally forgot to save and undid some changes, but I think it should all be fixed now.

[pointOfive]

I have some texty-style commentary request for updates/adds to the tutorial slides, which, are already very good; but, it's just that I've already had in mind a couple content pieces that I'd like to see added there. So here are my comments for that, and let's see if you can pull off the changes initially on your own:

It would be / is a fantastically good idea to have the TAs discussion about normal distributions… I’d like to expose the students to this topic as we make use of it in hypothesis testing and may again do so for regression.

• Perhaps add the note ‘Opening introductory conversation (5 minutes): “normal distributions”?’

Do you have access to the following? https://q.utoronto.ca/courses/277998/quizzes/279235/take?preview=1 It’s the week 2 practice quiz from last years class that you took. Would you see if you can find some ways to incorporate Questions 6-11 and 14 to the quiz (assuming you don’t think something there is redundant)? [You might need to add some slides for the the figures? Not sure how this might work… perhaps it’s not part of the quiz but more of a group activity that students then answer on their quiz?] Will you also add the following questions (if they’re not redundant):

• “What’s the difference between a histogram and a barplot?” (And show the brand barplot and one of the histograms from the homework.
• What’s the difference between mean/median and what makes them different?

On the “second order characteristics” slide, please add some sort of discussion note like the following:

• IQR is fairly simple to understand (right?); and standard deviation also usually has the easy interpretation (especially for symmetric unimodal data) that most (or all) of the data is within 2 (or 3) standard deviations to the left and to the right of the mean.

Would you incorporate the “Which do you like better?” slides (towards the end of my week 2 lecture from last year https://github.com/pointOfive/STA130_Week2_Slides/blob/main/STA130_F22_Week2_slides.pdf) as a quick little fun activity for tutorial? I’d like the TAs to point out that boxplots can be misleading for multimodal data but that they also potentially allow us compare a lot of distributions against each other efficiently in a single plot. Then add a reference to problem Q17 (I think it was) in the homework and have students discuss weather they like boxplots/histograms better (and add a picture showing KDEs as well to discuss if that’s further preferable to boxplots/histograms, as the Q17 in the homework sought to get students to think about. Finally, the TAs should go to https://plotly.com/python/violin/ to show off the attractive looking violin plot from plotly. I think this should be a 10-15 minute activity depending on if you think it works best as a small group activity or a full class activity.

• I see that you’ve added something along these lines to the discussion: perhaps that is enough; but, I still think having the TA lead the kind of class discussion I suggest above as a follow up to the discussion would be a good idea

So, with all these changes I think the time requirements might now looking like: Slide 1: 5 minutes Slides 2-5: 15-minutes still Slide 6 (quiz): let’s make this 20 minutes but then Slide 7 (discussion): drop this down to 15 minutes Slide 8-9: [20->15 minutes] Discussion activity looks great… I also think that showing these figures will allow students who weren’t able to start the tutorial assignment to get some ideas for what types of figures they could make)

• Let’s add my proposed “which do you like better” discussion (figures can just be screen captured) for an additional 10 minutes after the above? Slide 10: Hedging 10 minutes

That makes 90 minutes So that leaves 20 minutes to start the Tutorial Assignment

[lucieyang1]

I will take a look! Unfortunately, I do not have access to the Week 2 Quercus Quiz (it says quiz results are protected)

[pointOfive]

Okay -- no worries -- ignore that part for now then and I'll see about adding it after you revisions.

[lucieyang1]

HW3 and TUT3 should be updated with the changes now!

[pointOfive]

Just adding a comment here documenting changes that happened to the tutorial, as I just made them but have not put any of our offline conversation about it here.

Essentially, my proposals above were made; but, upon another review I felt there was room to flush out the content/topics to the tutorial, and did so.

• This wasn't "new" content at this point, but added and flushed out definitions, presentations, images, etc. which makes this more "slides/presentations" as opposed to outlines. Essentially, I decided we can make a tighter script for the TAs to follow more universally.
• This is an extremely large jump up in amount and complexity of content compared to week 2 tutorial, which was the template that was initially given and followed (very well); so, I will continue to think about if this is "too much" so tutorials should be somewhere between the week 2 and week 3 tutorials, or if this indeed does seem like a manageable set of slides for tutorial.

I then as well re-thought about the best way to use all the available questions in a tutorial setting, and settled on a shorter quiz, but then more "full class" discussions based on TAs asking questions to everybody and trying to get engagement through raising hands or class consensus style activities.

• There is a "tutorial activity" (essentially a participation/engagement score) that needs to be extremely easy for the TA to mark; basically, just checking if it was turned in; and, then there's a "tutorial assignment", which is turned in before the following Monday and which the TAs should spend a little more time grading and giving some feedback on.
• The "tutorial activity" here is then a short quiz that will be easy to collect and enter participation marks for.
• The "tutorial assignment" here involves making and describing some plots in the coffee_ratings.csv; so, these can be read and reviewed by the TAs
• There is also another "tutorial activity" which involves a group discussion. This might be overkill; but, for now I guess my plan is that both "tutorial activities" will be quickly noted by the TA for marking purposes.

[pointOfive]

I am finalizing some smaller edits on the HW and TUT, to make sure everything is well-integrated.

Additionally, I have adjusted some of the time demands on the TUT as the group/class quiz discussion and conversations parts as I think they did not realistically provide/estimate enough time for a smooth transition through the slides

• I am finding that even though the tutorial slides are complex and dense, that I think they are good, and that I think they are a good use and organization of tut time.
• That is, this tutorial should/will be an introduction and exploration/conversation/discussion about these topics/terms/concepts lead/disseminated by the TA to the students directly. I don't expect to lecture extensively on these topics in class lecture
• Specifically, all the location/scale terms that are introduced would have only been introduced briefly as aggregation functions in the previous week, and in this week three I would have only demonstrated plotting and likely said very little about skew and modality... I also would not have likely dwelled on location/scale topics in my in class lecture comments.

[pointOfive]

I want to make a clear record here noting that I have removed all discussion of "heavy tailed" distributions from the tutorial content (and it does not appear in the homework).

• I have mentioned this a little bit in my recorded lecture; but, thinking about it, I think this is too advanced of a concept to raise in this course. If I do choose to comment on it in class, I will likely refer to it as a more advanced concept, since "heavy tailed" could be confusing as I do not think the colloquial understanding of the word really lines up with the meaning of the statistical term.
• I.e., I don't think a casual visual sense of "heavy tailed" really lines up with the statistical behaviour of heavy tailed distributions which is perhaps much more "long and thin" and essentially actually just means there's a potential for crazy outliers of the black swan event variety.

[pointOfive]

I am going to make a point to have the TAs emphasize "outliers" in the tutorial content...

• This really wasn't done in the current incarnation materialized so far
• And, actually, this seems like a good/better way to begin moving towards a topic of "heavy tailed-ness"; and, actually, probably/definitely simply a better content choice for student consideration at this stage.

[pointOfive]

Having looked at the content amount and time requirements again, while I think the intention/outline/plan/material generally in this tutorial is in great shape, it is indeed going to be a little too much as currently envisioned; so, ...

• the hedging video doesn't really fit in here... we're going to push it to the next week with confidence intervals... it makes more sense there
• I'm chasing the format of what was the "second" group activity, and I'm changing it from a small group discussion into a full class conversation. This will allow the idea/content to be introduced and covered in a more efficient manner.
• I do think it's important/good content to include in this tutorial, but since we already have the quiz activity, we don't need another; and; thus, it seems this tutorial will not have a group component; but/so, hopefully the TAs will be able to facilitate effective class-wide discussions
• I don't want to drop the formatting of the practice and quiz questions... I think it's important to introduce these topics in this manner through this approach... especially since it will be likely be the primary learning/introduction location for these topics/terms/ideas.

[pointOfive]

Have also removed the follow-up on "what's your favourite"? Histograms? Boxplots? KDEs?

All of these things I've removed I initially asked for, so this is just a question of my reviewing critically and trying to be sensible about what can actually and reasonably be included into a tutorial...

• Truth was, it looks like I was cramming... trying to do more than is really reasonable in the amount of time available for tutorial

[pointOfive]

I have an idea for how make the general discussion a little more "group oriented" and I am going to implement that :)

• I think it's a good idea and will work well to move way from the "full class" discussion style that currently exists

[pointOfive]

I have ended up removing the full class discussion (about comparing boxplots/histograms counts information and modes, and then comparing them versus KDEs) which I had initially requested. [This content is not "wasted" as I expect it can be nicely repurposed into the kinds of questions I like to ask on exams, so I will likely use it for that :)].

• We did spend a descent amount of time adding that material in; but, now that I'm really trying to look at the time requirements carefully again with fresh eyes, it's just too much stuff/content/activity to cram into the tutorial
• I'm deciding that we need to target about 100 minutes of content, allowing for a 10-minute late start to tutorial, and the possibility/allowability that the TAs may (or may not) give a 10-minute or so break in the middle
• So, the time requirements as now listed total to <92-minutes; so, that leaves about 10 minutes left over to introduce the "tutorial assignment"... even like this the TAs could still be a little pressed to get to introducing the tutorial assignment; but, they will to manage there time effectively to do as this will be the demand/requirement of the teaching/leading/mentoring aspect of this role

[pointOfive]

Edits for tester file:

• I noticed that the testing cells get printed out and give away the answer when sometimes it would be better not to do so so that the students can think about the hints a little bit more: I've made the necessary changes so please look into how you can do that!
  • I didn't think this needed to be done for all of the testing file cells; but, definitely some, so I changed just those!
• superbowl_ads_csv_asdf = pd.read_csv("superbowl_ads.csv") instead of d = so that students won't accidentally name their imported file the same thing... perhaps this wouldn't cause the testing to fail, but this way there's really no chance of overlapping/overwriting naming, which generally would be something we'd want to look to avoid.
  • actually, a problem with the tester file later was that df was used instead of d: I think this worked for you because of this issue: df was probably defined in the student file and so the tester succeeded for your homework submission because it defined df in the student submission file; whereas, I named by df something different, so the tester file error'd for me because df was never defined... only d in the tester file (which then went on to accidentally use df in the assert statement tests.
  • changed df to superbowl_ads_csv_asdf three times for Q4
  • changed d to superbowl_ads_csv_asdf three times for Q8
  • changed df to superbowl_ads_csv_asdf three times for Q12
• Actually, the tester file didn't work for me because it was missing scipy which caused a warning error in the MarkUs output for the distplot/KDE plot: certainly, this is the kind of error I'd prefer was caught in what gets sent to me: I want to make sure everything is running perfectly and has no issues so that there are no unexpected surprises that cause fires which need to be put out during the semester when we're actually live with all this stuff!
• Your Q0 hint was better than the one I wrote as a draft of something I wanted said for the tester file that I wrote before looking at your tester file: awesome job with your hints: they've all been super excellent and really fun and engaging helping!
  • I have moved the "hint" variable into another cell above... when a hint is long and somewhat complex, I think it's better to do this so that the text that's outputted in MarkUs is a little more streamlined and less redundant (which could help make it less initially overwhelming)
  • I did this for Q1 as well (and some of the later questions below) but I'll stop noting them all here explicitly
  • I have added a little bit of language to your hints per my sensibilities; but, again, this is just because all the hints were extremely excellent, and easy to engage with (so I did in my usual over-editing manner)
• For automatically failing MarkUs autotests, I have included an advisory header information notice indicating that these "failing" tests do not count against the student.
• Note also that I've updated some of these answers to include the example code solution as well, as I think this is also worth giving the students; since, as MarkUs autotesting output can't provide an actual visualization as an output the code will have to suffice as help for the students.
• Q9 should have been 'A' rather than 'B', right?
• I've change stylization of "bar plots" to "barplots" as I like that better; and, that's also how I write "boxplots". Just a preference thing, but trying to be consistent to give this all the most polished feel possible.
  • This actually remind me that I'd really like the stylizations of hints and comments and notes or whatever to be as consistent as possible throughout the different homeworks (to again increase the sense of uniformity through good consistency that makes for a "polished final product" kind of feel that keeps everything very professional and excellent feeling for the students.
  • Please do try to help me enforce this in the materials that you're working with (yours and others) as you're able to!
• All edits to the student facing file noted below were propagated into their corresponding cells in the tester file so there's not any weird misagreements between the contents of the files that might lead to the desired edits being lost

Edits for student facing hw:

• Outstanding catch on the ddof difference between numpy and pandas: I've updated the hint in Q8 to extend the conversation of the hint a little bit more (as well as perhaps subliminally "hint" as well that documentation is available and excellent as a reference for particulars and details!)
• I've change stylization of "bar plots" to "barplots" as I like that better; and, that's also how I write "boxplots". Just a preference thing, but trying to be consistent to give this all the most polished feel possible. And I've also just noticed we've stylized "scatterplots" as well, which is perfect for this again in terms of consistency. Arbitrary, but, consistent ;)
  • I've added a comment into Q13 noting the distinction between barplots an histograms.
• Q15 labels should be automatic, so I've removed “, specifying meaningful axis labels where appropriate”; and, I've gone back to the celebrity stylization for referring to that column/variable... which is really how I'd prefer to do things for consistency even though I'm the one who initially removed that stylization because of how it made the text lines overlap... but, they're not doing that now and it's definitely better this way.
  • Changed the hints a bit for this problem so they're actually more generally correct
  • Improved the hints for Q16 as well
• Reworked the tutorial prompt so it aligns/coincides/integrates with the prompt in tutorial, i.e., the text contents now exactly match between the tutorial notebook and the "head start encouragement" in the homework notebook.

[Bortoise]

Overall the homework took me a bit under 2 hours, a good length.

In Q15 you are asked to write an answer in the markdown cell: there is no question prompting a written answer.

Other than that the homework looks great! Very good hints and example answers!

[pointOfive]

Good Cole -- I see your MarkUs submission for 0teston June 22 about 20 minutes or so before your comment above. Your comments suggests so; but did you also submit a blank notebook to see how the MarkUs output looks when questions are all wrong?

In the future, I'd like you to make suggested/needed edits directly following: https://sta130build.slack.com/archives/C056VLG4NLD/p1687360001759389

Will you be able to commit edits to PRs in the future in this manner?

I will make your suggested edit above (if still needed, as I have some updated header information to include in the file based on your good considerations listed in your header of HW9).

[Bortoise]

Yes I will be able to commit edits directly to PRs, I just was not sure what the intended question should have been so I didn't want to add something incorrect.

[pointOfive]

Yep -- good catch -- I am making the edit now: I likely over-edited this question leading to the issue you found (and I'm very pleased you found it -- this is exactly what I'm looking to catch with all these reviews).