Authoring questions

[betatim]

Not quite sure where to place this, so putting it here.

What is the current user story for authoring questions/ok-tests?

I think having an instructor version of the notebook that is then split into ok-tests and student notebook is nice. It would allow you to keep your questions/ok-tests together with the material. I can also imagine how that authoring tool would work.

One thing that I can't quite imagine: I set a question to create a plot (say). I have a standard set of ok-tests that I want to use for every question that produces a plot. Is there a x axis label? Is there a y axis label? Is there a legend? etc As a user I'd want to have a library (python file?) that contains all of them and then in my instructor notebook I somehow reference that ok-test.

Is discussing this here the right place? Should we prototype the authoring tool?

[choldgraf]

Interesting idea - I use a little tool I made called nbclean for this, though that's just for scraping out student answers as opposed to splitting them off into a separate file for OK tests. I wonder if a small modification to that library could let you collect tests for a notebook and then write them to a file...

[choldgraf]

also re: your point about defining your own functions, I think you could do this by defining these functions and passing them in initial_env to grade_notebook. Is that true @yuvipanda ?

[choldgraf]

one extra thought I had - it'd be pretty easy to add something to the nbclean remove_cell method that, instead of just removing cells that match some tag, instead "pops" them out into a list of cells. (e.g. cleaner.remove_cells(tag="TEST", pop=True) would return a list of all cells with that tag, and remove them from the original notebook). Then you could use that to quickly construct the necessary okgrade tests.

[yuvipanda]

I agree having 'instructor notebook' from which student notebooks and tests can be generated is nice.

It should exist outside of okgrade, however - as a notebook extension most likely. I have ideas on how it should be, but the key here should be that it doesn't matter - multiple implementations of this notebook extension should work for different people's use cases...

Is discussing this here the right place? Should we prototype the authoring tool?

yes, prototyping is the way to go :)

[betatim]

Roping in @lwasser: could you post a notebook (and link it here) that somehow looks like your dream instructor's notebook in terms of specifying tests/grading things?

Once there is a better idea (in my head) I will go off and create a prototype and maybe move discussions there.

[lwasser]

hey all!! sure - i can do that. nbgrader has the functionality that you can implement tests in an instructor notebook and then export to the student assignment notebook. However the problem with it is it's difficult to hide homework answers from the students for some of the tests that are run.

so @betatim i have several assignments with tests in them. however i'd like to be able to do other types of tests too like multiple choice questions etc. would you like me to include one of those here and then add the other things i'd like to see ? i can show you one i think today when we talk

[choldgraf]

So re: something like multiple choice questions, it'd be clunky but you could already do that with an ipywidget (or you could even hard-code it into a markdown file, e.g. show the options in the markdown, then the answer could be given be assigning one of the letters as a string to a variable called q2_answer or something)

[lwasser]

i didn't know that! thanks @choldgraf and then we could grade against the answer? how did you create that little animated gif so quickly?

[choldgraf]

https://getsharex.com/ :-)

[papajohn]

Here's a module for authoring assignments as notebooks and then converting them to okpy format:

https://github.com/okpy/jupyter-assignment

I think the issue can be closed now.