Development of complementary exercises - local knowledge

[mjptak]

I'm opening this up to foster discussion and ideas of how to promote/manage/inspire/coordinate local exercise development using the Khan Academy Exercise framework. The most creative examples off of the KA servers for the moment are at www.daveconservatoire.org based in Great Britain. There are a number of others on the web translating the math exercises. However, these exercises are specific to music and make use of things like png and sound files. These can enhance the experience and potentially lower the bar for initial exercise development but some basic organization of the content files should perhaps be formulated. Promoting participant exercise (and video) development will give local teachers both a voice in and a direct attachment to the KA and KA Lite effort.

More prototype examples are available at www.we4dkids.com and hopefully at other sites shared by participants in this issue. My own creations were sparked by a desire to guide my own teenagers in their applied learning....a definite work in progress.

[ivanistheone]

There is a lot in /khan-exercises/utils/ which is VERY high quality tools for building exercises. I only know that because (1) I previously tried to work on an exercise framework and (2) I looked at the source code.

Could there be a "fork an exercise" type of question? (easily create an exercise similar to one you want to copy -- this is how I use

Another idea. Could there be a "new exercise wizard"?

1. What time of exercise is it? [ shows five siz exersies ]
2. What variables do you want to use? [ select them + dropbox for different rand functions ]
3. Then a nice editing interface shows up with problem, question and solution divs...
4. Tag with metadata and Save

[rjcorwin]

An "exercise wizard" is something I'm definitely interested in but it may be very difficult to write a form based wizard that is as powerful as just writing the HTML and Javascript code for the exercise. See the examples in the Khan Exercises repository. From all of the exercises I've looked at it looks like questions are generated dynamically, which is great for something like practicing a math concept, program the logic to change a couple variables in an equation and have the student figure out the answer, but this doesn't translate so well for testing things like comprehension of historical facts. I attempted to create a simple exercise that consisted of static multiple choice questions but I failed in my attempt. I'll try again sometime soon but until then I've posted in the Khan Exercises issue queue to see if anyone knows of an example of how to accomplish static multiple choice questions.

[jamalex]

Just a quick note to say: great to have this brainstorming here -- thanks!

Local content creation is going to be our next big push, after we finish internationalization and easy installation packages. There will be room for custom khan-exercises in this, as well as other types of content (such as multiple choice questions). Khan Academy is also currently working on what they're calling "static questions", which are essentially multiple choice questions -- this is specifically so that they can start including exercises and quizzing materials for their less-mathy content.

[mjptak]

As we await the formalization of how it will all work I suggest that those of us who have been tinkering away with the KA Exercise framework share a little bit of what we have done especially with the goal of coming to some common themes/question types and the types of metrics that could/should be pulled from them. I think this will especially be important as the code for the utiliities develops through github. To get the ball rolling I offer this with apologies to those outside of the US (I'm working on localizing this for Kenya and Tanzania (:))). http://www.we4dkids.com/khan-exercises/exercises/aaaa_presidents.html. or this.. http://www.we4dkids.com/khan-exercises/exercises/aaaa_final_topology_27a.html

[ivanistheone]

Perhaps the first step towards pushing this forward will be to think (realistically) who is most likely to contribute exercises to the ka-lite and upstream to khan-exercises.

Possible content creators:

1. Teachers
2. Parents of students (tech saavy?)
3. Students (higher grades)
4. General audience

In all of the above, except for the possible small cast of the hacker-parent, the current khan-exercises API might be too complicated to use as is. Well, there are docs, but I am not sure an average teacher/parent will be able to deal with this.

On the other hand we don't want to regress to offering only multiple choice questions and other basic type of questions (everyone can build those yes, but they suck!).

Here are some possible ways we can go forward with this.

1. One way would be to create a "simplified" API for khan-exercises which the average teacher or non-techie can understand. (This will require lots of work, and an will introduce an extra layer of abstraction.)
2. The new exercise wizard idea described above.
3. ?

I really like the fact that each khan-exercise is a self-contained .html file which can be copied around easily. This will definitely make things easier, but I think there are questions that we need to start thinking about.

• Is there a taxonomy of math "subjects" which we can use to "tag" exercises?
• How will auxiliary assets (images) be attached to the .html file? (base64 string at the bottom?)
• How will exercises be ranked for quality and shared between?
• How will exercises be shared (intra school and inter school)?

[mjptak]

Thanks for the post Ivan. All important things to think about and ones which I presume will evolve as this catches wind. One thing that also might be useful short of building volumes of exercises are some nice prototypes that can be adjusted to content. An example might be identifying the parts of the heart is in some way the same question type as identifying the countries in Africa or the parts of a combine. I have found Ben Eater's magic orange dot particularly useful for such.

Timelines are another useful construct and here to there may be a way to adapt an existing exercise (ordering negative numbers) or the like. I guess what I am trying to get at is some set of robust templates that get at "forms of question" but somehow go beyond the typical matching, multiple choice, fill in the blank, etc.

My own feeble efforts in this regard has been to begin with a simple rote question answer. Example, "The opposite of up is ___? " and then to later fill it in with distractors. I also have built exercises that tend to group things together to get more bang for the buck out of each exercise and also take advantage of the ordered randomness of the workings of the KA exercise engine.

At very least it is great to see a forum where we can collect examples besides my own at my development site www.we4dkids.com as well as the more functional www.daveconservatoire.org. I know there must be many more out there.

[beneater]

As @jamalex mentioned, we're working on a "static question" feature which will eventually replace the dynamic khan-exercises we know today. This will let non-technical question writers create content. Right now we're still planning to maintain tight editorial control over the content, but we're building a small team of professors and teachers that are starting to write more interesting questions that don't fit in the khan-exercise framework.

There will be a long transition period where we will support both. Initially we'll just have a different flavor of exercise (indistinguishable to users) that draws from a pool of static questions. Since questions will simply be written in a mix of markdown and LaTeX, they'll be much easier to translate.

Questions will be tagged with the learning standards they cover. Right now we're looking most closely at the Common Core State Standards for Mathematics and the more concrete Smarter Balanced Assessment Consortium task models. These align well with the new curriculum adopted by 48 states in the US to be implemented in 2 years. We plan to provide a superset of the curriculum, so we may add additional tags for areas that we cover that are not included in those standards.

A lot of this is up in the air at the moment. I'm happy to answer questions, but gotta caveat it by saying the details are subject to change. It should be much more concrete once we start shipping static questions in a month or two. It might be best to hold off on any big development efforts in this direction until you get a chance to see what we ship. I suspect you'll be able to leverage our APIs and data formats (and data!) for static questions.

We're also open-sourcing the editor and renderer for static questions, so you might find that useful too!

[rjcorwin]

I really like the fact that each khan-exercise is a self-contained .html file which can be copied around easily. @ivanistheone I like this as well. Strange fact, if you open up one of the exercise files in a WebKit browser it will fail to load due to WebKit's lockdown of AJAX calls when going over file://. Those files depend on the khan-site.html file, which is probably just used for templating.

I guess what I am trying to get at is some set of robust templates that get at "forms of question" but somehow go beyond the typical matching, multiple choice, fill in the blank, etc. @mjptak I've seen this technique used a couple of times and it seems to work well. Unfortunately no open source examples I can point to at the moment :(.

We're also open-sourcing the editor and renderer for static questions, so you might find that useful too! @beneater That's fantastic news!

[mjptak]

Great to hear from more and more involved! The downside of some of that HTML portability is that the answers are relatively easily discovered by looking at the script file though to quote an unnamed javascript savant...if the kids do that we have already won. I noticed during Stanford SEE's short experiment with the KA Exercise engine that it only takes a small tweak to just take an answer right or wrong and move on to the next question. This is one of the things that teachers often feel they need...a testing engine. While I expect such to be soon developed with the assessments coming out of Palo Alto I also surmise that many hope to retain the emphasis on teaching exercises as opposed to testing exercises.

[mjptak]

I'll chime here to also talk about both local "knowledge" delivery mechanisms as well as exercises or mechanisms to reinforce/test such knowledge. Videos I presume should be straightforward but might there also be a need to include more "podcast" or "soundcloud" type resources (with or without timed slides)? For now, looking at both OLE Owl project video as well as other stuff being done with the KA Exercise Framework, an ability to stage and deliver videos and KA exercises locally and regionally to learners would be a welcome addition anywhere that is using KA Lite.

[mjptak]

I have branched off of develop in my local repo to just prove the concept of local material delivery (I presume this will not be how it is to be done but it mostly works). I am still unclear when/if this will be made available ( I thought for the Natala release) but do have a number of exercises as demos built on my own dev site that I will slowly add to this branch.

https://github.com/mjptak/ka-lite/tree/ptakdevelop

[bcipolli]

@mjptak This type of thing will be done in our new platform, in the https://github.com/learningequality/phase-2 repo. If you could open a similar issue in that repo, and link to this discussion, that would be great!

[bcipolli]

Moved discussion to https://github.com/phase-2 repo, where we'll address issues of authoring local content. Thanks @mjptak !