Feedback on course specs/dev process

[ncarchedi]

@hugobowne commented on Thu Aug 10 2017

This is feedback on @gvwilson's doc on course specs & development process as of this commit;

cc @ncarchedi as he'll be interested.

General

This provoked a great deal of thought on my part and I think the potential shiftings of process and specs are exciting.

• Something to note is that the process proposed doesn't resemble our current process much at all; this is fine, but worth thinking about as it seems to be more a new way of doing course development here at DC, rather than a re-working (to be clear, I'm not against this);
• These specs and process seem, to be honest, intuitive, natural and great for both instructors and students; I presume they also arise from years and years of Greg thinking about, reading literature on and practicing and honing such curriculum development techniques, which means they will be inevitably data-driven in more than one way; however, to play devil's advocate, if we're to make radical changes to the current specs/process, it would be good to see data or findings suggesting that they work; Can you point us to such resources? Perhaps this? Or any of Mark Guzdial's work?

I'll now provide feedback on each section. It's also tempting to provide feedback on the embedded specs for the unix course but I'll try to hold off.

Step 0: Learner profiles

• This is fantastic; the current approach to letting an instructor know who the students 'generally are' is skills-based and relatively ad hoc; in addition, I've had numerous instructors say, at various points in development, when I've said 'such and such material may be too advanced', for example, that they don't think they quite get who we're aiming the material at;
• Any specific hard-skill profile info.? e.g. 'knows what a group-by is'? I know that this is pretty fine-grained but it is a challenge we come up against all the time; thinking about re-structuring additional resources could help, e.g. doesn't matter if a given student doesn't know group-by, we can send them to a couple of exercises somewhere or a blog post

Step 1: Concept Map

I know I said I wouldn't provide feedback on the unix stuff but this unix concept map is a thing of beauty.

• This is great and useful for curriculum dev, thinking out most productive methods will be key, e.g. flow charts or directed acyclic graphs
• I agree mostly w/ resisting the assumption to writing out learning objectives at this point, however:
  • I think instructors and CLs will already be developing ideas of learning objectives in their heads while engaged in the process and, when multiple people are doing so, it may be worth jotting a few things down to make sure that we're not thinking at cross purposes (may be mitigated by concept map); i'm not suggesting anything extensive and set-in-stone, but rather some general notes;
  • learning objective have been a good way to communicate to the rest of the company and externally what courses contain or will contain; could a concept map take this place?

Step 2: Summative Assessment

• This is key and perhaps takes the role (and performs it better) of learning objectives.
• The focus on practical and exemplary examples of what the student will be able to do will be valuable for instructors, students, marketing and many more stake-holders

Step 3: Formative Assessment

• This is all reasonable so not so much critical feedback
• Only thing I'd point out is that, in the way the dev process currently works, I think this would be owned by Curriculum Lead as it will form part of the functional specs

Step 4: Sequencing

• the same feedback as Step 4

Step 5: Chapters

This is also the point at which it's worth writing formal learning objectives. Doing it earlier often wastes effort, since material may be cut in Step 4.

• As stated above, while doing all the above steps, it's not much wasted effort to jot down several notes to make sure that everybody is aligned (nothing formal); thoughts?

Step 6: Course Description

Brilliant that this now flows directly out of the work that's already been done. The unix course description is likely one of the best I've read: how did you write it before all the other steps were complete! ;)

[gvwilson]

@hugobowne I believe the major points have been incorporated - can you please confirm, and if so, close this issue?

[hugobowne]

I'll get to this by EOW at latest

[hugobowne]

is this where I should be looking? https://github.com/datacamp/courses-intro-to-unix-shell/blob/master/README.md

It has been fleshed out incredibly well.

It doesn't look as though the following points from the OP have been incorporated:

• These specs and process seem, to be honest, intuitive, natural and great for both instructors and students; I presume they also arise from years and years of Greg thinking about, reading literature on and practicing and honing such curriculum development techniques, which means they will be inevitably data-driven in more than one way; however, to play devil's advocate, if we're to make radical changes to the current specs/process, it would be good to see data or findings suggesting that they work; Can you point us to such resources? Perhaps this? Or any of Mark Guzdial's work?
• Any specific hard-skill profile info.? e.g. 'knows what a group-by is'? I know that this is pretty fine-grained but it is a challenge we come up against all the time; thinking about re-structuring additional resources could help, e.g. doesn't matter if a given student doesn't know group-by, we can send them to a couple of exercises somewhere or a blog post
• I agree mostly w/ resisting the assumption to writing out learning objectives at this point, -however: I think instructors and CLs will already be developing ideas of learning objectives in their heads while engaged in the process and, when multiple people are doing so, it may be worth jotting a few things down to make sure that we're not thinking at cross purposes (may be mitigated by concept map); i'm not suggesting anything extensive and set-in-stone, but rather some general notes;
  • learning objective have been a good way to communicate to the rest of the company and externally what courses contain or will contain; could a concept map take this place?
• Only thing I'd point out is that, in the way the dev process currently works, I think this would be owned by Curriculum Lead as it will form part of the functional specs (formative assessment)

[gvwilson]

@ncarchedi moved the example course spec to https://github.com/datacamp/example-course-specs - its README.md is the most up-to-date version. Looking at your comments:

1. I'm more familiar with descriptions of lesson design processes (like this one) than with data proving that they're better - I'll dig.
2. By "specific hard-skill profile info", do you mean in the learner profile or in the course pre-req description (or both)?
3. Yeah, it needs to be clearer that the process is iterative even though the document looks waterfall.
4. Yes, owned by CL.

[hugobowne]

should we move this issue to that repo?

1. cool! as stated, I'm playing devil's advocate; this processes seems wonderfully intuitive and modeled on how our brains/experiences work; also, you've been using and developing this process for years;
2. both;
3. +1
4. cool

[gvwilson]

:+1: to relocate (do you have ZenHub powers to do that?)

I'm iffy about listing hard skills: I worry that people will use e.g. the names of Unix commands rather than the ideas of moving and copying files. Try it and see what happens?

[hugobowne]

ohh yeah hard skills could be conceptual ( i would avoid using names of tech-specific commands also); i have no such zenhub powers