Open kevinvmware opened 4 years ago
hi @kevinvmware thanks for your feedack. We do try to ensure questions are covered in the materials, and receive feedback from everyone taking the assessment, and I havent recieved much feedback like this, but we also introduce new content regularly. It would be super helpful if next time you take an assessment if you could please note down which questions in particular or as much detail as you can, it would help tremendously. On your second bullet, I have received some similar feedback and will look to introduce improvements as we update and introduce new materials.
One thing I should note is, with these courses we are trying to find a happy medium between traditional formally structured elearning course and more informal live courses where, especially around agile/rapidly changing techs, the live instructor often uses their knowledge to deliver up-to-date info often on the fly verbally or via whiteboard, or sometimes in their own material but not following formally structured classes, to keep up with rapidly changing areas, live courses rely heavily on informal adaptations.
At the same time, elearning courses are typically more heavily structured and frequently lock content in between periodic updates or cohorts. This is not as much a dichotomy between elearning and live courses as it is a dichotomy between more rigorously structured formalized courses and more informal courses, where in the former a given instructor may teach based off of centrally controlled fixed content, and an informal course may have a degree of structure but requires an expert-level instructor with processes that allow them to update content directly as needed to address technology changes outside of the scope of formally structured periodic content updates.
The ModernApps Ninja program attempts to find a medium to remove layers of formality that typically exist between experts and students in structured elearning content, and to introduce increasingly agile and continuously updated lifecycle management for elearning courses.
This approach may require some user education since elearning content has historically been treated more like a waterfall versioning than anything continuous, and some user expectations may need reset, but I think for very compelling reasons that bring tremendous benefits to students.
In some of our other traditional elearning programs, we have thick layers of processes, project managers and dedicated content developers and the result is it can take months of meetings to produce an hour or two of content. With the ninja content, we dont want to sacrifice quality and so we have some QA in place to ensure a degree of quality, but we are also relying on agile and continuous integration capabilities such that we can take bug reports and address them on the fly and use crowdsourcing and work together as a community to deliver and maintain a greater quantity of up-to-date materials.
The intent is not to have the formality or same level of polish as a traditional program, but to have a greater breadth and variety of up-to-date content by using agile processes, crowdsourcing and community contribution. Much like you may expect from a live course on a cutting-edge subject where an expert instructor can easily adapt materials on the fly, we want to make it easy for experts in the community to adapt materials on the fly.
As it pertains to tests and certificates, formal certification programs usually have rigorously structured questions designed to meet a threshold of brand quality such that anyone who has a MCSx or VCx or CCIx has a certain gauntlet associated with it. However a certificate of completion for a course has an entirely different set of standards, its common to attend an informal course where there may be a simple informal test and certificate of completion for a given course. Currently on ModernApps Ninja, the only credentials offered are certificates of completion for courses, the only implication of these credentials is to say student x completed course y on date z, not to represent a branded certification program. We may introduce higher level credentials at some point, such that a student could attain some credential by completing some combination of individual courses in a learning path.
Sorry for all the detail, but hopefully this helps make sense or better understand what we have been trying to aim for.
Art, thanks for the thoughtful reply.
I totally understand the dynamic and agile content/delivery practices at ModernApps Ninja! Kudos! This is something I would do as well. I think it is marvelous that the team is trying to provide a more informal educational experience leveraging the expertise of the instructor.
IMO, as we all embrace this type of educational agility, we should attempt to ensure that our stories contain new elements that help us remember that content and self-assessment go hand-in-hand and both need to be updated. I'm a huge believer in TDD, so when I look to update the material, I tend to go through what exists at that moment in time, and then create the assessments based on the content. However, this implies that whenever content changes, the assessments need to be reviewed as well to keep everything in sync. So, my comments were more about adapting our stories and processes to remind us to check our assessments whenever we update content.
I think MAN is doing a great job and I want to make sure that my efforts integrate and enhance what we are both trying to accomplish.
Thanks @kevinvmware ... and sorry for all the length in my reply but I knew it was you and wanted to share some deeper insights as I know we are still getting started and most of the vision is not yet evident in the implementation, so eager to work together to identify the best ways to make the vision and implementation align as we build things out. Really great point about content updates needing to be checked against test questions, I am looking at making guided workflows for common administrative tasks that can incorporate these details into the workflow to minimize training and memorization requirements for process and practice guidance. I am thinking now would make a lot of sense to populate a service catalog with automated workflows for self-service content management as trying to communicate tips and process details for various content management tasks is a big challenge. I am getting caught up on some firedrills from being out last week but I will set something up later this week or next week as I hope to abuse as much of your time as you will give me to get your insight :)
Thank you @afewell ! No worries, sometimes we need to be verbose because it is hard to get all this in a picture! :-) I love the idea of automation of the process - hm, where have I heard that mantra before LOL.
Yes, please, I'd be very happy to contribute.
I have noticed several issues I'd like to bring to your attention.
In almost every self-assessment section there was a question that was not covered in the content that was delivered. In my opinion, this is inappropriate to ask a participant a question for which you have not provided coverage.
On several self-assessment questions, someone might be able to "intuit" the answer, particularly with the questions where one answer is "All of the above". The reason for this is that the question does not indicate how many possible answers might be correct as you do in other questions. This inconsistency is a tell! Secondly, the answers have radio buttons and if a participant clicked one answer and then did some thinking and selected a different answer, their second choice would unselect their first choice. This is also a tell! Thus, it might be reasonable to conclude that since there is only one correct answer, even though more than one answer appears correct, the obvious answer must be "All of the above". One way to fix this is to modify the question with a hint that says something like "Pick/select all the correct answers" (much like you do in many questions where you hint that participants need to select 3 answers) and having the answers have checkboxes instead of radio buttons.