How to implement the concept of "attempt"

[sfraysse]

Hi, Many thanks for the recent clarification of the meaning of registration. This was very usefull. This leads me to a question: how can we implement the concept of attempt as it was defined in SCORM 2004.

Let me give an example:

• The learner enrols to a course. The same registration will be used for the related statements.
• There is a Quiz in this course. When the learner launches the Quiz, a sessionid is defined and used for all the statements of the related session.
• Now, the learner suspends the Quiz and come back later in order to continue. Same registration. Different sessionid. Same attempt.
• Then, the learner fails the Quiz and wants to retry. Same registration. Different sessionid. Different attempt.

So registration, session and attempt are 3 different and complementary concepts.

Correct me if I am wrong, but CMI5 does not cover the concept of attempt. Does it make sense to add an attempt extension in the context as there is a sessionid extension?

[MrBillMcDonald]

Sébastien,

I believe your are correct. The notion of an "attempt" is not specified in cmi5 statements. The concept of an attempt (as discussed in the spec) is ""judged activity" (like a quiz) within an AU.

The cmi5 working group has not met the past 2 weeks due to vacation schedules. We will hopefully be able to discuss your issue next week (Aug 30th).

Bill McDonald

[sfraysse]

Thanks Bill for your feedback,

Yes, "judged activity" perfectly describes this use case.

Regarding the "completed" verb, the spec says:

The AU MUST NOT issue multiple statements with "Completed" for the same AU within a given AU session or course registration for a given learner.

which is fine. Maybe something similar for the "Passed" and "Failed" verbs could be usefull. For example:

The AU MAY issue multiple statements with "Passed/Failed" for the same AU within a given course registration for a given learner. In this case, the statement MAY contain a "http://........" extension in its context in order to specify an attempt number.

There is the http://id.tincanapi.com/extension/attempt-id extension on the ADL Vocab Server which seems to be fine for this use case.

But I don't know if it is a good practice to add such an extension in CMI5 compliant statements.

Sebastien

[brianjmiller]

Since this is a specification the group tried to stay away from adding language that wasn't specifically setting out requirements or restrictions. Most statements that would include a "MAY" are generally not needed because anything that isn't explicitly restricted or explicitly required is already a "MAY". So the rules around the verbs are that there can't be more than one passed, and that a failed can't follow a passed. This means that you may have multiple failures. Additionally the specification leaves it open that additional elements of a statement can be used so long as they don't collide with the ones defined in the specification and otherwise are allowed by xAPI. So there doesn't need to be additional language around whether a specific extension can be included. (See https://github.com/AICC/CMI-5_Spec_Current/issues/432 for more (there are others).)

All of these types of things become a slippery slope into either allowing/denying specifics around a concept like attempts (or really any other commonly used thing, i.e. assessments, other particular profiles, etc.) and are better left out of the specification document itself.

[liveaspankaj]

I am not an expert in CMI5, because I have not been following the recent developments. However, I have some understanding because I had influenced some changes in restrictions around completed verb in CMI5, because it was conflicting with xAPI Video Profile, and would conflict with any other sub-activity happening inside CMI5 framework that wanted to send a passed/completed verb.

Based on my understanding. You cannot have multiple attempts (with completion) for a single registration. Because that would mean your registration+object id+actor combination is the same for multiple completions. And, as far as I know, CMI5 doesn't allow that.

You might want to change the registration value everytime a passed or failed is received.

For the session, CMI5 already has a session id.

Regards, Pankaj Agrawal

On Mon, Aug 26, 2019 at 8:00 PM Brian J. Miller notifications@github.com wrote:

Since this is a specification the group tried to stay away from adding language that wasn't specifically setting out requirements or restrictions. Most statements that would include a "MAY" are generally not needed because anything that isn't explicitly restricted or explicitly required is already a "MAY". So the rules around the verbs are that there can't be more than one passed, and that a failed can't follow a passed. This means that you may have multiple failures. Additionally the specification leaves it open that additional elements of a statement can be used so long as they don't collide with the ones defined in the specification and otherwise are allowed by xAPI. So there doesn't need to be additional language around whether a specific extension can be included. (See #432 https://github.com/AICC/CMI-5_Spec_Current/issues/432 for more (there are others).)

All of these types of things become a slippery slope into either allowing/denying specifics around a concept like attempts (or really any other commonly used thing, i.e. assessments, other particular profiles, etc.) and are better left out of the specification document itself.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/AICC/CMI-5_Spec_Current/issues/597?email_source=notifications&email_token=AAN73GJQE2RUZNX632O7G73QGPSHDA5CNFSM4IL4APVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5ERLIA#issuecomment-524883360, or mute the thread https://github.com/notifications/unsubscribe-auth/AAN73GLGXO6TVKLIXFRIST3QGPSHDANCNFSM4IL4APVA .

[sfraysse]

Thank you @brianjmiller. I understand this is not a specification issue but a how to use the spec for a specific use case issue. Thanks @liveaspankaj, your answer is very clear.

So...

Use Case 1

Allow multiple "attempts" on a Quiz (a single AU) and close the access as soon as the learner passed the quiz (which means possibly multiple failed, but only one passed).

We are in a classic CMI5 scenario and the LMS should generate a single registration for all the "attempts" because they are under the same "enrolment".

Use Case 2

Allow multiple "attempts" on a Quiz (a single AU) and allow the learner to improve its score after a success (which means possibly multiple failed and multiple passed in any order).

The LMS should generate a new registration for each "attempt" because CMI5 accepts only one passed per registration and does not accept a failed statement after a passed statement.

Feeling

These are 2 very common use cases and the only difference may be a simple setting on your LMS. Something like "Allow new attempts after success".

This leads to different uses of the registration property, depending of the use case, and this may be a bit confusing.

My feeling is that the suggested approach for Use Case 2 looks like a workaround to the passed/failed restriction at the registration level.

But I am OK with that if you confirm that it is the best (or only) way to implement the Use Case 2 with CMI5.

[cawerkenthin]

@liveaspankaj I do not believe you can change a registration. The registration is created by the LMS when a student registers (enrolls) in the course. It does not change across AU. The purpose of the registration is to be able to query across the entire course for a complete picture of the student's activity. Changing the registration defeats that purpose.

@sfraysse You can use cmi5 allowed statements to send as many passed/failed as you wish. Each "cmi5 allowed" failed statement could indicate an attempt. The spec only has restrictions on use of passed/failed when the statement is "cmi5 defined". Or, as @brianjmiller noted, there is no restriction on extensions.

[MrBillMcDonald]

Sébastien,

The cmi5 working group discussed this and recommends that you use "cmi5 allowed" statements to indicate your attempts. A cmi5 allowed statement can use the cmi5 verbs (failed, etc) without restrictions.

What makes a statement cmi5 "allowed" is the absence of the the cmi5 category activity (see Section 9.6.2.1 of the spec)

Pankaj,

The group also discussed your response and you are incorrect about registration.

You might want to change the registration value every time a passed or failed is received.

Registration = Course + Learner (an enrollment instance). The registration ID should be created when the student is enrolled in the course and is immutable.

Registration: An enrollment instance of a learner in a course. (a registration ID uniquely identifies this). The registration ID persists throughout the course progress to completion and during review of a completed course. A new registration is created for new enrollment instances (such as recurrent courses or re-taking courses

This is not compliant unless you are doing multiple enrollments in a course.

Bill

[liveaspankaj]

Hi Bill,

I probably chose wrong wording to explain what you have very clearly explained about Registration: "An enrollment instance of a learner in a course. (a registration ID uniquely identifies this). The registration ID persists throughout the course progress to completion and during review of a completed course. A new registration is created for new enrollment instances (such as recurrent courses or re-taking courses)"

Assuming, multiple re-takes are allowed. The LMS could decide that once a passed or failed is received. the current registration (enrollment) is complete. The LMS could allow the user to retake the course with a new registration (new enrollment) The LMS could also store all old registration values, to allow the user to review each of his old enrollments.

Would that be a valid CMI5 workflow?

Regards, Pankaj Agrawal

On Fri, Aug 30, 2019 at 8:53 PM Bill McDonald notifications@github.com wrote:

Sébastien,

The cmi5 working group discussed this and recommends that you use "cmi5 allowed" statements to indicate your attempts. A cmi5 allowed statement can use the cmi5 verbs (failed, etc) without restrictions.

What makes a statement cmi5 "allowed" is the absence of the the cmi5 category activity (see Section 9.6.2.1 of the spec)

Pankaj,

The group also discussed your response and you are incorrect about registration.

You might want to change the registration value every time a passed or failed is received.

Registration = Course + Learner (an enrollment instance). The registration ID should be created when the student is enrolled in the course and is immutable.

Registration: An enrollment instance of a learner in a course. (a registration ID uniquely identifies this). The registration ID persists throughout the course progress to completion and during review of a completed course. A new registration is created for new enrollment instances (such as recurrent courses or re-taking courses

This is not compliant unless you are doing multiple enrollments in a course.

Bill

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AICC/CMI-5_Spec_Current/issues/597?email_source=notifications&email_token=AAN73GPZ6AYYZZLRTQMG4W3QHE3QZA5CNFSM4IL4APVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5R67WA#issuecomment-526643160, or mute the thread https://github.com/notifications/unsubscribe-auth/AAN73GORQD27SFHVJH5ZOFTQHE3QZANCNFSM4IL4APVA .

[liveaspankaj]

@MrBillMcDonald I understand that the ticket is closed, however, it might be helpful for everyone if you can confirm if my last response is correct.

[cawerkenthin]

@liveaspankaj Bill is out of the office for a few weeks. My opinon: a) The LMS can allow a student to re-register for a course. That is an implementation detail for the LMS. For example, my LMS must allow students to take a course annually for compliance with regulation. b) If creating a new registration is done simply for the purpose of creating an "attempts" concept for an AU, this seems like a waste. We have indicate a method for tracking attempts for an AU using cmi5-allowed statements. Also, the idea of a new registration because you fail one AU seems to ignore the concept of multi-AU courses. It also seems to break the idea of tracking everything a student has done in a course instance via the registration ID.

[liveaspankaj]

@cawerkenthin Thanks for the response. I will dig deeper when we implement this. I agree that in a multi-AU scenario it will get complicated so sticking to the Registration ID will be important at least till everything is completed. So, using an LMS generated attempt-id extension. And using CMI-allowed statements might be the best way to go.

Pankaj

[sfraysse]

@liveaspankaj I will also adopt this solution because I think it is important to use the registration properly, without changing its meaning, even for a single AU package. This is important from a data analytics perspective.

If I had to suggest something for a future version of the CMI5 spec, it would be to allow multiple passed and failed statements in any order for the same AU and registration.

The LMS would consider the AU as "passed" as soon as there is a passed statement, but not necessarily the last one. This is not a breaking change and this could improve the CMI5 flexibility to implement more use cases.

To go further, the moveOn property could also support more options, for example:

• OnePassed: one attempt must be passed
• LastPassed: the last attempt must be passed
• AllPassed: all attempts must be passed
• FirstPassed: the first attempt must be passed