Progress extension in statements that do not use the AU activity id

[thomasturrell]

Is it valid to use the progress extension in statements for activity id's that differ to the activity id of the AU and if so what is the meaning?

For example how should I interpret the progress of the AU from the following statements? Note that the statements have different activity id's from the AU. The AU activity ID is http://example.org/cmi5/au-lms-id

Statement 1

{
   "verb":{
      "id":"http://adlnet.gov/expapi/verbs/progressed"
   },
   "object":{
      "id":"http://example.org/cmi5/au-lms-id/page-1"
   },
   "result":{
      "extensions":{
         "https://w3id.org/xapi/cmi5/result/extensions/progress":25
      }
   }
}

Statement 2

{
   "verb":{
      "id":"http://adlnet.gov/expapi/verbs/progressed"
   },
   "object":{
      "id":"http://example.org/cmi5/au-lms-id/page-2"
   },
   "result":{
      "extensions":{
         "https://w3id.org/xapi/cmi5/result/extensions/progress":50
      }
   }
}

Please note the statements are abbreviated for readability.

[thomasturrell]

@MrBillMcDonald here is the real world example you asked for.

Statement 1

{
    "id": "f38913c5-ab72-49a2-a74c-a15ae8aed7c7",
    "timestamp": "2022-03-18T16:08:12.232Z",
    "actor": {
        "account": {
            "name": "Fred Bloggs",
            "homePage": "https://example.com"
        }
    },
    "verb": {
        "id": "http://adlnet.gov/expapi/verbs/experienced",
        "display": {
            "en-US": "experienced"
        }
    },
    "object": {
        "objectType": "Activity",
        "id": "learning.dev://au/9079d163-5cb1-4799-babb-c098d0740888/slide/0",
        "definition": {
            "name": {
                "en-US": "Introduction to Geology"
            }
        }
    },
    "context": {
        "contextActivities": {
            "grouping": [
                {
                    "id": "https://w3id.org/xapi/cmi5/catapult/lts/course/geology-intro-single-au-basic-responsive/1",
                    "definition": {
                        "name": {
                            "en-US": "Introduction to Geology - Responsive Style"
                        },
                        "description": {
                            "en-US": "This course will introduce you into the basics of geology. This includes subjects such as\n                plate tectonics, geological materials and the history of the Earth."
                        }
                    }
                }
            ]
        },
        "extensions": {
            "https://w3id.org/xapi/cmi5/context/extensions/sessionid": "5e9c0a70-09fa-4c1a-80b3-a2730b117385"
        },
        "registration": "a8e8d848-34d4-4931-b241-adf72b13e67c"
    },
    "result": {
        "extensions": {
            "https://w3id.org/xapi/cmi5/result/extensions/progress": 13
        }
    }
}

Statement 2

{
    "id": "cbe0718d-9e5b-4df3-ace6-0c63186d07a0",
    "timestamp": "2022-03-18T16:11:37.270Z",
    "actor": {
        "account": {
            "name": "Fred Bloggs",
            "homePage": "https://example.com"
        }
    },
    "verb": {
        "id": "http://adlnet.gov/expapi/verbs/experienced",
        "display": {
            "en-US": "experienced"
        }
    },
    "object": {
        "objectType": "Activity",
        "id": "learning.dev://au/9079d163-5cb1-4799-babb-c098d0740888/slide/1",
        "definition": {
            "name": {
                "en-US": "Geologic Materials"
            }
        }
    },
    "context": {
        "contextActivities": {
            "grouping": [
                {
                    "id": "https://w3id.org/xapi/cmi5/catapult/lts/course/geology-intro-single-au-basic-responsive/1",
                    "definition": {
                        "name": {
                            "en-US": "Introduction to Geology - Responsive Style"
                        },
                        "description": {
                            "en-US": "This course will introduce you into the basics of geology. This includes subjects such as\n                plate tectonics, geological materials and the history of the Earth."
                        }
                    }
                }
            ]
        },
        "extensions": {
            "https://w3id.org/xapi/cmi5/context/extensions/sessionid": "5e9c0a70-09fa-4c1a-80b3-a2730b117385"
        },
        "registration": "a8e8d848-34d4-4931-b241-adf72b13e67c"
    },
    "result": {
        "extensions": {
            "https://w3id.org/xapi/cmi5/result/extensions/progress": 25
        }
    }
}

Activity learning.dev://au/9079d163-5cb1-4799-babb-c098d0740888/slide/0 is at 13% and activity learning.dev://au/9079d163-5cb1-4799-babb-c098d0740888/slide/1 at 25%.

[phtyson]

Maybe a better verb would be http://adlnet.gov/expapi/verbs/progressed, with the AU as the object, and a context extension to indicate the page that was read or experienced. I agree the use of cmi5 "progress" result extension could be clarified in the spec. But it doesn't make sense to say that progress on slide0 is 13% and on slide1 is 25%. Don't you mean that overall progress is 13% after reading slide0, and 25% after reading slide1? I don't know the full story of the cmi5 "progress" extension. The meaning is clear enough, but there doesn't appear to be any cmi5-defined statement that can make good use of it, so it is up to each application to do something sensible.

[thomasturrell]

@phtyson thank you for your input.

Maybe a better verb would be http://adlnet.gov/expapi/verbs/progressed, with the AU as the object

I agree. It is impossible to retrieve statements from an LRS by statement extension. So I think it would be helpful if the progress extension was only used by a specific verb.

I think it would be valuable for an LMS to display the percentage of an AU that a user has completed. Many LMS's have this functionality.

From the example above, I understand statement 1 to mean:

Fred Bloggs experienced 13% of Introduction to Geology

And I understand statement 2 to mean:

Fred Bloggs experienced 25% of Geologic Materials

I would welcome the thoughts of @cawerkenthin @brianjmiller @gavbaa plus anyone else.

[cawerkenthin]

Per per the second sentence in section 9.4 of the spec, the activity id MUST be the AU in all statements except for Satisfied, which is sent by the LMS. I believe this to be a good thing; the AU, for example, can not send COMPLETED for AU's other than itself.

[thomasturrell]

the AU, for example, can not send COMPLETED for AU's other than itself.

@cawerkenthin I do not think that spec makes that clear (not saying that you are wrong).

From the spec 7.1.3 Types of Statements:

"cmi5 defined" - Statements using cmi5 defined verbs, category id as defined in section 9.6.2.1, and context template. "cmi5 allowed" - Statements using any verb and cmi5 context template (but NOT including cmi5 category id as defined in section 9.6.2.1).

I read 'any verb' to include cmi5 defined verbs. But I agree with you that could create an undesirable situation where one AU could complete or pass an another.

In the wild I see AU's sending passed, failed and completed statements for activity id's that are not the activity id of the AU. Some AU's have multiple and tests some AU's have multiple sections that need to be completed. This seems like desirable behaviour to me.

I am trying to understand if the progress extension is interoperable, I would say that it is not because there seems to be multiple interpretations of how it can be used.

[cawerkenthin]

I believe 9.4 is quite clear for cmi5 defined statements. For cmi5 allowed, the AU can do as it wishes as long as the statement conforms to the cmi5 allowed rules. Thus, it could send a COMPLETED for a test as a cmi5 allowed statement. The LMS would take no action (other than storing the statement) since it is not cmi5 defined.

If an AU is sending cmi5 defined statements that do not use the activity ID passed to it on the command line, then it is non-conformant.

[thomasturrell]

If an AU is sending cmi5 defined statements that do not use the activity ID passed to it on the command line

But isn't that the very definition of a cmi5 allowed statement?

In this context I understand command line to mean launching URL, is that correct?

[brianjmiller]

(Caveat: my tone here may be construed as authoritative, I don't intend it as such, this is opinion and based on things that could have been misinterpreted, misremembered, etc.)

Is it valid to use the progress extension in statements for activity id's that differ to the activity id of the AU and if so what is the meaning?

"Valid", yes, a good idea, probably not. The extension is designed for the cmi5 profile and therefore the statements that would be included in that profile and therefore the cmi5 defined statements. You'd likely be better off defining (or using) another profile with the cmi5 session wrapper and prescribe how progress is recorded for statements from that profile. For instance the video profile within a cmi5 session will track the concept of "progress" differently than the AU would necessarily be tracked.

For example how should I interpret the progress of the AU from the following statements? Note that the statements have different activity id's from the AU.

However you like, because they are not cmi5 defined then from this perspective they are completely outside of the scope of the cmi5 specification and should be interpreted in whatever manner was deemed by the author of the statements. (See why providing another profile within the cmi5 session would make sense.)

I agree the use of cmi5 "progress" result extension could be clarified in the spec. But it doesn't make sense to say that progress on slide0 is 13% and on slide1 is 25%. Don't you mean that overall progress is 13% after reading slide0, and 25% after reading slide1?

What is meant is the problem here. We can know what is meant when the statement is cmi5 defined and the progress extension is used, that means that that is the progress towards satisfaction of the AU for this registration. Any other meaning can't be determined and therefore are all potentially valid meanings. As far as whether it makes sense to say that progress on slide0 is 13% is entirely up to the person constructing the statement, that may well be exactly what they mean. Especially if you think of this in terms of items that feel less singular (i.e. a quiz, you've answered 2 of 4 questions so your progress on the quiz is 50%). Having said that a "slide" may well have animations on it, so it could be very valid that 13% of a slide has been viewed. I would (and have) advise people constructing statements to think of them as atomic data points, so in this case the progress recorded does need to be related to the object that is targeted and take the verb into account.

I read 'any verb' to include cmi5 defined verbs. But I agree with you that could create an undesirable situation where one AU could complete or pass an another.

This would be a really bad idea, but I think you'd find that it isn't possible because of the session mechanics, an AU sending a passed/completed, etc. statement for a different AU should not be accepted by a conformant LMS outside of the context of a specific launch session. So either the LMS would reject a statement attempting to be a cmi5 defined statement for a session it can't be defined for (because it is a different AU) or if that AU is acting with the knowledge it needs to represent that session then it is effectively the same AU as far as the LMS is concerned. The bottom line is that the session wrapper protects the LMS from interpreting that data incorrectly. Is it a bad idea to send those types of statements as allowed, yes.

In the wild I see AU's sending passed, failed and completed statements for activity id's that are not the activity id of the AU. Some AU's have multiple and tests some AU's have multiple sections that need to be completed. This seems like desirable behaviour to me.

Correct, that is working as intended and is the reason why the profile context activity is important.

+1 to @cawerkenthin's comments. Though I would state the following differently:

If an AU is sending cmi5 defined statements that do not use the activity ID passed to it on the command line, then it is non-conformant.

I'd probably say it is "non-conformant to attempt to send a cmi5 defined statement that does not use the activity ID as passed on the launch URL". (because a cmi5 defined statement from an AU must use that activity ID) But now we're very into pedantic land.

But isn't that the very definition of a cmi5 allowed statement?

No. A cmi5 allowed statement is effectively not a cmi5 defined statement, and is acceptable to the LMS. Whether it is acceptable to the LMS is determined by the other requirements, such as needing the context template, etc. The activity ID is not considered in whether a statement is an "allowed" statement.

In this context I understand command line to mean launching URL, is that correct?

Correct.

[thomasturrell]

So a cmi5 defined statement will alway include the following:

{
   "category":[
      {
         "id":"https://w3id.org/xapi/cmi5/context/categories/cmi5"
      },
      ...
   ]
}

Where as a cmi5 allowed statement do not?

So how do I interpret the following statement? (taken from the cmi5 website)

{
  "id": "a60dc721-fe51-4668-8c00-35bb9b809fcc",
  "actor": {
    "objectType": "Agent",
    "name": "Learner's Name",
    "account": {
      "homePage": "http://example.org/",
      "name": "Learner"
    }
  },
  "verb": {
    "id": "http://adlnet.gov/expapi/verbs/progressed",
    "display": {
      "und": "progressed"
    }
  },
  "object": {
    "objectType": "Activity",
    "id": "http://example.org/cmi5/au-lms-id/4756b42214cb0cb4825b3c9b5c15053d6a6cf7f5",
    "definition": {
      "name": {
        "en-US": "Introduction to Geology"
      },
      "description": {
        "en-US": "This course will introduce you into the basics of geology. This includes subjects such as\n        plate tectonics, geological materials and the history of the Earth."
      }
    }
  },
  "result": {
    "extensions": {
      "https://w3id.org/xapi/cmi5/result/extensions/progress": 25
    }
  },
  "context": {
    "registration": "ff76a00e-5b3f-4bfc-ba3d-6f4365f96753",
    "contextActivities": {
      "grouping": [
        {
          "objectType": "Activity",
          "id": "http://course-repository.example.edu/identifiers/courses/02baafcf/aus/4c07",
          "definition": {
            "name": {
              "en-US": "Introduction to Geology"
            },
            "description": {
              "en-US": "This course will introduce you into the basics of geology. This includes subjects such as\n        plate tectonics, geological materials and the history of the Earth."
            }
          }
        }
      ]
    },
    "extensions": {
      "https://w3id.org/xapi/cmi5/context/extensions/sessionid": "2d5437c4-7ca2-4c05-a5b9-22e9127e180a"
    }
  },
  "timestamp": "2016-06-09T19:39:46.383Z"
}

I take it that the above statement is cmi5 allowed and therefore:

outside of the scope of the cmi5 specification and should be interpreted in whatever manner was deemed by the author of the statements.

The section 9.5.5.1 states that:

The AU may set this value in statements to indicate level of completion.

Which seems odd if it can not be used to indicate the level of completion of the AU.

[brianjmiller]

Where as a cmi5 allowed statement do not?

Correct. The category list was always intended to capture Activities that indicate the profile(s) that the statement belongs to. That is essentially the definition of a "cmi5 defined" statement.

I take it that the above statement is cmi5 allowed and therefore:

Yes, from the page description: Session showing the use of the “progress” extension. Includes 3 “cmi5 allowed” Statements that have an increasing progress of 25% (25), 50% (50), and 75% (75) followed by a Completed Statement. Note the “progress” extension is not included in the Completed Statement.

Which seems odd if it can not be used to indicate the level of completion of the AU.

It can be and it may be used that way, but in so far as there is no MUST language here then it is not a requirement of the specification and is therefore left open to interpretation. And in the cases that it is, one would expect the object in that statement to be the AU's Activity based on the LMS id (the one provided via the launch URL).

[thomasturrell]

I would like to see the spec be more opinionated about the use of the progress extension.

[MrBillMcDonald]

Per the March 25th meeting, the following summary indicates the general agreement with group on progress extension usage:

• cmi5 Allowed statements - use of progress extension is outside the scope of the spec
• cmi5 Defined statements - progress extension defines overall progress of AU
• cmi5 Allowed statements using AU object ID - progress extension should define overall progress of AU
• cmi5 Allowed statements NOT using AU object ID - progress extension usage is up to developer

cmi5 defined statements are indicated by the "cmi5 Category Activity" (see 9.6.2.1 cmi5 Category Activity)

See best practice Number 11 (https://aicc.github.io/CMI-5_Spec_Current/best_practices/)