xapi statements should use UTF-8 encoding

[garemoko]

We've got some question text data in our LRS from an adapt course that contains the string isn’t when isn't is what's intended. This suggests an encoding issue somewhere along the line.

[brian-learningpool]

Can you paste a snippet of the component.json or whatever JSON contains the text which is getting corrupted?

[garemoko]

{
  "id": "5e821ae9-d322-4df0-8e0a-cba82467524d",
  "actor": {
    "objectType": "Agent",
    "name": "foo",
    "mbox": "mailto:foo@example.com"
  },
  "verb": {
    "id": "http://adlnet.gov/expapi/verbs/answered",
    "display": {
      "en-US": "answered"
    }
  },
  "object": {
    "objectType": "Activity",
    "id": "https://example.com/foo",
    "definition": {
      "name": {
        "en": "foo"
      },
      "description": {
        "en": "foo"
      },
      "type": "http://adlnet.gov/expapi/activities/question",
      "interactionType": "choice",
      "correctResponsesPattern": [
        "1[,]3[,]4"
      ],
      "choices": [
        {
          "id": "1",
          "description": {
            "en": "foo isn’t foo"
          }
        },
        {
          "id": "2",
          "description": {
            "en": "foo"
          }
        },
        {
          "id": "3",
          "description": {
            "en": "foo"
          }
        },
        {
          "id": "4",
          "description": {
            "en": "foo"
          }
        },
        {
          "id": "5",
          "description": {
            "en": "foo"
          }
        }
      ]
    }
  },
  "result": {
    "score": {
      "raw": 1
    },
    "success": true,
    "completion": true,
    "response": "1[,]3[,]4"
  },
  "context": {
    "registration": "d2a442f4-8821-4c9c-ade7-51f60befd87f"
  },
  "timestamp": "2019-06-07T15:48:53.401Z",
}

[brian-learningpool]

Can you paste the corresponding components.json entry for this statement? It could be a problem with the xAPIWrapper library.

[moloko]

it’s probably a curly apostrophe in components.json, author should be using ' or ‘ really - but since you can’t rely on people doing that the code might have to do some conversion

[garemoko]

@brian-learningpool It might be harder for me to get that. I just see the resulting statements in Watershed.

[brian-learningpool]

Hi @garemoko, I've tested curly apostrophes in components.json and they behave as expected, at least on SCORM Cloud and Learning Locker LRS.

Is it possible this was JSON which was returned from a translation company and the corruption is in the content itself?

[garemoko]

Not sure about the translation company side (it's a US only based org using content built by an Aussie company, so all english), but I guess the text could be an issue in the content itself.

@julesb any thoughts on this issue?

[julesb]

@garemoko - Andrew i've been able to replicate this behaviour using Adapt module -> xapiapps -> Learning Locker. I suspect that this is happening in the xapiapps backend during the sanitization etc of statements received from the content, before forwarding to the LRS. I've added an issue to our tracker.

[garemoko]

@julesb Thanks! I'll close this then.