adlnet / lrs-conformance-test-suite

A NodeJS project that tests the MUST requirements of the xAPI Spec and is based on the ADL testing requirements repository. The test suite website can be found here: https://lrstest.adlnet.gov/. The adopters website can be found here: https://adopters.adlnet.gov/
https://adlnet.gov/projects/xapi/
MIT License
67 stars 42 forks source link

XAPI-00057, invalid IRI in extensions #228

Open spsheni opened 6 years ago

spsheni commented 6 years ago

Second and Fourth test expects 400 error. But the test suite passes '{definition: {extensions: VALID_EXTENSION_COMPONENT}}'.

It is defined as, var VALID_EXTENSION_COMPONENT = { 'id': 'valid', 'description': { 'en-US': 'valid' } }

'id' and 'description' both seems to be a valid IRI to me.

Please let me know if the test case is correct? If so, what is the best solution to validate an given IRI? I tried IRIResolver from https://jena.apache.org/documentation/javadoc/arq/org/apache/jena/riot/system/IRIResolver.html, but it reports above IRI are fine. Is there any other java library that I can use?

spsheni commented 6 years ago

More detailed log from test-suite is, { "title": "An Activity Definition's \"extension\" property is an Object (Type, Data 2.4.4.1.s2.table1.row5, XAPI-00057)", "name": "An Activity Definition's \"extension\" property is an Object", "requirement": [ "Data.md#2.4.4.1.s2.table1.row5" ], "log": "", "status": "failed", "tests": [ { "title": "statement activity \"extension\" invalid iri", "name": "statement activity \"extension\" invalid iri", "requirement": "", "log": "REQUEST SUPERREQUEST\n_\nPOST /xapi/v2/statements HTTP/1.1\r\nX-Experience-API-Version: 1.0.3\r\nAuthorization: Basic dTpw\r\nhost: localhost:8080\r\naccept: application/json\r\ncontent-type: application/json\r\ncontent-length: 619\r\nConnection: close\r\n\r\n{\"actor\":{\"objectType\":\"Agent\",\"name\":\"xAPI account\",\"mbox\":\"mailto:xapi@adlnet.gov\"},\"verb\":{\"id\":\"http://adlnet.gov/expapi/verbs/attended\",\"display\":{\"en-GB\":\"attended\",\"en-US\":\"attended\"}},\"object\":{\"objectType\":\"SubStatement\",\"actor\":{\"objectType\":\"Agent\",\"name\":\"xAPI mbox_sha1sum\",\"mboxsha1sum\":\"cd9b00a5611f94eaa7b1661edab976068e364975\"},\"verb\":{\"id\":\"http://adlnet.gov/expapi/verbs/reported\",\"display\":{\"en-GB\":\"reported\",\"en-US\":\"reported\"}},\"object\":{\"objectType\":\"Activity\",\"id\":\"http://www.example.com/meetings/occurances/34534\",\"definition\":{\"extensions\":{\"id\":\"valid\",\"description\":{\"en-US\":\"valid\"}}}}}}\n\nRESPONSE SUPERREQUEST\n\nHTTP/1.1 200 OK\nserver: Apache-Coyote/1.1\nx-experience-api-version: 1.0.3\ncontent-type: application/json;charset=UTF-8\ncontent-length: 40\ndate: Mon, 19 Mar 2018 06:37:32 GMT\nconnection: close\n\n698d9ced-6d1f-45b9-87f8-3d0d5127fb00\n=======================================\n", "status": "failed", "error": "Error: Expected response status code to be 400 got 200\nat (C:\Projects\TEST\lrs-conformance-test-suite\test\templatingSelection.js:53:44)", "tests": [] }, { "title": "statement substatement activity \"extension\" invalid iri", "name": "statement substatement activity \"extension\" invalid iri", "requirement": "", "log": "REQUEST SUPERREQUEST\n_\nPOST /xapi/v2/statements HTTP/1.1\r\nX-Experience-API-Version: 1.0.3\r\nAuthorization: Basic dTpw\r\nhost: localhost:8080\r\naccept: application/json\r\ncontent-type: application/json\r\ncontent-length: 619\r\nConnection: close\r\n\r\n{\"actor\":{\"objectType\":\"Agent\",\"name\":\"xAPI account\",\"mbox\":\"mailto:xapi@adlnet.gov\"},\"verb\":{\"id\":\"http://adlnet.gov/expapi/verbs/attended\",\"display\":{\"en-GB\":\"attended\",\"en-US\":\"attended\"}},\"object\":{\"objectType\":\"SubStatement\",\"actor\":{\"objectType\":\"Agent\",\"name\":\"xAPI mbox_sha1sum\",\"mboxsha1sum\":\"cd9b00a5611f94eaa7b1661edab976068e364975\"},\"verb\":{\"id\":\"http://adlnet.gov/expapi/verbs/reported\",\"display\":{\"en-GB\":\"reported\",\"en-US\":\"reported\"}},\"object\":{\"objectType\":\"Activity\",\"id\":\"http://www.example.com/meetings/occurances/34534\",\"definition\":{\"extensions\":{\"id\":\"valid\",\"description\":{\"en-US\":\"valid\"}}}}}}\n\nRESPONSE SUPERREQUEST\n\nHTTP/1.1 200 OK\nserver: Apache-Coyote/1.1\nx-experience-api-version: 1.0.3\ncontent-type: application/json;charset=UTF-8\ncontent-length: 40\ndate: Mon, 19 Mar 2018 06:37:32 GMT\nconnection: close\n\nc9e137c7-ce90-4361-a448-cbc525be4db3\n=======================================\n", "status": "failed", "error": "Error: Expected response status code to be 400 got 200\nat (C:\Projects\TEST\lrs-conformance-test-suite\test\templatingSelection.js:53:44)", "tests": [] } ] }

garemoko commented 6 years ago

'id' and 'description' are not valid IRIs, so the test is fine.

The wikipedia page for IRIs is here: https://en.wikipedia.org/wiki/Internationalized_Resource_Identifier

But it's not very detailed and doesn't make a great introduction if you're not already familiar with URIs. You can read about URIs here: https://en.wikipedia.org/wiki/Uniform_Resource_Identifier

Or to go even simpler, "the most common form of URI is the Uniform Resource Locator (URL)", which you can read about here: https://en.wikipedia.org/wiki/URL

spsheni commented 6 years ago

thank you for sharing the details. After reading through, for my Java IRI validator, I used below method to validate while deserializing statement,

  1. create URI from the string
  2. try uri.toURL() method of java.net.URI
  3. if no exceptions IRI is fine Please share if any other/better IRI validations.
garemoko commented 6 years ago

Not all IRIs are URLs.

We should make sure the test suite sends some non-URL and even non-URI IRIs.

spsheni commented 6 years ago

Apache Jena IRI seems to be the right one for IRI validations.

Yes, Test suite does not seem to have an case with IRI having non-ASCII chars.

garemoko commented 6 years ago

@vbhayden does the test suite currently have tests that include IRIs that are not URIs?

vbhayden commented 6 years ago

I'll look through it, but I don't see anything right now that checks for the nested case of IRI and not URI.

My hunch is that it won't, as this is one of the tests for 1.0.3 verb conformance:

{   //see above
    name: 'A "verb" property\'s "id" property is an IRI (Type, Data 2.4.3.s3.table1.row1, XAPI-00044)',
    config: [
        {
            name: 'statement verb "id" not IRI',
            templates: [
                {statement: '{{statements.verb}}'},
                {verb: '{{verbs.default}}'},
                {id: INVALID_URI}
            ],
            expect: [400]
        },
        {
            name: 'statement substatement verb "id" not IRI',
            templates: [
                {statement: '{{statements.object_substatement}}'},
                {object: '{{substatements.verb}}'},
                {verb: INVALID_URI}
            ],
            expect: [400]
        }
    ]
},

Since this is a template for confirming a valid IRI (and passes an invalid URI to do this), the proper subset case probably wasn't considered.