onc-healthit / onc-certification-g10-test-kit

ONC Certification (g)(10) Standardized API Tests
Apache License 2.0
34 stars 13 forks source link

1.2.10 ONC Certification (g)(10) Error: 783: unexpected token at '' #232

Closed SavannahDearing closed 2 years ago

SavannahDearing commented 2 years ago

ONC Certification (g)(10)
1 Standalone Patient App - Full Access part 1.2.10

Full error message: Error: 783: unexpected token at ''/usr/local/lib/ruby/2.7.0/json/common.rb:156:in `parse'

The only info on this part: INPUT | VALUE -- | -- standalone_patient_id | h1XbVXYwRsK-5v7lhAqqXw url | https://www.medentfhirtest.com/fhir/R4/savcw237 standalone_smart_credentials | {"access_token":"XXX","refresh_token":"YYY","token_url":"https://www.medentfhirtest.com/fhir/R4/token/index.php?medent_practice_id=savcw237","client_id":"AAA","client_secret":"BBB","token_retrieval_time":"2022-08-25T14:11:04+00:00","expires_in":900}

Note: this is a test patient, test data, test server, so no PHI exposed. Replaced authentication and token texts.

Overall, I'm unsure of what this error is trying to point me to.
All previous tests are green, except a warning on 1.2.06: "Token exchange response did not include all requested scopes. These may have been denied by user: patient/.read." I'm assuming because the scopes I return are a list of all approved scopes, not a literal 'patient/.read'

yunwwang commented 2 years ago

The error means that the token response is not a valid JSON. In this test, Inferno tries to read a Patient resource using the smart credentials provided. It is possible that the server response is not a valid json object.

SavannahDearing commented 2 years ago

Is the error message for the Token endpoint, or the Patient endpoint?

Token:

{
    "access_token" : "AAA",
    "expires_in" : 900,
    "id_token" : "BBB.CCC.DDD",
    "patient" : "8755db55-7630-46c2-bfe6-fee5840aaa5f",
    "refresh_token" : "EEE",
    "scope" : "launch/patient openid fhirUser patient/Patient.read patient/AllergyIntolerance.read patient/CarePlan.read patient/CareTeam.read patient/Condition.read patient/Device.read patient/DocumentReference.read patient/Encounter.read patient/Goal.read patient/Immunization.read patient/Medication.read patient/MedicationRequest.read patient/MedicationOrder.read patient/MedicationStatement.read patient/Observation.read patient/Procedure.read patient/DiagnosticReport.read patient/Practitioner.read patient/Organization.read patient/Provenance.read offline_access",
    "token_type" : "bearer"
}

Patient:

{
   "active" : true,
   "address" : [
      {
         "city" : "SYRACUSE",
         "country" : "US",
         "line" : [ "123 test lane", "apt b" ],
         "postalCode" : "13210",
         "state" : "NY",
         "type" : "postal",
         "use" : "home"
      }
   ],
   "birthDate" : "1986-02-28",
   "communication" : [
      {
         "language" : {
            "coding" : [
               {
                  "code" : "en",
                  "display" : "English",
                  "system" : "urn:ietf:bcp:47"
               }
            ],
            "text" : "English"
         },
         "preferred" : true
      }
   ],
   "contact" : [
      {
         "address" : {
            "country" : "US",
            "type" : "postal",
            "use" : "home"
         },
         "name" : {
            "given" : [ "asd" ]
         },
         "period" : {
            "start" : "2018-08-16"
         },
         "relationship" : [
            {
               "coding" : [
                  {
                     "code" : "U",
                     "display" : "Unknown",
                     "system" : "http://terminology.hl7.org/CodeSystem/v2-0131"
                  }
               ]
            }
         ]
      },
      {
         "address" : {
            "country" : "US",
            "type" : "postal",
            "use" : "home"
         },
         "name" : {
            "given" : [ "asdf" ]
         },
         "period" : {
            "start" : "2018-08-16"
         },
         "relationship" : [
            {
               "coding" : [
                  {
                     "code" : "U",
                     "display" : "Unknown",
                     "system" : "http://terminology.hl7.org/CodeSystem/v2-0131"
                  }
               ]
            }
         ]
      },
      {
         "address" : {
            "country" : "US",
            "type" : "postal",
            "use" : "home"
         },
         "name" : {
            "given" : [ "asdf" ]
         },
         "period" : {
            "start" : "2018-08-16"
         },
         "relationship" : [
            {
               "coding" : [
                  {
                     "code" : "U",
                     "display" : "Unknown",
                     "system" : "http://terminology.hl7.org/CodeSystem/v2-0131"
                  }
               ]
            }
         ]
      },
      {
         "address" : {
            "country" : "US",
            "type" : "postal",
            "use" : "home"
         },
         "name" : {
            "given" : [ "asdf" ]
         },
         "period" : {
            "start" : "2018-08-16"
         },
         "relationship" : [
            {
               "coding" : [
                  {
                     "code" : "U",
                     "display" : "Unknown",
                     "system" : "http://terminology.hl7.org/CodeSystem/v2-0131"
                  }
               ]
            }
         ]
      },
      {
         "address" : {
            "country" : "US",
            "type" : "postal",
            "use" : "home"
         },
         "name" : {
            "given" : [ "NEWTEST" ]
         },
         "period" : {
            "start" : "2020-09-18"
         },
         "relationship" : [
            {
               "coding" : [
                  {
                     "code" : "U",
                     "display" : "Unknown",
                     "system" : "http://terminology.hl7.org/CodeSystem/v2-0131"
                  }
               ]
            }
         ]
      }
   ],
   "deceasedBoolean" : false,
   "extension" : [
      {
         "extension" : [
            {
               "url" : "text",
               "valueString" : "Unknown"
            },
            {
               "url" : "ombCategory",
               "valueCoding" : {
                  "code" : "UNK",
                  "display" : "Unknown",
                  "system" : "http://terminology.hl7.org/CodeSystem/v3-NullFlavor"
               }
            }
         ],
         "url" : "http://hl7.org/fhir/us/core/StructureDefinition/us-core-race"
      },
      {
         "extension" : [
            {
               "url" : "text",
               "valueString" : "Unknown"
            }
         ],
         "url" : "http://hl7.org/fhir/us/core/StructureDefinition/us-core-ethnicity"
      },
      {
         "url" : "http://hl7.org/fhir/us/core/StructureDefinition/us-core-birthsex",
         "valueCode" : "UNK"
      }
   ],
   "gender" : "female",
   "id" : "8755db55-7630-46c2-bfe6-fee5840aaa5f",
   "identifier" : [
      {
         "system" : "urn::oid::2.16.840.1.113883.3.227",
         "use" : "usual",
         "value" : "8755db55-7630-46c2-bfe6-fee5840aaa5f"
      }
   ],
   "managingOrganization" : {
      "display" : "MEDENT Practice savcw237",
      "reference" : "https://www.medentfhirtest.com/fhir/R4/savcw237/Organization/savcw237_practiceid"
   },
   "name" : [
      {
         "family" : "fhirtest",
         "given" : [ "Savannah" ],
         "suffix" : [ "Sr" ],
         "use" : "usual"
      }
   ],
   "resourceType" : "Patient",
   "telecom" : [
      {
         "system" : "phone",
         "use" : "home",
         "value" : "(555)-555-5555"
      },
      {
         "system" : "phone",
         "use" : "work",
         "value" : "(651)-981-6565"
      },
      {
         "system" : "phone",
         "use" : "mobile",
         "value" : "(561)-651-6516"
      },
      {
         "system" : "email",
         "use" : "home",
         "value" : "savannahw@medent.com"
      }
   ]
}
yunwwang commented 2 years ago

These two JOSN looks good. Are these the same as what send over the wire?

If you tested with inferno.healthit.gov, can you email me the url of your test session?

SavannahDearing commented 2 years ago

Email sent, thank you :)

yunwwang commented 2 years ago

Thank you for the information. I noticed that the FHIR endpoint in a tenant on a server. Is that correct? Can you try step 7.1.5 in SMART on FHIR IG (http://hl7.org/fhir/smart-app-launch/1.0.0/) on your server (https://www.medentfhirtest.com/fhir/R4/savcw237/Patient/8755db55-7630-46c2-bfe6-fee5840aaa5f) with the access token using any REST client (for example postman)?

SavannahDearing commented 2 years ago

I am able to use our internal test program to send the query as well as postman, sending: Authorization: Bearer TOKEN Accept: application/json

get request to https://www.medentfhirtest.com/fhir/R4/savcw237/Patient/8755db55-7630-46c2-bfe6-fee5840aaa5f

I tried simplifying the contents of the patient return, still the same error from the inferno test, but no errors elsewhere, just not sure of what to look at.

yunwwang commented 2 years ago

The error indicates that this is a JSON parsing error.

In this test, there are two JSON objects. One is the token response JOSN object. I don't think this one has any parsing issue because it passed test 1.2.6-1.2.9.

Another one is the FHIR resource returned from server. I don't have any raw data since the JSON parsing crashed. Since you mentioned that the test passed internal test program, I suspect that external facing intermediate server either blocks the payload or attaches some information to the payload.

SavannahDearing commented 2 years ago

Hmm, according to my tests it should be returning that json container I linked before. I don't see anything in my logs that is denying the connection or returning an error.

Is the parser being used available as some kind of standalone tool that I can test with?

Any way we can see the raw response before parsing?

We are attempting to set up the test suite in house too, maybe Ill be able to troubleshoot myself there.

I can also check with our networking team and see if they can think of anything. I'd imagine if we're blocking things it would also block the previous communications but maybe there's some other filter in place I'm not aware of.

Thanks for helping me out, let me know if you think of anything else I can try!

yunwwang commented 2 years ago

We used ruby JSON library to do the parsing. Here is the source code on github: https://github.com/ruby/ruby/blob/ruby_2_7/ext/json/lib/json/common.rb. The failure happens on line L:156

SavannahDearing commented 2 years ago

Hi, a couple of followup questions:

Does the patient query run thru AWS? Our networking team is having a hard time pinpointing where the response is going to, and thus why they may or may not be blocking it.

For the Ruby JSON parser, are there any options on your line 156? It looks like the standalone tool uses some: def parse!(source, opts = {}) opts = { :max_nesting => false, :allow_nan => true }.merge(opts) Parser.new(source, **(opts||{})).parse end

the code on github only uses Parser.new(source, **(opts||{})).parse Thanks!

yunwwang commented 2 years ago

Yes. inferno.healthit.gov is hosted on AWS.

Here is the how ruby FHIR module parses JOSN string:

      if json.is_a? String
        begin
          if json.encoding.names.include? 'UTF-8'
            json.gsub!("\xEF\xBB\xBF".force_encoding('UTF-8'), '') # remove UTF-8 BOM
          end
          hash = JSON.parse(json)
          ...

https://github.com/fhir-crucible/fhir_models/blob/master/lib/fhir_models/fhir_ext/structure_definition_finding.rb#L22

SavannahDearing commented 2 years ago

HI, Our networking team is blocking everything but port 443, Is there an expected port range that needs to be open for AWS? we see connection attempts from several different ports including 47924, 47154, 55902 which we would be blocking.

SavannahDearing commented 2 years ago

I've run the json that should be coming back to Inferno thru a few builds of the Ruby parser, but not recreating any errors. This is assuming you're getting back what my logs say you should be.

Any ideas of urls or ports that may need to be unblocked?

yunwwang commented 2 years ago

Inferno uses standard https.

SavannahDearing commented 2 years ago

After several reviews, found a few compounding errors. I think the major problem was on our end, where some messages were adding a Tab or Endline character. Closing as resolved, thanks for the assistance!