search

onc-healthit / onc-certification-g10-test-kit

ONC Certification (g)(10) Standardized API Tests
Apache License 2.0
34 stars 12 forks source link

Terminology count tests are failing #87

Closed javacup73 closed 2 years ago

javacup73 commented 2 years ago

Hi, I pulled the latests from main branch and built the docker image by following the instructions documented here: https://github.com/onc-healthit/onc-certification-g10-test-kit

This step: docker-compose -f terminology_compose.yml up took over 6 hours.. but eventually did finish.

When running some of the tests as mentioned in the above page I'm getting the following output:

kalyan@Kalyans-MacBook-Pro onc-certification-g10-test-kit % bundle exec rake terminology:check_built_terminology Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. /usr/local/lib/ruby/gems/3.1.0/gems/pry-byebug-3.8.0/lib/pry-byebug/control_d_handler.rb:5: warning: control_d_handler's arity of 2 parameters was deprecated (eval_string, pry_instance). Now it gets passed just 1 parameter (pry_instance) Terminology build results different than expected. http://hl7.org/fhir/ValueSet/clinical-findings: Expected codes: 158602 Actual codes: 156854 http://hl7.org/fhir/us/core/ValueSet/us-core-careteam-provider-roles: Expected codes: 1731 Actual codes: 1723 http://hl7.org/fhir/us/core/ValueSet/us-core-condition-code: Expected codes: 297182 Actual codes: 295232 http://hl7.org/fhir/ValueSet/device-kind: Expected codes: 16557 Actual codes: 16507 http://hl7.org/fhir/us/core/ValueSet/us-core-diagnosticreport-report-and-note-codes: Expected codes: 270991 Actual codes: 266795 http://hl7.org/fhir/us/core/ValueSet/us-core-diagnosticreport-lab-codes: Expected codes: 57824 Actual codes: 57115 http://hl7.org/fhir/us/core/ValueSet/us-core-documentreference-type: Expected codes: 10602 Actual codes: 10446 http://hl7.org/fhir/us/core/ValueSet/us-core-encounter-type: Expected codes: 15951 Actual codes: 15654 http://hl7.org/fhir/us/core/ValueSet/us-core-medication-codes: Expected codes: 130098 Actual codes: 129310 http://hl7.org/fhir/ValueSet/observation-codes: Expected codes: 270991 Actual codes: 266795 http://hl7.org/fhir/us/core/ValueSet/simple-language: Expected codes: 8212 Actual codes: 8239 http://hl7.org/fhir/us/core/ValueSet/us-core-procedure-code: Expected codes: 285523 Actual codes: 284662 urn:ietf:bcp:47: Expected codes: 8533 Actual codes: 8560 http://www.nlm.nih.gov/research/umls/rxnorm: Expected codes: 210338 Actual codes: 208889 http://snomed.info/sct: Expected codes: 489140 Actual codes: 485094 http://loinc.org: Expected codes: 270991 Actual codes: 266795 http://www.ama-assn.org/go/cpt: Expected codes: 14654 Actual codes: 14385 http://hl7.org/fhir/sid/cvx: Expected codes: 211 Actual codes: 206 http://unitsofmeasure.org: Expected codes: 1423 Actual codes: 1400 urn:oid:2.16.840.1.113883.6.285: Expected codes: 7783 Actual codes: 7563

Ruby version: kalyan@Kalyans-MacBook-Pro onc-certification-g10-test-kit % ruby --version ruby 3.1.1p18 (2022-02-18 revision 53f5fc4236) [x86_64-darwin21]

kalyan@Kalyans-MacBook-Pro onc-certification-g10-test-kit % bundle --version
Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. Bundler version 2.2.32

kalyan@Kalyans-MacBook-Pro onc-certification-g10-test-kit % which ruby /usr/local/Cellar/ruby/3.1.1/bin/ruby

kalyan@Kalyans-MacBook-Pro onc-certification-g10-test-kit % which bundle /usr/local/Cellar/ruby/3.1.1/bin/bundle

When I run the app (http://localhost) I'm able to select a Test suite, e.g. g(10) Standardized Api but then I get an error message: "There is a problem with the terminology resources. "

Am I missing something?

Jammjammjamm commented 2 years ago

Do you see any errors in the logs from the terminology build process? This looks like the build did not complete successfully.

javacup commented 2 years ago

The logs did not report anything abnormal in the terminology build process. Can you point me to the location of the log file so I can review it? Thank you

javacup commented 2 years ago

terminology_builder.log I ran the terminology builder script again and captured the log file for review. Please let me know if you find anything odd. Thank you

Jammjammjamm commented 2 years ago

That run skipped most of the build because the intermediate files were already present. In tmp/terminology you will see folders for 2019, 2020, and 2021. Could you delete all of the files in those folders except for umls.zip and provide us with the logs from running the build again? That will perform the entire build again except for the downloads.

javacup73 commented 2 years ago

terminology_builder.log Please find attached the log file generated after your suggestions.

javacup73 commented 2 years ago

I pulled the latest from git and ran the scripts that validates the terminologies installed from UMLS.. following is one of the errors

bundle exec rake "terminology:check_code[91935009,http://snomed.info/sct]" Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. /usr/local/lib/ruby/gems/3.1.0/gems/pry-byebug-3.8.0/lib/pry-byebug/control_d_handler.rb:5: warning: control_d_handler's arity of 2 parameters was deprecated (eval_string, pry_instance). Now it gets passed just 1 parameter (pry_instance) rake aborted! ArgumentError: wrong number of arguments (given 1, expected 0; required keywords: code, system, valueset) ~/inferno/onc-certification-g10-test-kit/lib/inferno/terminology/tasks/validate_code.rb:13:in initialize' ~/Workspaces/inferno/onc-certification-g10-test-kit/Rakefile:120:innew' ~/Workspaces/inferno/onc-certification-g10-test-kit/Rakefile:120:in `block (2 levels) in <top (required)>' Tasks: TOP => terminology:check_code (See full trace by running task with --trace)

javacup73 commented 2 years ago

bundle exec rake terminology:check_built_terminology Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. Calling DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please callDidYouMean.correct_error(error_name, spell_checker)' instead. /usr/local/lib/ruby/gems/3.1.0/gems/pry-byebug-3.8.0/lib/pry-byebug/control_d_handler.rb:5: warning: control_d_handler's arity of 2 parameters was deprecated (eval_string, pry_instance). Now it gets passed just 1 parameter (pry_instance) Terminology built successfully.

Some terminology not based on UMLS did not match, but this can be a result of these terminologies having a different update schedule than UMLS. As long as the actual number of codes is close to the expected number, this does not does not reflect a problem with the terminology build.

http://hl7.org/fhir/us/core/ValueSet/simple-language: Expected codes: 8212 Actual codes: 8239 urn:ietf:bcp:47: Expected codes: 8533 Actual codes: 8560

Running the above script does say that the "Terminology built successfully" but when running the tool, it gives the following message "There is a problem with the terminology resources. "

Jammjammjamm commented 2 years ago

Do you see another error message? There is a problem with the terminology resources. should only appear together with other error messages.

javacup commented 2 years ago

I'm sorry I didn't follow the question. Are you asking me to post the other error messages from the tool?

Jammjammjamm commented 2 years ago

Yes.

yashpalk commented 2 years ago

Seeing the same issue. Noticing that for some of the ValueSets that have mismatched counts, I'm seeing this message during the build:

terminology_builder_1  | Processing http://hl7.org/fhir/ValueSet/clinical-findings
terminology_builder_1  | ValueSet too costly or unclosed: http://hl7.org/fhir/ValueSet/clinical-findings
terminology_builder_1  | Processing http://hl7.org/fhir/ValueSet/device-kind
terminology_builder_1  | ValueSet too costly or unclosed: http://hl7.org/fhir/ValueSet/device-kind

I also don't see the 2021 terminology being downloaded for some reason, even though the terminology build finishes successfully.

$ ls tmp/terminology
2019  2020  fhir
jsmith5964 commented 2 years ago

Was there a resolution to this issue? I'm experiencing the same.

Nbhatin commented 2 years ago

I see this configuration warning on the UI:

Some terminology not based on UMLS did not match, but this can be a result of these terminologies having a different update schedule than UMLS. As long as the actual number of codes is close to the expected number, this does not does not reflect a problem with the terminology build.

http://hl7.org/fhir/us/core/ValueSet/simple-language: Expected codes: 8212 Actual codes: 8239 urn:ietf:bcp:47: Expected codes: 8533 Actual codes: 8560

Jammjammjamm commented 2 years ago

If you're missing the 2021 terminology, you can run the following commands to just generate the 2021 terminology:

docker-compose -f terminology_compose.yml run terminology_builder bin/prepare_terminology.sh 2021
docker-compose -f terminology_compose.yml run terminology_builder bundle exec rake terminology:create_vs_validators["preferred","2021","false"]
yashpalk commented 2 years ago

Thanks! Generating the 2021 terminology cleared the errors I had under Configuration Messages

Nbhatin commented 2 years ago

@yashpalk : Do you not see any warnings now ?

I have all 3 but still see warnings on actual count <> expected count ls tmp/terminology 2019 2020 2021 fhir

yashpalk commented 2 years ago

I have no errors, but still have warnings. It seems like the warnings are ignorable:

As long as the actual number of codes is close to the expected number, this does not does not reflect a problem with the terminology build.

MTBC-GO-SQ23 commented 2 years ago

I am getting this error, need help

image

Jammjammjamm commented 2 years ago

Have you tried running the commands in this comment?

https://github.com/onc-healthit/onc-certification-g10-test-kit/issues/87#issuecomment-1151055429

MTBC-GO-SQ23 commented 2 years ago

@Jammjammjamm thanks for the reply.

I already did that

MTBC-GO-SQ23 commented 2 years ago

I think the setup is not picking up 2021 version.

MTBC-GO-SQ23 commented 2 years ago

getting this error while running

sudo docker-compose -f terminology_compose.yml run terminology_builder bundle exec rake terminology:create_vs_validators["preferred","2021","false"]

its says some table is missing, any help @Jammjammjamm ?

Processing http://hl7.org/fhir/ValueSet/languages
Processing expanded valueset: http://hl7.org/fhir/ValueSet/languages
Processing http://hl7.org/fhir/ValueSet/allergyintolerance-clinical
Processing expanded valueset: http://hl7.org/fhir/ValueSet/allergyintolerance-clinical
Processing http://hl7.org/fhir/ValueSet/allergyintolerance-verification
Processing expanded valueset: http://hl7.org/fhir/ValueSet/allergyintolerance-verification
Processing http://hl7.org/fhir/ValueSet/allergy-intolerance-type
Processing expanded valueset: http://hl7.org/fhir/ValueSet/allergy-intolerance-type
Processing http://hl7.org/fhir/ValueSet/allergy-intolerance-category
Processing expanded valueset: http://hl7.org/fhir/ValueSet/allergy-intolerance-category
Processing http://hl7.org/fhir/ValueSet/allergy-intolerance-criticality
Processing expanded valueset: http://hl7.org/fhir/ValueSet/allergy-intolerance-criticality
Processing http://hl7.org/fhir/us/core/ValueSet/us-core-allergy-substance
Processing composed valueset: http://hl7.org/fhir/us/core/ValueSet/us-core-allergy-substance
Processing http://hl7.org/fhir/us/core/ValueSet/us-core-allergy-substance
Processing http://hl7.org/fhir/ValueSet/clinical-findings
ValueSet too costly or unclosed: http://hl7.org/fhir/ValueSet/clinical-findings
Processing http://hl7.org/fhir/ValueSet/clinical-findings
rake aborted!
SQLite3::SQLException: no such table: mrrel
MTBC-GO-SQ23 commented 2 years ago

@Jammjammjamm can you help me with the above one please

Jammjammjamm commented 2 years ago

Did you run docker-compose -f terminology_compose.yml run terminology_builder bin/prepare_terminology.sh 2021 first? That creates the database.

MTBC-GO-SQ23 commented 2 years ago

yes @Jammjammjamm I already did that Just to inform you these all setup I am doing on AWS LINUX 2 machine

MTBC-GO-SQ23 commented 2 years ago

getting this after running docker-compose -f terminology_compose.yml up @Jammjammjamm

image

MTBC-GO-SQ23 commented 2 years ago

@javacup how did you resolve this issue?

Jammjammjamm commented 2 years ago

Delete tmp/terminology/2020/umls.zip and try again.

MTBC-GO-SQ23 commented 2 years ago

Its worked :) @Jammjammjamm

Thanks

yunwwang commented 2 years ago

Hello @MTBC-GO-SQ23

I close this issue as no activity in the last three month. If you are still experiencing the same problem, you can either create a new issue or reopen this one.

Thank you.

MTBC-GO-SQ23 commented 2 years ago

thank you, all is good from my side. much appreciated :)

kiwisky commented 2 years ago

Delete tmp/terminology/2020/umls.zip and try again.

@Jammjammjamm Hello Jamm, could you please have a look at #274 and give some instruction? I always got zero size umls.zip. Thank you very much!