Report columns and Not Met

[Lanadot]

These are two things we have talked about doing in the past. They came up recently in a wb meeting with a client:

First item -Alter the column titled EXCLUSIONS to read EXCEPTIONS

Second item- Add in a column that functions to capture that the eligible patient data has simply not yet been collected versus being an actual PERFORMANCE NOT MET.

[tmccormi]

Second item - This is really about Inverse rules like 001 where Not Met is a good thing and needs to be understood as different from "no data"

[aethelwulffe]

UPDATE: We have commenced adding the NotMet class. This is currently untested, and not implemented in the report itself.

[Trodrige]

First item - DONE.

Second Item - @aethelwulffe are you referring to the PQRS_000x_NotMet classes?

[aethelwulffe]

We need to involve @Lanadot here. I think we may have altered out intentions in reference to Exclusion vs. Exception.

Second item does indeed refer to the new NotMet classes. I have implemented a couple of classes in a few measures, and I think I have gotten all the bits necessary for the engine to run the classes (untested...just implementing logic), but the report itself is not yet set up to collate the data from report_results and report_itemized tables.

The Main columns are the Initial patient population, Denominator (pass denominator class), Performance Met (Numerator class pass), and Performance Not Met (Fail Numerator).

In previous years, there were things called Exclusion or Exception (with no consistency) for either Denominator or Numerator Exclusions/Exceptions.

In the current standards http://suncoastconnection.com/standards/index.php/ Denominators have Exclusions, but those are simply subtracted from the total denominator, and are included in the Denominator class code. They are not recorded separately, and are just a different semantic of a way to not include a patient in the denominator.

Numerators may still, in the current year (confirm @Lanadot ) have something called an exclusion or an exception...possibly not anymore. I need to check. The code has not all been upgraded to the 2019 standards yet. If so, these will be included in the Exception category.

If a patient passes the Numerator Class, they are show as "Performance Met". If they do not pass the numerator class, currently they are shown in the Performance Not Met category.

So, to pass a Numerator, you should have a set of patient criteria, and that pretty much always includes a code recorded in the billing table. If they have that code, they pass. If they do not have that code, they fail. That isn't really where it should end though, as the patient is supposed to either have a Passing code, or a Failing Code. If they don't have either one, they are shown as "fail", when they should show in a new column called "Not Reported". So, after the Numerator check, the next class should be "Not Met". This class should check for positive presence of a FAILURE code, and report those in the Performance Not Met column. This is what we refer to as a Hard Fail. If they don't have either one, they need to be recorded a "Not reported" or "Unreported" column.

So, if for 2019 there are measures that require the "Exclusion/Exception" column or class (unverified, might not be needed anymore), then the report columns should be: Total Patients (calculated as all the patients seen in that reporting period). Denominator (Counted if you pass the Initial Patient Population and Denominator classes). Exclusions (might need to be called Exceptions, but is Exclusion class check). Performance Not Met (NotMet class pass). Performance Met (Numerator class pass). Unreported (Fail both Numerator and NotMet class). Performance Rate calculated percentage with Numerator pass/(Denominator - Exclusions) expressed in clinical_rules.php as

 function calculate_percentage($pass_filt,$exclude_filt,$pass_targ) {
  if ($pass_filt > 0) {
    $perc = number_format(($pass_targ/($pass_filt-$exclude_filt))*100) . xl('%');
  }
  else {
    $perc = "0". xl('%');
  }
  return $perc;

[Lanadot]

If we stick to the specifications, that column should be labeled Denominator Exception(s) Perhaps we could just give it the title EXCEPTION(S)?  Here’s the difference: In contrast to Denominator Exclusions, which eliminate a patient from the Denominator at the beginning of the process, a Denominator Exception is applied later, when a patient who was originally determined to be included in denominator is found to have an exceptional situation - defined in the measure specifications) that is considered just cause to remove them from the denominator. 

[Lanadot]

To clarify: during collection for Numerator data there needs to be a field that gives the data collector an option to indicate that an Exception applies for a patient who was originally eligible to be included in the denominator. Often the exception is for medical reasons, like an allergy to a medication that is required to be prescribed in order to be considered a “performance met”.  Any exception will be figured out of the denominator during final performance calculations. 

[aethelwulffe]

So. The short answer is that we have the columns as I have laid them out correct, the name of the 4th column will be Exceptions, and we are on the right track.

1. Title
2. Total Patients (calculated as all the patients seen in that reporting period).
3. Denominator (Counted if you pass the Initial Patient Population and Denominator classes).
4. Exceptions (Exclusion class...classes can be renamed throughout the code base, or leave it as is)
5. Performance Not Met (NotMet class pass).
6. Performance Met (Numerator class pass).
7. Unreported (Fail both Numerator and NotMet class).
8. Performance Rate calculated percentage with Numerator pass/(Denominator - Exclusions)

[Trodrige]

Ok, now I get the point. What is left here is getting the new column (Unreported) which indicates that a patient neither has a "failing code" nor a "passing code".

I see lots of PQRS_000* folders with different classes with the different column names.

@aethelwulffe please which files will these be made?

[aethelwulffe]

OK, Let's start from the beginning. History of the QA program: Originally, this was called PQRI. The name changed to PQRS. Now it is MIPS. When you see PQRS, it is just a legacy naming convention. In the past, the reporting engine was also used to process "CQM", "AMC", and "Standard" reports. Those are now legacy, and generally speaking, most of the references to them are bits of code that are orphaned (within the code under modules/MIPS).

Setup:

1. To work with any of this stuff, you need a test data set that will give you results. I will ensure you have one of those databases once I whip one up.

2. You must use QA_Measures-->MIPS Module Installer to get the necessary tables installed.

3. You must use the test files by placing them into \sites\default\filemanager\files\Importer.

4. Import the test database by using QA_Measures-->Database Import.

5. Check for proper import by opening patient search and make sure you have patients there.

6. Use QA_Measures-->Select Measures and pick a few measures. Pick a few numbers higher than say 100 to ensure you don't try to use any we are working on right now (and have added new classes for).

7. Use QA_Measures-->Create MIPS Report. Enter a date range appropriate for the data you have (which I will tell you when I give you the database inserts). Select "Group,Clinic" as the provider, and "Group" as the last criteria.

8. Run a report.

9. This report will run, cycling through each measure. It runs each patient through each measure in batches of 100, and the completeness overall updates on the screen every 10 seconds or so. Eventually you will see something like this: [REFERENCE 1] Please note the fact that some of the numbers are actually links. Clicking on the link under the "Not Met" column takes you to the Direct Entry screen. This allows the user to correct/change the passing status of the patient for that measure. Doing entry there will not update the numbers on the report itself (though that would be nice) but it removes them from this list, and ensures that if you re-run the report, the next one will reflect the new results.

10. If you close this screen, the report will keep running (unless you kill the mysql process). You may view the status of the report completion and access previously run reports under QA_Measures-->View Reports You may run multiple reports at one time.

OK. So you will find that all the tables in modules/MIPS/SQL/Install are important for this. Those are generally static data sources that hold the current standard code lists, register what measure calculations the engine should look for etc... You have also imported the patient data and other stuff with the data set I have given (will have given) you. Now there are two other tables that the system used when you ran the report: report_results which will list the reports you have run, their criteria, and stores a JSON string that holds the data of the displayed report columns. report_itemized which lists every denominator and numerator check for each patient and each measure combo.
In the above, report_itemized.report_id is just the id row of the matching report in report_results.report_id Itemized_test_id is just the sequence index of the measures listed in the report_results table where the field_id = 'data'. This is a JSON string, and here is the first measure string from that report_id=34 shown above: {"is_main":true,"population_label":"Population Criteria","numerator_label":"Numerator","concatenated_label":"","total_patients":1059,"excluded":0,"pass_filter":582,"pass_target":502,"percentage":"86%","id":"PQRS_0047","pid":"0","active_alert_flag":"0","passive_alert_flag":"0","patient_reminder_flag":"0","release_version":"2017","web_reference":"","access_control":"patients:med","pqrs_code":"PQRS_0047","pqrs_individual_2016_flag":"1","pqrs_group_type":"X","active":"1","itemized_test_id":2} You can see there that the itemized_test_id in that string is '2', but it is actually measure 'PQRS_0047'. Just making sure that isn't a point of confusion.

• In the above, the lines with a blank "Numerator" line just indicate that the Denominator was run for the patient. The result of "3" there is a neutral value.
• For the "numerator_label" lines, a "1" indicates they passed the "Performance Met" check, which means they passed that measure's Numerator.php or numerator class.
• A '2' is where they passed the Exclusion.php check.
• A '0' is a failed both exclusion and numerator check.
• A '9' only happens when [REFERENCE 1 above] direct entry happens. While it would be good to update the JSON string with the new status, the Direct Entry screen currently just adds medical codes to the patient record (billing table), and sets that report_itemized row value to '9' to keep it from showing up again. We can deal with that more later.

OK. So, We need the new Not_Met class to run if the Numerator class fails to return true. Currently:

• If Exclusion.php in a measure returns true, it sets the report_itemized.pass to '2' for the row with its numerator_label.
• If Numerator.php in a measure returns true, it sets the report_itemized.pass to '1'. If it fails, it sets it to '0'.

What we want is: For the engine to also run NotMet.php after numerator.php, and if that returns true, set the report_itemized.pass to perhaps '4'. And the report to reflect this "hard fail" condition ('4') as a separate column. And for the report to list the "no result, returned false on Exclusion, Numerator AND NotMet" '0' value as a new column "Unreported".

Currently merged for the NotMet class are some of the base setup tables that register its existence, and a handful of the first few classes defined under \modules\MIPS\rulesets\PQRS\reports\MIPSCQM\PQRS_0xxx. I am working to implement all of those, but we need the change to the engine to do the rest of the magic of recording the results in the report_itemized table, the report_results.field_id[data] JSON to reflect that in the "total_patients":1059,"excluded":0,"pass_filter":582,"pass_target":502,"percentage":"86%" section as "total_patients":1059,"excluded":0,"pass_filter":582,"pass_target":502,"fail_target":0,"percentage":"86%" in which the above:

• "pass_filter" is the number of patients where denominator.php returned true,
• "excluded" is the number of patients where Exclusion.php returned true (value '2' in report_itemized),
• "pass_target" is the number of patients where Numerator.php returned true (value '1'),
• and "fail_target" is the number of patients where NotMet.php returned true (value '4').
  Once the above is done, the viewed report should indicate the appropriate columns adding the Unreported column. Not Met will list the patients with a value of '4' instead of '0', and the '0' patients will be under "Unreported". The percentage calculation will be (Numerator Exclusions + Performance Met)/Denominator. "Unreported" is still essentially equal to "Not Met" in that calculation, but allows the user to distinguish between reported failures and status unknown. That allows the user to click on the link in the "Unreported" column, and get not both the unreported and known fails, but just get the list of patients where we don't know what their status is.

-I know this is a lot, but... There might be multiple numerator classes in a single measure, and they will be called something like Numerator1, Numerator2 or the like in the numerator_label column.

[Trodrige]

This is becoming clearer now. so we need the test data to see how these work on our side.

[aethelwulffe]

Added test files to modules/MIPS/test_and_docs.  Run your report using the 2017 calendar year as the date range.  Been using this fake data for a while.  I will try to identify measures that will have "hits" but you can select all of the MIPS measures (don't do premeasures or HCC measures) in QA_Measures-->Select Measures if you want to make sure you get some numbers...at least passing denominator, if not numerator.

I will improve this data set for this purpose very soon so we have defined "hits" and a data range in 2019.  This is something though, and may be fine.

Unzip this and place in the \sites[yoursitename]\filemanager\files\Importer directory.

On 2019-07-08 16:15, Tigpezeghe Rodrige wrote:

This is becoming clearer now. so we need the test data to see how these work on our side.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SunCoastConnection/TheRealLibreEHR/issues/406?email_source=notifications&email_token=AAEHGF3HKRL24XKU76OP3D3P6ON55A5CNFSM4FUI7IOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZOHDLI#issuecomment-509374893, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEHGF34JKYUXNE3UTB25ALP6ON55ANCNFSM4FUI7IOA.

[muarachmann]

hI @aethelwulffe can i have a test db.

[aethelwulffe]

I committed the test DB files to modules/MIPS/test_and_docs. Look in the "big" instruction post to see how to load them through the UI. You will also want to load the ICD10 codes (Administration>Other>External Data Loads) as well. While the above is MUCH better for purposes of testing MIPS, for general testing and load/performance stuff use http://suncoastconnection.com/downloads/PrintaPatient_A7_2_2.zip with either a windows machine or WineHQ to generate loads of random patient data.

[muarachmann]

Oops @aethelwulffe i have a linux OS can't run the exe stuff. Any ideas how i can walk around this

[leebc]

Try WINE.

[muarachmann]

Okay thanks @leebc giving it a try

[aethelwulffe]

WineHQ actually. Might work. I can give you test DB's in any case. We can pop up a repo with them.

[Trodrige]

@aethelwulffe When I choose provider=='Group, Clinic' and 'Group' as the last field. The report doesn't show any entry.

If I don't select provider =='Group Clinic' and take provider=='Beringer, Ned'. The thing runs forever. More than 5 minutes with no results

I'm not sure I missed any of the steps.

[aethelwulffe]

OK, so there is a QAmeasures>remove provider assignments thing that lets you set the provider in all patient's demographics (second entry field in form) to the group user. I am going to get you a better database (full database) and make sure it is configured to run right. What is more, it is possible that there is one of the measures that has failing code (all were updated in the last couple of days). I have not tested them all yet. With current code, check your logs and see if a measure hung up. It should be recording a custom error_log message like "Running measure blah-blah-blah". -definetly pull today's code though.

[Trodrige]

My log is clear. No errors, and I have the latest code.

If @muarachmann doesn't have the same issue, then I think we'll need that database.

[aethelwulffe]

I will post a good one tomorrow morning as soon as possible.

[aethelwulffe]

OK. I merged the last @Trodrige PR. This is the current result: As you see, the titles and such are not showing in many cases. Logs:

[Fri Jul 26 16:38:13.845666 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] *DEBUG*: clinical_rules: Site: DASHealth001  About to runReport for PQRS_0001, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.005075 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.005223 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] *DEBUG*: clinical_rules: Site: DASHealth001  About to runReport for PQRS_0047, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.018630 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.018866 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] *DEBUG*: clinical_rules: Site: DASHealth001  About to runReport for PQRS_0130, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.036343 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.036580 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] *DEBUG*: clinical_rules: Site: DASHealth001  About to runReport for PQRS_0226, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.078866 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 78, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.078961 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.078997 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079024 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079057 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 78, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079085 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079115 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079135 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079166 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 78, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079194 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079223 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] PHP Notice:  Undefined index: NotMet in /var/www/MIPS_Gateway/modules/MIPS/rulesets/PQRS/library/PQRSResult.php on line 99, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079247 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.079528 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] *DEBUG*: clinical_rules: Site: DASHealth001  About to runReport for PQRS_0317, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs
[Fri Jul 26 16:38:14.096636 2019] [php7:notice] [pid 32175] [client 73.23.8.59:65014] UNREPORTED: 0, referer: https://samwise.suncoastconnection.com:3443/MIPS_Gateway/modules/MIPS/clinical_measures.php?type=pqrs

[aethelwulffe]

OK, testing on local looked believable. Merged as we have no major things running in production. Good real world testing is upcoming.

[aethelwulffe]

I have been meaning to re-review this for a while. I need to provide specifics and a test DB to demonstrate the issue, but basically, the NotMet !=HardFail+Unreported. I desperately need to look into what exactly is going on there, but this and the eRx modal window are our top priorities.