ericaVoss commented 5 months ago

MEASUREMENT.VALUE_AS_NUMBER/VALUE_AS_CONCEPT_ID/UNIT_CONCEPT_ID/MEASUREMENT_CONCEPT_ID THEMIS Rules

CDM or THEMIS convention?

THEMIS

Table or Field level?

Field

Is this a general convention?

No

Summary of issues

VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID, do they need to be mutually exclusive? [REF]

Summary of answer

Only records where the source value maps to a CONCEPT_ID in the 'Measurement' domain should be included in this table.
- For some MEASUREMENT concepts the result is included in the test. For example, ICD10 CONCEPT_ID 45548980 ‘Abnormal level of unspecified serum enzyme’ indicates a Measurement and the result (abnormal). In those situations, the CONCEPT_RELATIONSHIP table in addition to the ‘Maps to’ record contains a second record with the RELATIONSHIP_ID set to ‘Maps to value’. In this example, the ‘Maps to’ relationship directs to 4046263 ‘Enzyme measurement’ as well as a ‘Maps to value’ record to 4135493 ‘Abnormal’.
- If the raw data provides categorial results as well as continuous results for measurements, it is a valid ETL choice to preserve both values. Thus, VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID are not mutually exclusive. The continuous value should go in the VALUE_AS_NUMBER field and the categorical value should be mapped to a standard concept and put in the VALUE_AS_CONCEPT_ID field. For example, it would be feasible to receive a measurement for a LOINC 8867-4-heart rate from a source system that both indicates that a patient experienced a heart rate of 60 beats per minute and this measurement was considered normal. The 200 can be stored in VALUE_AS_NUMBER, the beats per minute in UNIT_CONCEPT_ID (8541-per minute), and finally the "normal" in VALUE_AS_CONCEPT_ID (4069590-Normal).
- It is also not mandatory to have one or both VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID as it is possible for the result to not be given in the source data. When the result is not known, the MEASUREMENT record represents just the fact that the corresponding measurement was carried out, which in itself is already useful information for some use cases.

Technically this part of a ratified THEMIS concept: CONCEPT_ID (to NULL or not to NULL) #11

Other comments/notes

@MelaniePhilofsky - This wasn't ratified though. It looks like there might have been a discussion (see the forum thread below) but I don't see the entry. However, if I were to assert what to do here is we take the values the data give us. I think our DQD checks have shown us how difficult it is holistically to assert when things are "high" or "low" using the data. We may get there one day, but I don't think we are there yet.

This is in the CDM documentation though: _If the raw data provides categorial results as well as continuous results for measurements, it is a valid ETL choice to preserve both values. The continuous value should go in the VALUE_AS_NUMBER field and the categorical value should be mapped to a standard concept in the ‘Meas Value’ domain and put in the VALUE_AS_CONCEPT_ID field. This is also the destination for the ‘Maps to value’ relationship. If there’s no categorial result in a source_data, set value_as_concept_id to NULL, if there is a categorial result in a source_data but without mapping, set value_as_conceptid to 0.

clairblacketer commented 5 months ago

@ericaVoss since this text is in the CDM documentation, we can assert that value_as_number and value_as_concept_id are not mutually exclusive. Thank you for putting this together

clairblacketer commented 5 months ago

Issue # and location

Issue summary

For a given record in the MEASUREMENT table, are VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID mutually exclusive, meaning if one has a value can the other also have a value?

Convention type

Table

CDM table

measurement

CDM field

value_as_number, value_as_concept_id

Links to issue discussion

Provenance of data

General

The ratified convention

Only records where the source value maps to a CONCEPT_ID in the 'Measurement' domain should be included in this table.

For some MEASUREMENT concepts the result is included in the test. For example, ICD10 CONCEPT_ID 45548980 ‘Abnormal level of unspecified serum enzyme’ indicates a Measurement and the result (abnormal). In those situations, the CONCEPT_RELATIONSHIP table in addition to the ‘Maps to’ record contains a second record with the RELATIONSHIP_ID set to ‘Maps to value’. In this example, the ‘Maps to’ relationship directs to 4046263 ‘Enzyme measurement’ as well as a ‘Maps to value’ record to 4135493 ‘Abnormal’.

If the raw data provides categorial results as well as continuous results for measurements, it is a valid ETL choice to preserve both values. Thus, VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID are not mutually exclusive. The continuous value should go in the VALUE_AS_NUMBER field and the categorical value should be mapped to a standard concept and put in the VALUE_AS_CONCEPT_ID field. For example, it would be feasible to receive a measurement for a LOINC 8867-4-heart rate from a source system that both indicates that a patient experienced a heart rate of 60 beats per minute and this measurement was considered normal. The 60 can be stored in VALUE_AS_NUMBER, the beats per minute in UNIT_CONCEPT_ID (8541-per minute), and finally the "normal" in VALUE_AS_CONCEPT_ID (4069590-Normal).

It is also not mandatory to have one or both VALUE_AS_NUMBER and VALUE_AS_CONCEPT_ID as it is possible for the result to not be given in the source data. When the result is not known, the MEASUREMENT record represents just the fact that the corresponding measurement was carried out, which in itself is already useful information for some use cases.

Date of ratification/published

Sept 24, 2021 (CDM v5.4 release date)

Downstream implications

No

Link to DQD check

NA

Related conventions/further information

NA

#Tags labs, measurement_values, value_as_number, value_as_concept_id, lab_result, numerical_lab_result

MelaniePhilofsky commented 5 months ago

Fixed a couple of typos.

OHDSI / Themis