The ETL is currently duplicating rows because there are duplicate rows in one of the concept mappings:
select
c3.*, c.numobs
from gcpt_prescriptions_ndcisnullzero_to_concept c3
inner join
(
select label, count(*) as numobs from gcpt_prescriptions_ndcisnullzero_to_concept group by label having count(*)>1
) c
on c3.label = c.label
order by c3.label, c3.concept_id;
The ETL is currently duplicating rows because there are duplicate rows in one of the concept mappings:
gives:
The offending rows are lines 308-309, lines 575-576, and lines 1211-1212.
@aparrot89 can you recommend which concept we should keep?