jmbanda
commented
4 years ago 1) All the PheKB validation is done outside of APHRODITE. As you know, the PheKB algorithms are sets of rules that translate into SQL queries to select patients (cases and controls) for any given phenotype. Some PheKB algorithms are already translated into ATLAS queries (look in the public OHDSI instance of ALTAS).
2) APHRODITE does not offer any functionality to use PheKB algorithms.
3) Once you have cases identified with a PheKB algorithm, you can then cross reference to see if you also found those cases using a model built by APHRODITE. This is how you validate that your APHRODITE model is good or bad.
4) In a way yes. You can also validate using PheKB algorithms (as I mentioned above), but in those cases you can't be sure that the patients selected by the PheKB algorithms, in your own dataset, are true cases. So you would need to do a manual review for that.
Hello @jmbanda ,
I was reading you paper and came to know about PheKB repository. I see that you have T2DM PheKB algorithm in your study.
For ex, If I would like to validate the PheKB T2DM algorithm using APHRODITE, How do I go about it?
Meaning I saw the tutorial where you use a concept to create Keyword list and then use that to select case and controls. Later use them tp extract data and build model but where and how should I use the Phenotype algorithm like PheKB T2DM in APHRODITE?
2) Which function in your package does this? I went through the doc but a bit confused.
3) Or is it like, APHRODITE only helps in labeling our dataset (because it is time consuming and costly do manually)? Later, we again apply the rule based algorithm on our dataset where labels are generated using APHRODITE and manually calculate the sensitivity, specificity, PPV of the algorithm and compare it with the algorithm performance from the gold standard dataset (Same dataset which is manually labelled). Have I understood this right?
4) So is it like to use APHRODITE and validate a Phenotype algorithm, we always need to have atleast subset of our dataset with gold standard labels to assess the performance of APHRODITE itself?