This study uses a probabilistic reference standard validation method to rapidly estimate the measurement errors of 5 acute pancreatitis phenotype definitions. It also computes comprehensive, population-level characteristics of the cohorts returned by the 5 phenotype algorithms.