When performing genetic association studies, the methods assume that the genotypes and phenotypes/covariates match 1:1. But usually the genotypes are a subset of the full sample set in the phenotypes and covariates files.
Right now the solution is the extract the sample IDs using glow.wgr.get_sample_ids, convert this list to a pandas dataframe, add an index, then inner join to the phenotypes / covariates, and back to a spark dataframe and sort on the index (at least until delta lake adds ingest order preservation functionality).
This filtering should either be performed on the fly or there should be explicit recommendations / notebooks to show how to do it in the docs
When performing genetic association studies, the methods assume that the genotypes and phenotypes/covariates match 1:1. But usually the genotypes are a subset of the full sample set in the phenotypes and covariates files.
Right now the solution is the extract the sample IDs using glow.wgr.get_sample_ids, convert this list to a pandas dataframe, add an index, then inner join to the phenotypes / covariates, and back to a spark dataframe and sort on the index (at least until delta lake adds ingest order preservation functionality).
This filtering should either be performed on the fly or there should be explicit recommendations / notebooks to show how to do it in the docs