Write a function to python package to generate Precisely VCF from 23andMe files

precisely / bioinformatics

0 stars 0 forks source link

Write a function to python package to generate Precisely VCF from 23andMe files #6

Closed aneilbaboo closed 6 years ago

aneilbaboo commented 6 years ago

Test with several 23andMe files
Assume raw files are in s3://{env}-precisely-genetics-raw-23andme
Output files go in s3://{env}-precisely-genetics-vcf Where env = dev for the time being, but will be stage on staging, prod on prod
Files will be named {opaque_id}_{23andMe-fileHash}.txt; copy the filename to the output bucket (e.g., to s3://{env}-precisely-genetics-vcf/{opaque_id}_{23andme-filehash}.json)
Write errors as to a file s3://{env}-precisely-genetics-vcf/{opaque_id}{23andme-filehash}.error As a JSON object with a message field describing the error, and possibly other values.

taltman commented 6 years ago

Committed working code to convert23andMe branch. I need to wrap up some documentation and test code, and then I'll send a pull request against the dev branch.

taltman commented 6 years ago

For the next sprint, the following needs to be changed:

Output includes all variants, whether or not within gene
Output is now compressed VCF, not compressed JSON This involves using bcftools annotate to create new columns to the initial VCF file
Error states need to be trapped, and written out to a file
Gene association based on transcript, not CDS, coordinates
Document VCF format in output header, and also on wiki: https://github.com/precisely/web/wiki/Database-Architecture
Change "systematic_name" to "svn", or whatever is the standard name from VEP.
Document function, module, and package
Create a unit test that serves as example usage for Vishesh & others

taltman commented 6 years ago

The 23andMe data file generation timestamp needs to be captured, as it informs on what chip version was used.

taltman commented 6 years ago

@visheshd , please see the comment that I left on the associated PR. Thanks!