Clean up data in temp bucket

[ch-kr]

• Filter new additional breaks rerun results to breakpoints only and overwrite; delete temporary chi square HT
• Filter temp table generated searching for single break with lower chi square cutoff to "rescued" transcripts only and overwrite

[ch-kr]

Cluster:

hailctl dataproc start kc --autoscaling-policy max-20 --master-machine-type n1-standard-8 --worker-machine-type n1-standard-8 --project broad-mpg-gnomad --num-workers 2 --pkgs="git+https://github.com/broadinstitute/gnomad_methods.git@main" --max-age=6h --labels=gnomad_release=v2,rmc_step=organize_hts_in_temp --requester-pays-allow-all

NB: gs://regional_missense_constraint/notebooks/clean_up_temp_hts.ipynb ^ Used NB to investigate Tables and overwrite any as necessary. Used console to delete Tables

Summary:

• Deleted gs://regional_missense_constraint/temp/max_chisq_all.ht
• Dropped unnecessary annotations from gs://regional_missense_constraint/temp/first_break_all.ht, filtered to is_break, wrote new HT gs://regional_missense_constraint/temp/round1_chisq_5.ht, deleted HT at old path
• Deleted gs://regional_missense_constraint/temp/temp_first_break_lower_chisq.ht
• Deleted gs://regional_missense_constraint/temp/pre_break.ht (checkpointed table before break search in "pre" sections)
• Deleted gs://regional_missense_constraint/temp/post_break.ht (checkpointed table before break search in "post" sections
• Deleted gs://regional_missense_constraint/temp/break_2_pre_max_chisq.ht (max chi square values by transcript for "pre" sections)
• Deleted gs://regional_missense_constraint/temp/break_2_post_max_chisq.ht (max chi square values by transcript for "post" sections)
• Deleted gs://regional_missense_constraint/temp/break_2.ht (temp table)
• Dropped unnecessary annotations from gs://regional_missense_constraint/temp/multiple_breaks.ht, filtered to breakpoints, wrote to gs://regional_missense_constraint/temp/additional_break_Aug22_rerun.ht, and deleted table at multiple_breaks.ht path