Create weekly QC for `histologies.tsv`

[jharenza]

What data file(s) does this issue pertain to?

histologies.tsv

Put your question or report your issue here.

Implement https://github.com/d3b-center/D3b-codes/tree/master/weekly_qc (https://github.com/d3b-center/D3b-codes/blob/master/weekly_qc/01-QC_histology.Rmd) for histologies-base.tsv

Who will do this?

@HuangXiaoyan0106 @yuankunzhu

cc @aadamk

[jharenza]

@HuangXiaoyan0106 for your next weekly QC, can you grab the v11 histologies-base.tsv for the file against which you will compare? Thanks!

[HuangXiaoyan0106]

@HuangXiaoyan0106 for your next weekly QC, can you grab the v11 histologies-base.tsv for the file against which you will compare? Thanks!

@jharenza Weekly QC has been done, but I ran the script manually with v11 histologies-base.tsv, and the results have been uploaded.

• s3://d3b-bix-data-merge-release/histology-weekly-qc-report/2022-0711_tmp_combined_inconsistent_files.xlsx
• s3://d3b-bix-data-merge-release/histology-weekly-qc-report/2022-0711_tmp_combined_changes_file.xlsx

[jharenza]

Thank you for your patience @HuangXiaoyan0106 !

A few notes on the combined_changes file :

• Can you check OS_days? You will have to convert to numeric coming out of the WH since the WH cannot retain these as numeric without creating NULL fields in place of NA. Many of the OS_days are matching but come through in the report as not matching, presumably because one is numeric and one is character.
• For age_last_update_days, OS_status, pathology_diagnosis, pathology_free_text_diagnosis, primary_site, seq_center, tumor_descriptor can you only print those samples which have a difference between last and previous?
• NOTE: we need to check the following OS_status which went from DECEASED to LIVING! @youngnm can you take a look at these please?

  BS_0YBAGHZ1
  BS_3P9BQCDK
  BS_C7FRTK6P
  BS_DWRM8B7J

• NOTE: to check - the following samples had an EPN dx and now have no diagnosis

  BS_BETQ48X0
  BS_CK0BY1MB
  BS_SZ9SVSBH

• Can you check what might be going on with sample_id? The leading zeroes are missing, but last week, they were fine in a file directly downloaded from the WH? Could it be a conversion issue?
• **Note: to check - the following samples went from Tumor to Not Reported sample_type:

  BS_BETQ48X0
  BS_CK0BY1MB
  BS_SZ9SVSBH

[jharenza]

@HuangXiaoyan0106 can you also put the histologies-base-adapt.tsv file which was used for each QC into the S3 bucket please (only starting with this latest one)?

[chinwallaa]

@ewafula check to see if you also have access to this s3 bucket - v12 base histology file

[chinwallaa]

Create meeting for Nick/Eric/JoLynne/Asif/AdamK to review histologies and d3b warehouse.

[chinwallaa]

looking to streamline the histology workflows. Decision to put in PRs directly in the histology repository. Will modify the GTEx file to modify subgroups, and remove that code from downstream modules. Creating tables for mapping broad, short histologies to path dx, within the d3b wareshouse. (TBD- who maintains this - to discuss in the next toolkit meeting).