search

Joker-Jerome / UTMOST

UTMOST (unified test for molecular signatures) is a method for cross-tissue gene expression imputation for transcriptome-wide association analyses.
34 stars 16 forks source link

single_tissue_covariance.py not working #3

Closed BinglanLi closed 3 years ago

BinglanLi commented 5 years ago

I am trying to generate single tissue covariance files for my own dataset, but when I ran the following codes, no results were generated.

for TISSUE in ${DBNAMES[@]}; do python ${UTMOST_DIR}single_tissue_covariance.py \ --weight_db ${UTMOST_MODEL_DIR}${TISSUE}.db \ --input_folder ${DOSAGE_DIR} \ --covariance_output ${COVARIANCE_SINGLE_TISSUE_DIR}${TISSUE}.txt.gz done

Joker-Jerome commented 5 years ago

Did you generate your own weight database?

BinglanLi commented 5 years ago

No, I was using the UTMOST database in ./sample_data/weight_db_GTEx/ named after tissue.db.

On Mon, Jun 17, 2019 at 4:30 PM Zhaolong(Jerome) Yu < notifications@github.com> wrote:

Did you generate your own weight database?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Joker-Jerome/UTMOST/issues/3?email_source=notifications&email_token=AEEAARDDULGQPUXWWUWEDITP27X5VA5CNFSM4HYZ44XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX4LOSA#issuecomment-502839112, or mute the thread https://github.com/notifications/unsubscribe-auth/AEEAARB22XTRV3GOHEOP4KTP27X5VANCNFSM4HYZ44XA .

-- Binglan Li Ph.D. candidate, Genomics and Computational Biology University of Pennsylvania, Ritchie Lab Mail: binglan.li@pennmedicine.upenn.edu

Joker-Jerome commented 5 years ago

For some iterations, the program will take a longer time to run. Which environment are you using?

BinglanLi commented 5 years ago

I am using linux with python 2.7.14. And I don't think there is any iteration. The output says the following,

INFO - Loading Weights

INFO - Building files

INFO - Ran successfully However, there was nothing in the output covariance file as if no computation has been run.

On Tue, Jun 18, 2019 at 8:30 AM Zhaolong(Jerome) Yu < notifications@github.com> wrote:

For some iterations, the program will take a longer time to run. Which environment are you using?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Joker-Jerome/UTMOST/issues/3?email_source=notifications&email_token=AEEAARDFRKW6LJAVZJHOJCDP3DIP7A5CNFSM4HYZ44XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6G7AQ#issuecomment-503082882, or mute the thread https://github.com/notifications/unsubscribe-auth/AEEAARAK2XRYRECO2T7Y2ETP3DIP7ANCNFSM4HYZ44XA .

-- Binglan Li Ph.D. candidate, Genomics and Computational Biology University of Pennsylvania, Ritchie Lab Mail: binglan.li@pennmedicine.upenn.edu

Joker-Jerome commented 5 years ago

Before running this command, you should delete the previous ${TISSUE}.txt.gz in the output folder.

BinglanLi commented 5 years ago

Yes, I did. The program does create an error and does not run if there is any pre-existed files in the output folder.

On Tue, Jun 18, 2019 at 10:07 AM Zhaolong(Jerome) Yu < notifications@github.com> wrote:

Before running this command, you should delete the previous ${TISSUE}.txt.gz in the output folder.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Joker-Jerome/UTMOST/issues/3?email_source=notifications&email_token=AEEAARA4DQUQU4STYR5QHXLP3DT2RA5CNFSM4HYZ44XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6XYJY#issuecomment-503151655, or mute the thread https://github.com/notifications/unsubscribe-auth/AEEAARBJ2N75MQNX5WEWBUTP3DT2RANCNFSM4HYZ44XA .

-- Binglan Li Ph.D. candidate, Genomics and Computational Biology University of Pennsylvania, Ritchie Lab Mail: binglan.li@pennmedicine.upenn.edu

Joker-Jerome commented 5 years ago

That's pretty strange. Can you paste your code here?

BinglanLi commented 5 years ago

The dosage files were generate by the PrediXcan script.

DBNAMES=("Whole_Blood" "Vagina" "Uterus" "Thyroid" "Testis" "Stomach" " Spleen" "Small_Intestine_Terminal_Ileum" "Skin_Sun_Exposed_Lower_leg" " Skin_Not_Sun_Exposed_Suprapubic" "Prostate" "Pituitary" "Pancreas" "Ovary" " Nerve_Tibial" "Muscle_Skeletal" "Lung" "Liver" "Heart_Left_Ventricle" " Heart_Atrial_Appendage" "Esophagus_Muscularis" "Esophagus_Mucosa" " Esophagus_Gastroesophageal_Junction" "Colon_Transverse" "Colon_Sigmoid" " Cells_Transformed_fibroblasts" "Cells_EBV-transformed_lymphocytes" " Breast_Mammary_Tissue" "Brain_Putamen_basal_ganglia" " Brain_Nucleus_accumbens_basal_ganglia" "Brain_Hypothalamus" " Brain_Hippocampus" "Brain_Frontal_Cortex_BA9" "Brain_Cortex" " Brain_Cerebellum" "Brain_Cerebellar_Hemisphere" "Brain_Caudate_basal_ganglia " "Brain_Anterior_cingulate_cortex_BA24" "Artery_Tibial" "Artery_Coronary" " Artery_Aorta" "Adrenal_Gland" "Adipose_Visceral_Omentum" " Adipose_Subcutaneous")

list directories

UTMOST_MODEL_DIR="${HOME} /group/datasets/UTMOST/models/gtex-v6p_starnet_blueprint/sample_data/weight_db_GTEx/ "

UTMOST_DIR="${HOME}/group/datasets/UTMOST/"

DOSAGE_DIR="${HOME} /group/personal/victoria/twas_model_choices/data/actg_phase1-4/allele_dosage/ "

COVARIANCE_SINGLE_TISSUE_DIR="${HOME} /group/personal/victoria/twas_model_choices/data/actg_phase1-4/covariance_single_tissue/ "

COVARIANCE_CROSS_TISSUE_DIR="${HOME} /group/personal/victoria/twas_model_choices/data/actg_phase1-4/covariance_cross_tissue/ "

mkdir ${COVARIANCE_SINGLE_TISSUE_DIR}

mkdir ${COVARIANCE_CROSS_TISSUE_DIR}

calculate single tissue covariance matrix

for TISSUE in ${DBNAMES[@]}; do

python ${UTMOST_DIR}single_tissue_covariance.py \

--weight_db ${UTMOST_MODEL_DIR}${TISSUE}.db \

--input_folder ${DOSAGE_DIR} \

--covariance_output ${COVARIANCE_SINGLE_TISSUE_DIR}${TISSUE}.txt.gz

done

calculate cross tissue covariance matrix

python ${UTMOST_DIR}joint_covariance.py \

--weight_db ${UTMOST_MODEL_DIR} \

--input_folder ${DOSAGE_DIR} \

--gene_info ${UTMOST_DIR}/intermediate/gene_info.txt \

--covariance_output ${COVARIANCE_CROSS_TISSUE_DIR}

On Tue, Jun 18, 2019 at 10:28 AM Zhaolong(Jerome) Yu < notifications@github.com> wrote:

That's pretty strange. Can you paste your code here?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Joker-Jerome/UTMOST/issues/3?email_source=notifications&email_token=AEEAARDGJTH4TQUZ73TC3ZTP3DWIVA5CNFSM4HYZ44XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6Z7RQ#issuecomment-503160774, or mute the thread https://github.com/notifications/unsubscribe-auth/AEEAARD535QFTLXLIQZJUMTP3DWIVANCNFSM4HYZ44XA .

-- Binglan Li Ph.D. candidate, Genomics and Computational Biology University of Pennsylvania, Ritchie Lab Mail: binglan.li@pennmedicine.upenn.edu

Joker-Jerome commented 5 years ago

How many samples are there in your dosage files?

BinglanLi commented 5 years ago

Over four thousand individuals

On Wed, Jun 19, 2019 at 6:43 AM Zhaolong(Jerome) Yu < notifications@github.com> wrote:

How many samples are there in your dosage files?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Joker-Jerome/UTMOST/issues/3?email_source=notifications&email_token=AEEAARBEL5GMALSMNB3XNJ3P3IEWPA5CNFSM4HYZ44XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYBOXSI#issuecomment-503507913, or mute the thread https://github.com/notifications/unsubscribe-auth/AEEAARF56VUKQMW3DTPOLN3P3IEWPANCNFSM4HYZ44XA .

-- Binglan Li Ph.D. candidate, Genomics and Computational Biology University of Pennsylvania, Ritchie Lab Mail: binglan.li@pennmedicine.upenn.edu

duaghk commented 4 years ago

I think, the code is wrong...

for TISSUE in ${DBNAMES[@]}; do python $UTMOST_DIR/single_tissue_covariance.py --weight_db $UTMOST_MODEL_DIR/${TISSUE}.db --input_folder $DOSAGE_DIR --covariance_output $COVARIANCE_SINGLE_TISSUE_DIR/${TISSUE}.txt.gz done

it will be work... maybe :)

Ludmila6114 commented 4 years ago

Which p-value correction should we use? 0.05/(amount of Tissues)?

yiminghu commented 4 years ago

If you are using the joint test, then you can use 0.05/number_of_genes_tested.

On Oct 17, 2020, at 3:30 AM, Pro_milka notifications@github.com wrote:

 Which p-value correction should we use? 0.05/(amount of Tissues)?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.