search

CBIIT / R-cometsAnalytics

R package development for COMETS Analytics
12 stars 10 forks source link

COMETS 1.3: "Where" functionality not working #31

Closed steven-moore closed 6 years ago

steven-moore commented 6 years ago

The "where" functionality no longer appears to be working. This issue needs to be fixed before I can complete testing on categorical adjustment, since the model we have been testing includes a "where" statement.

ellatemprosa commented 6 years ago

eeks, what is the issue? @ewymathe implemented this functionality. what's happening? i can take a look if needed.

steven-moore commented 6 years ago

Feel free to look..I have to get back to the Demo though, so won’t be doing more work on this today.

S

From: ellatemprosa [mailto:notifications@github.com] Sent: Monday, February 26, 2018 2:46 PM To: CBIIT/R-cometsAnalytics R-cometsAnalytics@noreply.github.com Cc: Moore, Steve (NIH/NCI) [E] steve.moore@nih.gov; Author author@noreply.github.com Subject: Re: [CBIIT/R-cometsAnalytics] COMETS 1.3: "Where" functionality not working (#31)

eeks, what is the issue? @ewymathehttps://github.com/ewymathe implemented this functionality. what's happening? i can take a look if needed.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/CBIIT/R-cometsAnalytics/issues/31#issuecomment-368625887, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhG9TdhFga3XLxW8QlbbjIWo5EvIZcjEks5tYwnvgaJpZM4STzsU.

ellatemprosa commented 6 years ago

if you can quickly describe what you saw would be helpful or is it not at all working?

steven-moore commented 6 years ago

Not working at all.

For instance, I ran models of BMI*all metabolites in interactive mode. Then I ran again, but with a where statement of BMI>=28. There was no change in results or listed N (despite the fact that this should cut the dataset in half). Other uses of where statement also did not work. This includes in the batch mode.

S

From: ellatemprosa [mailto:notifications@github.com] Sent: Monday, February 26, 2018 2:55 PM To: CBIIT/R-cometsAnalytics R-cometsAnalytics@noreply.github.com Cc: Moore, Steve (NIH/NCI) [E] steve.moore@nih.gov; Author author@noreply.github.com Subject: Re: [CBIIT/R-cometsAnalytics] COMETS 1.3: "Where" functionality not working (#31)

if you can quickly describe what you saw would be helpful or is it not at all working?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/CBIIT/R-cometsAnalytics/issues/31#issuecomment-368628898, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhG9TXcA4dUqmZH6De_IxLM_ObEkAhBQks5tYwwXgaJpZM4STzsU.

ewymathe commented 6 years ago

I"ll take a look tonight.

ewymathe commented 6 years ago

I tested the "where" functionality and it works on my end.

`

Set up model without where

modeldata <- getModelData(exmetabdata,colvars="bmi",adjvars=c("age","race_grp","educ_grp","smk_grp","alc_grp","nested_case"),modelspec="Interactive") [1] "Analysis will run on 'All metabolites"

Run correlations without where

corrmatrix <-calcCorr(modeldata,exmetabdata, "DPP")

Set up the same model but with the where

modeldata <- getModelData(exmetabdata,colvars="bmi",adjvars=c("age","race_grp","educ_grp","smk_grp","alc_grp","nested_case"),modelspec="Interactive",where="bmi>1") [1] "Analysis will run on 'All metabolites'" [1] "Filtering subjects according to the rule(s) bmi>1 . 990 of 1000 are retained"

Run correlations on the model with where

corrmatrix.where <-calcCorr(modeldata,exmetabdata, "DPP")

First several lines of output of model without where:

corrmatrix[1:4,] cohort spec model outcomespec exposurespec 1 DPP Interactive _1_2_3_benzenetriol_sulfate_2 bmi 2 DPP Interactive _1_2_dipalmitoylglycerol bmi 3 DPP Interactive _1_2_propanediol bmi 4 DPP Interactive _1_3_7_trimethylurate bmi corr n pvalue adjvars 1 -0.01985724 1000 0.5317556 age race_grp educ_grp smk_grp alc_grp nested_case 2 0.04321913 1000 0.1733481 age race_grp educ_grp smk_grp alc_grp nested_case 3 -0.05580307 1000 0.0786620 age race_grp educ_grp smk_grp alc_grp nested_case 4 0.03067453 1000 0.3339893 age race_grp educ_grp smk_grp alc_grp nested_case outcome_uid outcome exposure_uid 1 _1_2_3_benzenetriol_sulfate_2 _1_2_3_benzenetriol_sulfate_2 bmi 2 _1_2_dipalmitoylglycerol _1_2_dipalmitoylglycerol bmi 3 _1_2_propanediol _1_2_propanediol bmi 4 _1_3_7_trimethylurate _1_3_7_trimethylurate bmi exposure 1 BMI as a continuous value 2 BMI as a continuous value 3 BMI as a continuous value 4 BMI as a continuous value

First several lines of output of model with where:

corrmatrix.where[1:4,] cohort spec model outcomespec exposurespec 1 DPP Interactive _1_2_3_benzenetriol_sulfate_2 bmi 2 DPP Interactive _1_2_dipalmitoylglycerol bmi 3 DPP Interactive _1_2_propanediol bmi 4 DPP Interactive _1_3_7_trimethylurate bmi corr n pvalue adjvars 1 -0.01435470 990 0.65289784 age race_grp educ_grp smk_grp alc_grp nested_case 2 0.04297383 990 0.17799792 age race_grp educ_grp smk_grp alc_grp nested_case 3 -0.06299329 990 0.04821453 age race_grp educ_grp smk_grp alc_grp nested_case 4 0.03844825 990 0.22820831 age race_grp educ_grp smk_grp alc_grp nested_case outcome_uid outcome exposure_uid 1 _1_2_3_benzenetriol_sulfate_2 _1_2_3_benzenetriol_sulfate_2 bmi 2 _1_2_dipalmitoylglycerol _1_2_dipalmitoylglycerol bmi 3 _1_2_propanediol _1_2_propanediol bmi 4 _1_3_7_trimethylurate _1_3_7_trimethylurate bmi exposure 1 BMI as a continuous value 2 BMI as a continuous value 3 BMI as a continuous value 4 BMI as a continuous value `

This does not look obviously wrong to me but let me know if you see something I don't? I've also attached the output here since the printed output looks ugly.

Where_vs_NoWhere.xlsx

steven-moore commented 6 years ago

Thanks Ewy—I really appreciate it.

S

From: Ewy Mathe [mailto:notifications@github.com] Sent: Monday, February 26, 2018 9:09 PM To: CBIIT/R-cometsAnalytics R-cometsAnalytics@noreply.github.com Cc: Moore, Steve (NIH/NCI) [E] steve.moore@nih.gov; Author author@noreply.github.com Subject: Re: [CBIIT/R-cometsAnalytics] COMETS 1.3: "Where" functionality not working (#31)

I tested the "where" functionality and it works on my end.

`

Set up model without where

modeldata <- getModelData(exmetabdata,colvars="bmi",adjvars=c("age","race_grp","educ_grp","smk_grp","alc_grp","nested_case"),modelspec="Interactive") [1] "Analysis will run on 'All metabolites"

Run correlations without where

corrmatrix <-calcCorr(modeldata,exmetabdata, "DPP")

Set up the same model but with the where

modeldata <- getModelData(exmetabdata,colvars="bmi",adjvars=c("age","race_grp","educ_grp","smk_grp","alc_grp","nested_case"),modelspec="Interactive",where="bmi>1") [1] "Analysis will run on 'All metabolites'" [1] "Filtering subjects according to the rule(s) bmi>1 . 990 of 1000 are retained"

Run correlations on the model with where

corrmatrix.where <-calcCorr(modeldata,exmetabdata, "DPP")

First several lines of output of model without where:

corrmatrix[1:4,] cohort spec model outcomespec exposurespec 1 DPP Interactive _1_2_3_benzenetriol_sulfate_2 bmi 2 DPP Interactive _1_2_dipalmitoylglycerol bmi 3 DPP Interactive _1_2_propanediol bmi 4 DPP Interactive _1_3_7_trimethylurate bmi corr n pvalue adjvars 1 -0.01985724 1000 0.5317556 age race_grp educ_grp smk_grp alc_grp nested_case 2 0.04321913 1000 0.1733481 age race_grp educ_grp smk_grp alc_grp nested_case 3 -0.05580307 1000 0.0786620 age race_grp educ_grp smk_grp alc_grp nested_case 4 0.03067453 1000 0.3339893 age race_grp educ_grp smk_grp alc_grp nested_case outcome_uid outcome exposure_uid 1 _1_2_3_benzenetriol_sulfate_2 _1_2_3_benzenetriol_sulfate_2 bmi 2 _1_2_dipalmitoylglycerol _1_2_dipalmitoylglycerol bmi 3 _1_2_propanediol _1_2_propanediol bmi 4 _1_3_7_trimethylurate _1_3_7_trimethylurate bmi exposure 1 BMI as a continuous value 2 BMI as a continuous value 3 BMI as a continuous value 4 BMI as a continuous value

First several lines of output of model with where:

corrmatrix.where[1:4,] cohort spec model outcomespec exposurespec 1 DPP Interactive _1_2_3_benzenetriol_sulfate_2 bmi 2 DPP Interactive _1_2_dipalmitoylglycerol bmi 3 DPP Interactive _1_2_propanediol bmi 4 DPP Interactive _1_3_7_trimethylurate bmi corr n pvalue adjvars 1 -0.01435470 990 0.65289784 age race_grp educ_grp smk_grp alc_grp nested_case 2 0.04297383 990 0.17799792 age race_grp educ_grp smk_grp alc_grp nested_case 3 -0.06299329 990 0.04821453 age race_grp educ_grp smk_grp alc_grp nested_case 4 0.03844825 990 0.22820831 age race_grp educ_grp smk_grp alc_grp nested_case outcome_uid outcome exposure_uid 1 _1_2_3_benzenetriol_sulfate_2 _1_2_3_benzenetriol_sulfate_2 bmi 2 _1_2_dipalmitoylglycerol _1_2_dipalmitoylglycerol bmi 3 _1_2_propanediol _1_2_propanediol bmi 4 _1_3_7_trimethylurate _1_3_7_trimethylurate bmi exposure 1 BMI as a continuous value 2 BMI as a continuous value 3 BMI as a continuous value 4 BMI as a continuous value `

This does not look obviously wrong to me but let me know if you see something I don't? I've also attached the output here since the printed output looks ugly.

Where_vs_NoWhere.xlsxhttps://github.com/CBIIT/R-cometsAnalytics/files/1760992/Where_vs_NoWhere.xlsx

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/CBIIT/R-cometsAnalytics/issues/31#issuecomment-368721626, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhG9TdXHW86J-213ElW4Vc3EvEbZqgG_ks5tY2PCgaJpZM4STzsU.

ewymathe commented 6 years ago

Ah, I saw another bug! The package did not support "<=" and ">=" for the "where" filtering. Now it does so all should be OK. The 'where' command support ">", "<", "=", "<=", ">=" now. It's pushed onto master.

ewymathe commented 6 years ago

OK, made another push since I had not had the code grab the where in batch mode.

It's working now: `

mymod<-getModelData(exmetabdata,modlabel="BMI.2.0 Gender stratified",modelspec="Batch") [1] "Filtering subjects according to the rule(s) bmi>=1 . 990 of 1000 are retained"

corrmatrix <-calcCorr(mymod,exmetabdata, "test") [1] "running adjusted"

corrmatrix[1:4,] cohort spec model outcomespec 1 test Batch BMI.2.0 Gender stratified _1_2_3_benzenetriol_sulfate_2 2 test Batch BMI.2.0 Gender stratified _1_2_dipalmitoylglycerol 3 test Batch BMI.2.0 Gender stratified _1_2_propanediol 4 test Batch BMI.2.0 Gender stratified _1_3_7_trimethylurate exposurespec corr n pvalue 1 bmi -0.01435470 990 0.65289784 2 bmi 0.04297383 990 0.17799792 3 bmi -0.06299329 990 0.04821453 4 bmi 0.03844825 990 0.22820831 adjvars outcome_uid 1 age race_grp educ_grp smk_grp alc_grp nested_case CHEM100006374 2 age race_grp educ_grp smk_grp alc_grp nested_case HMDB07098 3 age race_grp educ_grp smk_grp alc_grp nested_case HMDB01881 4 age race_grp educ_grp smk_grp alc_grp nested_case HMDB02123 outcome exposure_uid exposure 1 1,2,3-benzenetriol sulfate (2) bmi BMI as a continuous value 2 1,2-dipalmitoylglycerol bmi BMI as a continuous value 3 1,2-propanediol bmi BMI as a continuous value 4 1,3,7-trimethylurate bmi BMI as a continuous value `

Pushed to master

steven-moore commented 6 years ago

"Where" functionality is now working in interactive mode. Issue is closed.