tpq / exprso

[DEPRECATED] An R package to rapidly build and deploy supervised machine learning workflows
16 stars 6 forks source link

t-test tryCatch issue #1

Closed tpq closed 8 years ago

tpq commented 8 years ago

Should embed tryCatch statement within fsStats to return p(x)=1 instead of throwing error and crashing script.

dantylee commented 8 years ago

R version 3.3.0 (2016-05-03) -- "Supposedly Educational"

The latest exprso release fixed this issue very nicely when calling plMonteCarlo + T-test + SVM/ANN (and thank you for that! ), but when I combine plMonteCarlo + T-Test with RF, the console readout freezes with the first plMonteCarlo call following and the analysis does not complete.

fsStats failed for feature: ZNF214 . Setting p(x)=1... fsStats failed for feature: ZNF385D . Setting p(x)=1... fsStats failed for feature: ZPLD1 . Setting p(x)=1... fsStats failed for feature: ZSCAN10 . Setting p(x)=1... Now building machine at gridpoint: probes ntree 1 2 500

(frozen w/ stop sign, but not completing in the anticipated amount of time and R session terminates on clicking stop. Does not freeze here with SVM or ANN. Also plMonteCarlo does not freeze when RF combined with KS-test or MRMRE)

tpq commented 8 years ago

Thanks for the feedback! This is really peculiar. I cannot think of any reason right now why t-test => SVM AND ks-test => RF would succeed, but t-test => RF would fail.

If I had to guess, it may be because the top two features selected by t-test (when probes = 2) is insufficient input for RF? I wonder whether the model will succeed if you change the ctrlGridSearch probes argument to at least 3/4/5 instead of 2?

dantylee commented 8 years ago

Hey Thom, Quick response, RD had worked fine with 2 features (as supplied by the probeCount object ) many many times in the past....

How are you doing? Have you started the new Phd program?

Dan

On Wednesday, September 21, 2016, Thom Quinn notifications@github.com wrote:

Thanks for the feedback! This is really peculiar. I cannot think of any reason right now why t-test => SVM AND ks-test => RF would succeed, but t-test => RF would fail.

If I had to guess, it may be because the top two features selected by t-test (when probes = 2) is insufficient input for RF? I wonder whether the model will succeed if you change the ctrlGridSearch probes argument to at least 3/4/5 instead of 2?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tpq/exprso/issues/1#issuecomment-248775108, or mute the thread https://github.com/notifications/unsubscribe-auth/AMHfCpHLsAiia1R1CuNmgCar6gsObfy4ks5qscEpgaJpZM4J0beK .

Sent from Gmail Mobile

tpq commented 8 years ago

Ah, sorry. I should have been more clear-- I was wondering if the problem is those 2 features specifically. It's hard for me to conceive otherwise why it might throw an error, but if the data files are small enough, you could send them to me by e-mail (along with the script) and I'll dig deeper into the problem. It is possible that this particular combination has revealed a very rare but serious bug that I need to track down and fix.

Otherwise, things are good here! Officially, I start Monday, but I've kept busy these past few weeks tidying up some software. Right now, I am shooting to get exprso on CRAN by the end of October, and will look into writing up a small manuscript for announcing its birth.

Any word on whether Jon's still going to set us all up with our very own "Slack" channel? I would enjoy having an informal means like that to keep in touch about science and more.

Cheers -tpq

dantylee commented 8 years ago

Hey Thom,

Ah okay, now I understand. I suppose it could be specific to the data I'm using, but because plMonteCarlo is resampling the data differently each time I try it, then I would expect that the top 2 t-test features should change each time. Will bundle and send R/data files in a few moments. Exciting! Can't wait to point people toward a very official CRAN page. And I'll be looking forward to learning about the projects you take on as grad student!

Our lab's slack is here: https://psychgene.slack.com I'm not sure if you need to be invited (I can't seem to find a button for inviting you....) or maybe you can request to join? It looks like Jon is making good use of it. I'm going to start sharing my projects there soon.

Best wishes, Dan

Daniel Tylee, BA MD/PhD Candidate Psychiatric Genetic Epidemiology & Neurobiology Laboratory (PsychGENe Lab) Cell: 631-375-2588

SUNY Upstate Medical University 750 East Adams Street Room 3738B, Neuroscience Research Building Institute for Human Performance Syracuse, NY 13210 Website: www.psychgenelab.com

On Thu, Sep 22, 2016 at 4:22 AM, Thom Quinn notifications@github.com wrote:

Ah, sorry. I should have been more clear-- I was wondering if the problem is those 2 features specifically. It's hard for me to conceive otherwise why it might throw an error, but if the data files are small enough, you could send them to me by e-mail (along with the script) and I'll dig deeper into the problem. It is possible that this particular combination has revealed a very rare but serious bug that I need to track down and fix.

Otherwise, things are good here! Officially, I start Monday, but I've kept busy these past few weeks tidying up some software. Right now, I am shooting to get exprso on CRAN by the end of October, and will look into writing up a small manuscript for announcing its birth.

Any word on whether Jon's still going to set us all up with our very own "Slack" channel? I would enjoy having an informal means like that to keep in touch about science and more.

Cheers -tpq

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tpq/exprso/issues/1#issuecomment-248840697, or mute the thread https://github.com/notifications/unsubscribe-auth/AMHfCl4hpqj3P2W9yko3pxGPfbdA_FInks5qsjrMgaJpZM4J0beK .