search

bmbolstad / affyPLM

0 stars 0 forks source link

'Realloc' could not re-allocate memory (18446744057211967488 bytes) #1

Open axitamm opened 4 years ago

axitamm commented 4 years ago

Thank you for this amazing package (and others).

I am trying to pass the fitPLM function to a Large AffyBath object (5829 arrays, using a custom CDF "hgu133plus2hsentrezgcdf" from MBNI ("BrainArray")). I am using a computer with a large amount of RAM (~2TB). While running this function, I note that my total RAM utilization never exceeds >0.5TB (or ~500MB). However, after a long while (~10-15min), I get the following error:

library(affy) expdata <- ReadAffy(cdfname = "hgu133plus2hsentrezgcdf") library(arrayQualityMetrics) pd<-prepdata(expdata, intgroup=character(0), do.logtransform=T) pd$dataPLM = fitPLM(expdata)

Error in fitPLM(expdata) : 'Realloc' could not re-allocate memory (18446744057211967488 bytes)

May you please help?

Thank you!

bmbolstad commented 4 years ago

That is an very large amount of memory that it is trying to Realloc. I'm not sure I would expect that, though to tell you the truth I have not actively looked at any of this code in many years and so might be forgetting something.

Some questions:

  1. Can you process the same size data using say rma() with the custom CDF?
  2. Can you process using fitPLM the same size data using the standard CDFenv?

On 2019-12-09 06:44, axitamm wrote:

Thank you for this amazing package (and others).

I am trying to pass the fitPLM function to a Large AffyBath object (5829 arrays, using a custom CDF "hgu133plus2hsentrezgcdf" from MBNI ("BrainArray")). I am using a computer with a large amount of RAM (~2TB). While running this function, I note that my total RAM utilization never exceeds >0.5TB (or ~500MB). However, after a long while (~10-15min), I get the following error:

library(affy) expdata <- ReadAffy(cdfname = "hgu133plus2hsentrezgcdf") library(arrayQualityMetrics) pd<-prepdata(expdata, intgroup=character(0), do.logtransform=T) pd$dataPLM = fitPLM(expdata)

Error in fitPLM(expdata) : 'Realloc' could not re-allocate memory (18446744057211967488 bytes)

May you please help?

Thank you!

-- You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].

Links:

[1] https://github.com/bmbolstad/affyPLM/issues/1?email_source=notifications&amp;email_token=ABLUKCGOVAFNWATKCNLSZTDQXZKVXA5CNFSM4JYKKQKKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H7DDY4Q [2] https://github.com/notifications/unsubscribe-auth/ABLUKCDZU66PPFKPGY4ZOATQXZKVXANCNFSM4JYKKQKA

axitamm commented 4 years ago

Thank you for your prompt reply!

Answers to your questions:

  1. Can you process the same size data using say rma() with the custom CDF? - Yes! rma generates a a quantile normalized matrix without any issues
  2. Can you process using fitPLM the same size data using the standard CDFenv? No! whether I use the custom or standard CDF, fitPLM gives me the same error.

Your help in this matter will be greatly appreciated, as I have hit a road block! Thank you!

bmbolstad commented 4 years ago

Unfortunately I do not have a good answer to you, except to say that fitPLM() is memory hungry. If you really have a machine with 2TB (rather than 2GB) of RAM there is not much I can do or suggest.

your answers to 1 and 2 rule out (or at least greatly reduced) the chances that there is something too weird with the customcdfenv

What exactly were you looking to achieve with fitPLM()?

Also what is your operating system and R version?

On 2019-12-09 15:41, axitamm wrote:

Thank you for your prompt reply!

Answers to your questions:

  • Can you process the same size data using say rma() with the custom CDF? - Yes! rma generates a a quantile normalized matrix without any issues
  • Can you process using fitPLM the same size data using the standard CDFenv? No! whether I use the custom or standard CDF, fitPLM gives me the same error.

Your help in this matter will be greatly appreciated, as I have hit a road block! Thank you!

-- You are receiving this because you commented. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].

Links:

[1] https://github.com/bmbolstad/affyPLM/issues/1?email_source=notifications&amp;email_token=ABLUKCBILR4YTK5FWRRWWFDQX3JR3A5CNFSM4JYKKQKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGLDCXQ#issuecomment-563491166 [2] https://github.com/notifications/unsubscribe-auth/ABLUKCDYDIN67LQQ2VOX7S3QX3JR3ANCNFSM4JYKKQKA

axitamm commented 4 years ago

Yes, I do have a machine with 2000MB RAM running Windows 10 and R 3.6.1.

I'm looking to generate RLE and NUSE plots and compare the distribution of error among the arrays.