Cannot allocate of vector of size ...

[ebecht]

Hey Etienne, I thought it is easier to ask my question here again.

I am now in the training session and constantly get this error during the Linear and polynomial models part. Error: cannot allocate vector of size 401.6 MB. Do you have an advice of how to solve it?

Much appreciated, Steven

Originally posted by @sschepanski in https://github.com/ebecht/infinityFlow/issues/3#issuecomment-950124378

[ebecht]

Hi @sschepanski

I'm transferring this to a new issue with an informative title for the question, this will help future users that run into the same issues find a relevant solution or answer.

In this case it means you are running out of RAM. There are a few options you can explore to help with this :

• Free up memory (closing other programs, cleaning up your R session),
• Lowering input events (downsampling)
• Playing with regression models' hyperparameters (e.g. decrease nrounds and slightly increase eta for xgboost)
• Using a computer with more RAM

Polynomial models with degrees >= 2 are very memory intensive and not that useful so I would advise against using them. In our paper we used them mostly for benchmarking purposes.

How much RAM do you have access to? The dataset in the training is very small, if you are running into this issue there you may have problems processing real life datasets.

There is an another ongoing discussion to reduce memory usage but that necessitates quite a bit of rewriting of the code so I don't know when it'll be implemented :-/

Best, Etienne

[sschepanski]

Hey Etienne, I am working on a computer with 256 GB RAM, so I do not think this might be a problem. I will then try out all of your suggestions and hope this will somehow work then out.

Thanks for your feedback, I am sure this will help me a lot. Best, Steven

[denvercal1234GitHub]

You are certain that 256GB is the RAM and not the storage? 256GB seemed like a lot for RAM.