Open lumyx2 opened 7 years ago
This sounds like a bad bug in the matrix class, I'll look into this now.
Can you give me a better idea of what you are doing to cause this error so I can recreate it. How big are the files you are trying to load? I'll try loading some really large CSV files and see if I can recreate the memory leaks.
As i said, there is the instant crash when loading very large files and then there is progressive leaks that creep in.
For the instant crash, I happens very liably when I try to just load a CSV with 576 dimensions and of a size of over 200MB.
The progressive crash is a lot more elusive. Basically some memory is not released correctly, but it's difficult to pin point where. It tends to happens when while of going through my loop. I noticed that when the models generated are larger, it tends to crash faster. I say this because I have been doing a lot of grid searches for various different size files and the crashes are very dependant of file sizes.
M
On 18 October 2016 at 06:30, Nicholas Gillian notifications@github.com wrote:
This sounds like a bad bug in the matrix class, I'll look into this now.
Can you give me a better idea of what you are doing to cause this error so I can recreate it. How big are the files you are trying to load? I'll try loading some really large CSV files and see if I can recreate the memory leaks.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nickgillian/grt/issues/102#issuecomment-254410721, or mute the thread https://github.com/notifications/unsubscribe-auth/AHyvO9RDD8L7FkyZDkXx18H7Gr_W8LVFks5q1FldgaJpZM4KYglG .
@lumyx2 @nickgillian same with me here. I am getting “Invalid Allocation Size: 4294967295 Bytes” when i try to load a .csv file of 35 MB with 17 columns. But when i reduce the file size to 746KB it works for DTW but for only single class. Any solutions yet?
Hi Nick,
There seems to be something funny with the memory. I notice that very large files crash it instantly. It basically fails to load them.
Also, I notice that it seems to leak memory. After building 100s of models, the program crashes with bad_alloc. If I make it resume from where it was, it runs fine till it crashes again later on. I can't seem to see anything wrong with my code.
I basically have something along the lines of:
for (i -> end) classificationdata testd; load testd for (j -> end) classificationdata traind; load traind; train using traind and testd;
Everything is on the stack.
Cheers, M