search

trentool / TRENTOOL3

Open-Source MATLAB toolbox for transfer entropy estimation
http://trentool.github.io/TRENTOOL3/
GNU General Public License v3.0
56 stars 26 forks source link

A code change to reduce RAM footprint (also speed-up) #16

Closed bewantbe closed 8 years ago

bewantbe commented 8 years ago

I found that the transfer entropy functions in your toolbox consume a lot of RAM for long data (>1e5), I mean a hundred gigabyte or more.

Trace down to TEC_dsvalues.m and range_search.cpp, the code requiring some unused variable like "neighbors_p21_2": [ncount_p21_2, neighbors_p21_2] = range_search(...) Actually the code range_search.cpp treat "count only" case specially neighbors_p21_2 = range_search(...) which avoids Matlab's mysterious memory management (about the cell array).

Function TEC_dsvalues_noSourceEmb.m has the same problem.

Best wishes~

mwibral commented 8 years ago

Thanks,

this looks like a promising improvement.

Michael

On 13.11.2015 01:01, count wrote:

I found that the transfer entropy functions in your toolbox consume a lot of RAM for long data (>1e5), I mean a hundred gigabyte or more.

Trace down to TEC_dsvalues.m and range_search.cpp, the code requiring some unused variable like "neighbors_p21_2": [ncount_p21_2, neighbors_p21_2] = range_search(...) Actually the code range_search.cpp treat "count only" case specially neighbors_p21_2 = range_search(...) which avoids Matlab's mysterious memory management (about the cell array).

Function TEC_dsvalues_noSourceEmb.m has the same problem.

Best wishes~

— Reply to this email directly or view it on GitHub https://github.com/trentool/TRENTOOL3/issues/16.Web Bug from https://github.com/notifications/beacon/AIqYGgk2rvdz1vpOzJljehrtj0q8zqpWks5pFR_UgaJpZM4GhfT_.gif

pwollstadt commented 8 years ago

@bewantbe Thank you for the suggestion, I removed the unnecessary output variables in both functions TEC_dsvalues.m and TEC_dsvalues_noSourceEmb.m.