search

heal-research / HeuristicLab

HeuristicLab - An environment for heuristic and evolutionary optimization
https://dev.heuristiclab.com
GNU General Public License v3.0
38 stars 16 forks source link

Importing large CSV files can lead to out of memory exceptions #1609

Closed HeuristicLab-Trac-Bot closed 12 years ago

HeuristicLab-Trac-Bot commented 13 years ago

Issue migrated from trac ticket # 1609

milestone: HeuristicLab 3.3.6 | component: Problems.DataAnalysis | priority: medium | resolution: done

2011-08-01 16:48:04: @gkronber created the issue

Because the data is stored in a large monolithic two-dimensional double array (double[,]) it can become difficult to allocate a continuous memory block of the necessary size.

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:14:13: @mkommend changed status from new to accepted

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:14:13: @mkommend changed owner from gkronber to mkommend

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:14:13: @mkommend changed milestone from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.6

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:49:15: @mkommend commented

r6740:

  • Corrected TableFileParser to handle empty rows correctly.
  • Refactored DataSet to store values in List instead of a two-dimensional array.
  • Enable importing and storing string and datetime values.
  • Changed data access methods in dataset and adapted all concerning classes.
  • Changed interpreter to store the variable values for all rows during the compilation step.
HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:49:42: @mkommend changed status from accepted to reviewing

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-12 13:49:42: @mkommend changed owner from mkommend to gkronber

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-16 08:59:20: @gkronber changed status from reviewing to readytorelease

HeuristicLab-Trac-Bot commented 13 years ago

2011-09-16 08:59:20: @gkronber commented

Reviewed r6740 and made some additional changes (see #1640)

HeuristicLab-Trac-Bot commented 12 years ago

2012-01-03 05:22:57: @s-wagner changed status from readytorelease to closed

HeuristicLab-Trac-Bot commented 12 years ago

2012-01-03 05:22:57: @s-wagner removed resolution