search

heal-research / HeuristicLab

HeuristicLab - An environment for heuristic and evolutionary optimization
https://dev.heuristiclab.com
GNU General Public License v3.0
31 stars 14 forks source link

Persistence Overhaul #2520

Closed HeuristicLab-Trac-Bot closed 4 years ago

HeuristicLab-Trac-Bot commented 8 years ago

Issue migrated from trac ticket # 2520

milestone: HeuristicLab 3.3.16 | component: Persistence | priority: high | resolution: done | keywords: merged

2015-11-20 22:45:31: @s-wagner created the issue

The HeuristicLab persistence layer should be overhauled for HeuristicLab 4.0.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-20 22:47:23: @s-wagner commented

r13323: Created new branch for persistence overhaul

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-20 23:20:24: @s-wagner commented

r13324 (not migrated): Removed new empty branch in order to create branch of full trunk

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-20 23:22:31: @s-wagner commented

r13325: Created branch PersistenceOverhaul of trunk r13321.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-22 23:31:23: @s-wagner commented

r13326: Created plugin for new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-23 18:47:52: @s-wagner commented

r13347: Worked on new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 15:22:38: @s-wagner commented

r13358: Worked on new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 17:00:24: @s-wagner commented

r13367: Worked on new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 17:08:30: @Shabbafru commented

r13368 added guids to storable classes

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 17:10:54: @Shabbafru commented

r13370 fixed unit tests

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 17:29:46: @s-wagner commented

r13375: Worked on new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 18:24:20: @Shabbafru commented

r13386

  • fixed duplicate guids
  • adapted/added unit tests for new persistence
HeuristicLab-Trac-Bot commented 8 years ago

2015-11-24 18:33:35: @s-wagner commented

r13387: Worked on new persistence implementation.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-26 17:07:37: @Shabbafru commented

r13407 worked on type and enum transformers

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-27 14:23:17: @Shabbafru commented

r13409

  • added profiling to unit tests
  • added more unit tests
HeuristicLab-Trac-Bot commented 8 years ago

2015-11-27 14:28:16: @Shabbafru commented

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851
HeuristicLab-Trac-Bot commented 8 years ago

2015-11-27 14:55:47: @gkronber commented

Replying to [comment:15 ascheibe]:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-27 14:59:48: @Shabbafru commented

r13410 increased repetitions to get more realistic results

HeuristicLab-Trac-Bot commented 8 years ago

2015-11-27 15:04:13: @Shabbafru commented

Replying to [comment:16 gkronber]:

Replying to [comment:15 ascheibe]:

The Persistence4 unit tests now generate an output where you can see different performance measures, e.g.:

Performance Report for <TestBoolArray>b__28_0: 

Avg. old vs. new time for serializing a file: 00:00:00.0060189 / 00:00:00.0008129; Factor: 7,40452962958407
Avg. old vs. new time for deserializing a file: 00:00:00.0048589 / 00:00:00.0001852; Factor: 26,2386866832271
Avg. old vs. new file size (in bytes): 594 / 55; Factor: 10,8
Avg. old vs. new memory consumption for serializing a file (in bytes): 59849,6 / 24600; Factor: 2,43291056910569
Avg. old vs. new memory consumption for deserializing a file (in bytes): 178737,6 / 57392; Factor: 3,11432952327851

Micro-benchmark alarm. I'd be more interested to see how long it takes to store and load a full experiment with results.

Yeah, I know that this is not significant, but we don't have composite transformers yet, so this is all we can do at the moment ;-)

HeuristicLab-Trac-Bot commented 7 years ago

2017-01-03 18:22:41: @gkronber commented

r14537: worked on persistence

HeuristicLab-Trac-Bot commented 7 years ago

2017-01-05 17:37:52: @jkarder commented

r14549: worked on persistence

HeuristicLab-Trac-Bot commented 7 years ago

2017-01-20 11:16:47: @jkarder commented

r14594: worked on persistence

HeuristicLab-Trac-Bot commented 7 years ago

2017-03-03 12:00:59: @gkronber commented

r14711

  • renamed StorableClass -> StorableType
  • changed persistence to use GUIDs instead of type names
HeuristicLab-Trac-Bot commented 7 years ago

2017-03-03 17:33:36: @gkronber commented

r14712: added GUIDs for (almost) all interface types (probably still too many) also added newlines at end of all files

HeuristicLab-Trac-Bot commented 7 years ago

2017-03-06 10:19:52: @gkronber commented

r14714 added missing StorableType attributes and registered more types. All samples can be deserialized and serialized now

HeuristicLab-Trac-Bot commented 7 years ago

2017-03-08 10:49:31: @gkronber commented

r14731: added tool program which adds StorableType attributes to interfaces and enums

HeuristicLab-Trac-Bot commented 7 years ago

2017-03-09 13:54:46: @gkronber commented

r14739: added a sketch for a unit test for conversion (and set svn:ignore properties)

HeuristicLab-Trac-Bot commented 7 years ago

2017-03-20 15:57:33: @gkronber commented

r14771: added versions to storable types and implemented conversion unit test

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:05:05: @gkronber commented

r14919: created a branch for reintegration

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:24:04: @gkronber commented

r14920 (not migrated): deleted branch of trunk

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:27:02: @gkronber commented

r14921: created a new branch from stable for reintegration

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:51:18: @gkronber commented

r14922 (not migrated): deleted branch again

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:52:38: @gkronber commented

r14923: created branch from trunk (again)

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 09:59:29: @gkronber commented

r14924: copied relevant files from persistence overhaul branch

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 12:30:59: @gkronber commented

r14925: moved storable attributes (construction)

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 17:21:53: @gkronber commented

r14927: changed all usages of StorableClass to use StorableType with an auto-generated GUID (did not add StorableType to other type definitions yet)

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 18:16:32: @gkronber commented

r14928: added unit tests and fixed all test cases for old persistence

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 19:07:21: @gkronber commented

r14929: fixed all unit tests for new persistence - loading & storing all samples

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 19:53:45: @gkronber commented

r14930: changed optimizer to use new persistence per default

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 19:54:24: @gkronber commented

r14931: do not store graph visualization info for the operatorgraph

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 20:18:03: @gkronber commented

r14932: check if results are the same for the GA TSP sample test

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 20:20:26: @gkronber commented

r14933: added a code fix for generating StorableTypeAttributes

HeuristicLab-Trac-Bot commented 7 years ago

2017-05-04 20:21:08: @gkronber commented

r14934: svn:ignore

HeuristicLab-Trac-Bot commented 6 years ago

2017-05-05 16:03:09: @gkronber commented

r14935: support persistence for OperatorGraphVisualizationInfo

HeuristicLab-Trac-Bot commented 6 years ago

2017-05-05 16:03:15: @gkronber changed status from new to accepted

HeuristicLab-Trac-Bot commented 6 years ago

2017-05-05 16:03:15: @gkronber changed owner from @s-wagner to @gkronber

HeuristicLab-Trac-Bot commented 6 years ago

2017-05-16 09:27:09: @jkarder commented

r14985: worked on persistence code fix

  • split existing analyzer and code fix
  • added analyzer and code fix for missing storable ctors
HeuristicLab-Trac-Bot commented 6 years ago

2017-05-16 15:56:47: @jkarder commented

r14989: worked on persistence code fix

  • added SequentialFixAllProvider
HeuristicLab-Trac-Bot commented 6 years ago

2017-05-16 16:56:46: @jkarder commented

r14990: worked on persistence code fix

  • added missing UnitTests.cs
HeuristicLab-Trac-Bot commented 6 years ago

2017-06-01 15:15:59: @gkronber commented

  • r15018: introduced StorableConstructorFlag type for StorableConstructors
  • r15020: fixed unit tests checking for exceptions
HeuristicLab-Trac-Bot commented 6 years ago

2017-06-02 16:30:57: @jkarder commented

r15022: removed ColorTransformer for now as it only handled the color's argb values (StructTransformer will handle more, e.g. IsEmpty)