search

aimclub / GOLEM

Graph Optimiser for Learning and Evolution of Models
https://thegolem.readthedocs.io
BSD 3-Clause "New" or "Revised" License
60 stars 7 forks source link

Serialize: Add light-weight OptHistory saving #118

Closed gkirgizov closed 10 months ago

gkirgizov commented 1 year ago

Currently OptHistories can be very large objects, with a size > 1Gb e.g. for molecular search or any graphs with large state.

It's quite unmanagable if we just want to have, say, 10 trials and build comparison between 2 cases with 10 trials each.

So we need a light-weight history saving that doesn't save graphs, and stores only fitness & individuals id. Requirements are:

nicl-nno commented 1 year ago

А не будут ли такие 1Gb-истории хорошо сжиматься в zip?

gkirgizov commented 1 year ago

А не будут ли такие 1Gb-истории хорошо сжиматься в zip?

Нужно проверить

MorrisNein commented 1 year ago

Можно отказаться от аргумента indent, чтобы сэкономить немного места.