search

aimclub / GOLEM

Graph Optimiser for Learning and Evolution of Models
https://thegolem.readthedocs.io
BSD 3-Clause "New" or "Revised" License
63 stars 7 forks source link

Parallel optimizer #247

Open kasyanovse opened 11 months ago

kasyanovse commented 11 months ago

WIP

Продолжение работы https://github.com/aimclub/GOLEM/pull/199.

Этап 1. Воссоздание существующего генетического оптимизатора с использованием нового подхода

Этап 2. Добавление бандитов для мутаций

pep8speaks commented 11 months ago

Hello @kasyanovse! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 1:1: F401 'abc.abstractmethod' imported but unused Line 2:1: F401 'collections.deque' imported but unused Line 3:1: F401 'dataclasses.dataclass' imported but unused Line 4:1: F401 'multiprocessing.managers.DictProxy' imported but unused Line 5:1: F401 'typing.Dict' imported but unused Line 5:1: F401 'typing.List' imported but unused Line 5:1: F401 'typing.Callable' imported but unused Line 10:1: F401 'golem.core.optimisers.common_optimizer.node.Node' imported but unused Line 13:1: F401 'golem.core.optimisers.common_optimizer.scheme.Scheme' imported but unused Line 14:1: F401 'golem.core.optimisers.common_optimizer.stage.Stage' imported but unused Line 15:1: F401 'golem.core.optimisers.common_optimizer.task.Task' imported but unused Line 15:1: F401 'golem.core.optimisers.common_optimizer.task.TaskStatusEnum' imported but unused Line 17:1: F401 'golem.core.optimisers.graph.OptGraph' imported but unused Line 18:1: F401 'golem.core.optimisers.objective.ObjectiveFunction' imported but unused Line 20:1: F401 'golem.core.optimisers.opt_history_objects.opt_history.OptHistory' imported but unused Line 22:1: F401 'golem.core.optimisers.optimizer.GraphOptimizer' imported but unused Line 24:1: F401 'golem.core.optimisers.timer.OptimisationTimer' imported but unused

Line 1:1: F401 'abc.abstractmethod' imported but unused Line 2:1: F401 'collections.deque' imported but unused Line 4:1: F401 'typing.Callable' imported but unused Line 7:1: F401 'golem.core.optimisers.common_optimizer.node.Node' imported but unused Line 8:1: F401 'golem.core.optimisers.common_optimizer.old_config.default_stages' imported but unused Line 9:1: F401 'golem.core.optimisers.common_optimizer.scheme.Scheme' imported but unused Line 10:1: F401 'golem.core.optimisers.common_optimizer.stage.Stage' imported but unused Line 11:1: F401 'golem.core.optimisers.common_optimizer.task.Task' imported but unused Line 11:1: F401 'golem.core.optimisers.common_optimizer.task.TaskStatusEnum' imported but unused Line 12:1: F401 'golem.core.optimisers.genetic.operators.operator.EvaluationOperator' imported but unused Line 13:1: F401 'golem.core.optimisers.graph.OptGraph' imported but unused Line 14:1: F401 'golem.core.optimisers.objective.ObjectiveFunction' imported but unused Line 15:1: F401 'golem.core.optimisers.opt_history_objects.individual.Individual' imported but unused Line 18:1: F401 'golem.core.optimisers.optimizer.GraphOptimizer' imported but unused Line 19:1: F401 'golem.core.optimisers.populational_optimizer.PopulationalOptimizer' imported but unused Line 20:1: F401 'golem.core.optimisers.timer.OptimisationTimer' imported but unused Line 39:1: W391 blank line at end of file

Line 3:1: F401 'copy.deepcopy' imported but unused Line 4:1: F401 'enum.Enum' imported but unused Line 4:1: F401 'enum.auto' imported but unused Line 8:1: F401 'numpy as np' imported but unused Line 9:1: F401 'typing.Optional' imported but unused Line 9:1: F401 'typing.Tuple' imported but unused Line 9:1: F401 'typing.List' imported but unused Line 31:1: E303 too many blank lines (3) Line 40:1: E302 expected 2 blank lines, found 1 Line 49:1: E302 expected 2 blank lines, found 1 Line 52:1: E305 expected 2 blank lines after class or function definition, found 1 Line 75:1: E302 expected 2 blank lines, found 1 Line 86:1: E302 expected 2 blank lines, found 1 Line 98:1: E305 expected 2 blank lines after class or function definition, found 1 Line 114:1: E302 expected 2 blank lines, found 0 Line 118:1: E305 expected 2 blank lines after class or function definition, found 1 Line 125:1: E303 too many blank lines (3) Line 149:1: W391 blank line at end of file

Line 11:36: F821 undefined name 'CommonOptimizerParameters' Line 18:45: F821 undefined name 'CommonOptimizerParameters'

Line 3:1: F401 'golem.core.optimisers.genetic.operators.mutation.Mutation as OldMutation' imported but unused Line 7:36: F821 undefined name 'CommonOptimizerParameters' Line 12:45: F821 undefined name 'CommonOptimizerParameters' Line 15:1: E305 expected 2 blank lines after class or function definition, found 1 Line 15:1: E402 module level import not at top of file Line 15:1: F401 'golem.core.optimisers.fitness.null_fitness' imported but unused Line 16:1: E302 expected 2 blank lines, found 0

Line 7:1: F401 'golem.core.optimisers.common_optimizer.task.Task' imported but unused Line 8:1: F401 'golem.core.optimisers.genetic.operators.base_mutations.MutationTypesEnum' imported but unused Line 15:36: F821 undefined name 'CommonOptimizerParameters' Line 22:45: F821 undefined name 'CommonOptimizerParameters'

Line 7:36: F821 undefined name 'CommonOptimizerParameters' Line 14:45: F821 undefined name 'CommonOptimizerParameters'

Line 7:1: E303 too many blank lines (3) Line 8:36: F821 undefined name 'CommonOptimizerParameters' Line 14:45: F821 undefined name 'CommonOptimizerParameters'

Line 6:1: E302 expected 2 blank lines, found 1 Line 7:36: F821 undefined name 'CommonOptimizerParameters' Line 13:45: F821 undefined name 'CommonOptimizerParameters'

Line 7:36: F821 undefined name 'CommonOptimizerParameters' Line 14:45: F821 undefined name 'CommonOptimizerParameters'

Line 7:1: E303 too many blank lines (3) Line 8:36: F821 undefined name 'CommonOptimizerParameters' Line 15:45: F821 undefined name 'CommonOptimizerParameters'

Line 7:1: E303 too many blank lines (3) Line 8:36: F821 undefined name 'CommonOptimizerParameters' Line 13:45: F821 undefined name 'CommonOptimizerParameters'

Line 8:36: F821 undefined name 'CommonOptimizerParameters' Line 14:45: F821 undefined name 'CommonOptimizerParameters'

Line 1:1: F401 'copy.deepcopy' imported but unused Line 2:1: F401 'random.choice' imported but unused Line 4:1: F401 'typing.Optional' imported but unused Line 4:1: F401 'typing.Dict' imported but unused Line 4:1: F401 'typing.Callable' imported but unused Line 7:1: F401 'golem.core.optimisers.common_optimizer.task.Task' imported but unused Line 8:1: F401 'golem.core.optimisers.genetic.operators.base_mutations.base_mutations_repo' imported but unused Line 8:1: F401 'golem.core.optimisers.genetic.operators.base_mutations.MutationTypesEnum' imported but unused Line 9:1: F401 'golem.core.optimisers.graph.OptGraph' imported but unused Line 10:1: F401 'golem.core.optimisers.opt_history_objects.individual.Individual' imported but unused Line 11:1: F401 'golem.core.optimisers.opt_history_objects.parent_operator.ParentOperator' imported but unused Line 15:36: F821 undefined name 'CommonOptimizerParameters' Line 20:45: F821 undefined name 'CommonOptimizerParameters' Line 47:1: W391 blank line at end of file

Line 28:1: E302 expected 2 blank lines, found 0 Line 32:1: E305 expected 2 blank lines after class or function definition, found 1 Line 37:1: E302 expected 2 blank lines, found 1 Line 39:45: F821 undefined name 'CommonOptimizerParameters' Line 43:1: E305 expected 2 blank lines after class or function definition, found 1 Line 54:1: E302 expected 2 blank lines, found 1 Line 61:1: E302 expected 2 blank lines, found 1 Line 66:1: E305 expected 2 blank lines after class or function definition, found 1 Line 74:45: F821 undefined name 'CommonOptimizerParameters' Line 78:1: E305 expected 2 blank lines after class or function definition, found 1 Line 81:1: E302 expected 2 blank lines, found 1 Line 85:1: E305 expected 2 blank lines after class or function definition, found 1 Line 88:1: W391 blank line at end of file

Line 3:1: F401 'collections.deque' imported but unused Line 4:1: F401 'copy.copy' imported but unused Line 38:19: F821 undefined name 'Queue' Line 39:22: F821 undefined name 'Queue' Line 97:17: F841 local variable 'gen' is assigned to but never used

Line 6:1: F401 'golem.utilities.utilities.determine_n_jobs' imported but unused

Line 33:46: F821 undefined name 'CommonOptimizerParameters' Line 33:76: F821 undefined name 'CommonOptimizerParameters' Line 35:31: F821 undefined name 'CommonOptimizerParameters'

Line 4:1: F401 'typing.Any' imported but unused Line 18:45: F821 undefined name 'CommonOptimizerParameters' Line 26:1: E303 too many blank lines (3) Line 39:45: F821 undefined name 'CommonOptimizerParameters'

Line 1:1: F401 'typing.Any' imported but unused Line 1:1: F401 'typing.Dict' imported but unused Line 5:1: F401 'golem.core.optimisers.optimizer.OptimizationParameters' imported but unused Line 5:1: F401 'golem.core.optimisers.optimizer.GraphGenerationParams' imported but unused Line 5:1: F401 'golem.core.optimisers.optimizer.AlgorithmParameters' imported but unused Line 11:1: F401 'golem.core.optimisers.genetic.operators.mutation.Mutation as OldMutation' imported but unused Line 21:36: F821 undefined name 'CommonOptimizerParameters' Line 30:45: F821 undefined name 'CommonOptimizerParameters' Line 30:77: F821 undefined name 'CommonOptimizerParameters'

Comment last updated at 2023-12-14 12:40:45 UTC
codecov-commenter commented 11 months ago

Codecov Report

Attention: 387 lines in your changes are missing coverage. Please review.

Comparison is base (46b53a4) 71.94% compared to head (16a7a3e) 68.57%. Report is 2 commits behind head on main.

Files Patch % Lines
golem/core/optimisers/common_optimizer/runner.py 0.00% 78 Missing :warning:
golem/core/optimisers/common_optimizer/config.py 0.00% 77 Missing :warning:
...lem/core/optimisers/common_optimizer/old_config.py 0.00% 57 Missing :warning:
...re/optimisers/common_optimizer/common_optimizer.py 0.00% 54 Missing :warning:
golem/core/optimisers/common_optimizer/task.py 0.00% 35 Missing :warning:
...misers/common_optimizer/common_optimizer_params.py 0.00% 32 Missing :warning:
golem/core/optimisers/common_optimizer/scheme.py 0.00% 21 Missing :warning:
golem/core/optimisers/common_optimizer/stage.py 0.00% 18 Missing :warning:
golem/core/optimisers/common_optimizer/node.py 0.00% 15 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #247 +/- ## ========================================== - Coverage 71.94% 68.57% -3.37% ========================================== Files 136 145 +9 Lines 8127 8532 +405 ========================================== + Hits 5847 5851 +4 - Misses 2280 2681 +401 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

kasyanovse commented 1 month ago

ИМХО, этот PR на стадии MVP. Не хватает точных замеров прироста производительности в разных задачах, адаптированных под этот алгоритм мутаций (сейчас тут реализованы обертки) и нормальной системы кеширования.

А вообще, возможностей для дальнейшего развития тут так много, что проще с нуля написать.