What is the difference between samplers.CmaEsSampler and integration.PyCmaSampler

[fsmosca]

CmaEsSampler

https://optuna.readthedocs.io/en/stable/reference/generated/optuna.samplers.CmaEsSampler.html based on https://github.com/CyberAgentAILab/cmaes

PyCmaSampler

https://optuna.readthedocs.io/en/stable/reference/generated/optuna.integration.PyCmaSampler.html based on: http://cma.gforge.inria.fr/apidocs-pycma/cma.html

Was there any performance test comparison done with these two libraries?

I plan to do an extensive performance test comparing this two in game parameter tuner. Currently I use PyCmaSampler in the game parameter tuner.

[himkt]

Hello @fsmosca, please consider using optuna.samplers.CmaEsSampler because we deprecated optuna.integration.PyCmaSampler.

You can see the performance comparison in cmaes repository. There are small performance differences between them. https://github.com/CyberAgentAILab/cmaes#benchmark-results

@c-bata Please comment if you have any additional information. :pray:

[fsmosca]

According to the doc optuna.integration.PyCmaSampler is not deprecated.

But there is a note:

CmaEsSampler is deprecated and renamed to PyCmaSampler in v2.0.0. Please use PyCmaSampler instead of CmaEsSampler.

[himkt]

@fsmosca I'm sorry, you're right. Thank you for pointing out my mistake.

Let me update the answer as follows:

• We recommend optuna.samplers.CmaEsSampler, because it is faster than optuna.integration.PyCmaSampler
• There is not much difference in performance (ref. https://github.com/CyberAgentAILab/cmaes#benchmark-results)
• We might deprecate optuna.integration.PyCmaSampler in the future (it is not decided), because we focus on maintaining and improving optuna.samplersCmaEsSampler`

[fsmosca]

All right so I will only use CmaEsSampler then and cancel my planned comparison. Thank you.