marl-book / codebase

Official code repo for the MARL book (www.marl-book.com)
385 stars 58 forks source link

Suggest to loosen the dependency on stable-baselines3 #13

Closed Agnes-U closed 2 years ago

Agnes-U commented 2 years ago

Hi, your project fast-marl requires "stable-baselines3==1.0" in its dependency. After analyzing the source code, we found that the following versions of stable-baselines3 can also be suitable without affecting your project, i.e., stable-baselines3 0.11.0, 0.11.1, 1.0rc2, 1.0rc1, 1.0rc0, 1.1.0a3. Therefore, we suggest to loosen the dependency on stable-baselines3 from "stable-baselines3==1.0" to "stable-baselines3>=0.11.0,<=1.1.0a3" to avoid any possible conflict for importing more packages or for downstream projects that may use fast-marl.

May I pull a request to further loosen the dependency on stable-baselines3?

By the way, could you please tell us whether such dependency analysis may be potentially helpful for maintaining dependencies easier during your development?



We also give our detailed analysis as follows for your reference:

Your project fast-marl directly uses 4 APIs from package stable-baselines3.

stable_baselines3.common.vec_env.base_vec_env.VecEnv.step, stable_baselines3.common.vec_env.dummy_vec_env.DummyVecEnv.__init__, stable_baselines3.common.vec_env.subproc_vec_env.SubprocVecEnv.__init__, stable_baselines3.common.vec_env.subproc_vec_env.SubprocVecEnv.reset

Beginning from the 4 APIs above, 11 functions are then indirectly called, including -4 stable-baselines3's internal APIs and 15 outsider APIs. The specific call graph is listed as follows (neglecting some repeated function occurrences).

[/semitable/fast-marl]
+--stable_baselines3.common.vec_env.base_vec_env.VecEnv.step
|      +--stable_baselines3.common.vec_env.base_vec_env.VecEnv.step_async
|      +--stable_baselines3.common.vec_env.base_vec_env.VecEnv.step_wait
+--stable_baselines3.common.vec_env.dummy_vec_env.DummyVecEnv.__init__
|      +--stable_baselines3.common.vec_env.base_vec_env.VecEnv.__init__
|      +--stable_baselines3.common.vec_env.util.obs_space_info
|      +--collections.OrderedDict
|      +--numpy.zeros
+--stable_baselines3.common.vec_env.subproc_vec_env.SubprocVecEnv.__init__
|      +--multiprocessing.get_all_start_methods
|      +--multiprocessing.get_context
|      +--stable_baselines3.common.vec_env.base_vec_env.CloudpickleWrapper.__init__
|      +--stable_baselines3.common.vec_env.base_vec_env.VecEnv.__init__
+--stable_baselines3.common.vec_env.subproc_vec_env.SubprocVecEnv.reset
|      +--stable_baselines3.common.vec_env.subproc_vec_env._flatten_obs
|      |      +--collections.OrderedDict
|      |      +--numpy.stack

We scan stable-baselines3's versions and observe that during its evolution between any version from [0.11.0, 0.11.1, 1.0rc2, 1.0rc1, 1.0rc0, 1.1.0a3] and 1.0, the changing functions (diffs being listed below) have none intersection with any function or API we mentioned above (either directly or indirectly called by this project).

diff: 1.0(original) 0.11.0
['stable-baselines3.her.her.HER', 'stable-baselines3.common.distributions.TanhBijector', 'stable-baselines3.common.save_util.load_from_zip_file', 'stable-baselines3.common.save_util.json_to_data', 'stable-baselines3.common.base_class.BaseAlgorithm', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm', 'stable-baselines3.dqn.dqn.DQN', 'stable-baselines3.common.distributions.TanhBijector.atanh', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm._convert_train_freq', 'stable-baselines3.common.preprocessing.maybe_transpose', 'stable-baselines3.common.base_class.BaseAlgorithm.load', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm', 'stable-baselines3.common.utils.set_random_seed', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm._setup_model', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm._setup_model', 'stable-baselines3.common.policies.BasePolicy', 'stable-baselines3.her.her.HER.load', 'stable-baselines3.common.vec_env.obs_dict_wrapper.ObsDictWrapper']

diff: 1.0(original) 0.11.1
['stable-baselines3.her.her.HER', 'stable-baselines3.common.distributions.TanhBijector', 'stable-baselines3.common.save_util.load_from_zip_file', 'stable-baselines3.common.save_util.json_to_data', 'stable-baselines3.common.base_class.BaseAlgorithm', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm', 'stable-baselines3.dqn.dqn.DQN', 'stable-baselines3.common.distributions.TanhBijector.atanh', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm', 'stable-baselines3.common.preprocessing.maybe_transpose', 'stable-baselines3.common.base_class.BaseAlgorithm.load', 'stable-baselines3.common.utils.set_random_seed', 'stable-baselines3.common.policies.BasePolicy', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm._setup_model', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm._setup_model', 'stable-baselines3.her.her.HER.load', 'stable-baselines3.common.vec_env.obs_dict_wrapper.ObsDictWrapper']

diff: 1.0(original) 1.0rc2
[](no clear difference between the source codes of two versions)

diff: 1.0(original) 1.0rc1
['stable-baselines3.her.her.HER', 'stable-baselines3.her.her.HER.load']

diff: 1.0(original) 1.0rc0
['stable-baselines3.her.her.HER', 'stable-baselines3.common.distributions.TanhBijector', 'stable-baselines3.common.save_util.load_from_zip_file', 'stable-baselines3.common.save_util.json_to_data', 'stable-baselines3.common.base_class.BaseAlgorithm', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm', 'stable-baselines3.dqn.dqn.DQN', 'stable-baselines3.common.distributions.TanhBijector.atanh', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm', 'stable-baselines3.common.preprocessing.maybe_transpose', 'stable-baselines3.common.base_class.BaseAlgorithm.load', 'stable-baselines3.common.utils.set_random_seed', 'stable-baselines3.common.policies.BasePolicy', 'stable-baselines3.common.on_policy_algorithm.OnPolicyAlgorithm._setup_model', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm._setup_model', 'stable-baselines3.her.her.HER.load', 'stable-baselines3.common.vec_env.obs_dict_wrapper.ObsDictWrapper']

diff: 1.0(original) 1.1.0a3
['stable-baselines3.common.monitor.ResultsWriter.close', 'stable-baselines3.common.vec_env.vec_transpose.VecTransposeImage', 'stable-baselines3.common.vec_env.vec_monitor.VecMonitor.step_wait', 'stable-baselines3.common.monitor.ResultsWriter.__init__', 'stable-baselines3.common.vec_env.vec_monitor.VecMonitor', 'stable-baselines3.common.monitor.Monitor.close', 'stable-baselines3.sac.sac.SAC', 'stable-baselines3.common.vec_env.vec_monitor.VecMonitor.close', 'stable-baselines3.common.monitor.ResultsWriter.write_row', 'stable-baselines3.common.monitor.ResultsWriter', 'stable-baselines3.common.vec_env.vec_extract_dict_obs.VecExtractDictObs.step_wait', 'stable-baselines3.common.vec_env.vec_extract_dict_obs.VecExtractDictObs.__init__', 'stable-baselines3.common.base_class.BaseAlgorithm', 'stable-baselines3.common.vec_env.vec_normalize.VecNormalize.step_wait', 'stable-baselines3.common.vec_env.vec_monitor.VecMonitor.__init__', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm.__init__', 'stable-baselines3.common.monitor.Monitor.reset', 'stable-baselines3.common.vec_env.vec_extract_dict_obs.VecExtractDictObs', 'stable-baselines3.dqn.dqn.DQN.train', 'stable-baselines3.sac.sac.SAC.__init__', 'stable-baselines3.common.vec_env.vec_transpose.VecTransposeImage.step_wait', 'stable-baselines3.common.vec_env.vec_normalize.VecNormalize', 'stable-baselines3.common.torch_layers.MlpExtractor', 'stable-baselines3.common.monitor.Monitor.step', 'stable-baselines3.common.vec_env.vec_extract_dict_obs.VecExtractDictObs.reset', 'stable-baselines3.common.vec_env.vec_monitor.VecMonitor.reset', 'stable-baselines3.common.monitor.Monitor.get_episode_rewards', 'stable-baselines3.common.off_policy_algorithm.OffPolicyAlgorithm', 'stable-baselines3.ddpg.ddpg.DDPG.__init__', 'stable-baselines3.td3.td3.TD3', 'stable-baselines3.common.monitor.Monitor', 'stable-baselines3.ddpg.ddpg.DDPG', 'stable-baselines3.td3.td3.TD3.__init__', 'stable-baselines3.td3.td3.TD3.train', 'stable-baselines3.dqn.dqn.DQN']

As for other packages, the APIs of collections, numpy and multiprocessing are called by stable-baselines3 in the call graph and the dependencies on these packages also stay the same in our suggested versions, thus avoiding any outside conflict.

Therefore, we believe that it is quite safe to loose your dependency on stable-baselines3 from "stable-baselines3==1.0" to "stable-baselines3>=0.11.0,<=1.1.0a3". This will improve the applicability of fast-marl and reduce the possibility of any further dependency conflict with other projects.

semitable commented 2 years ago

Yes, please do a pull request

Agnes-U commented 2 years ago

#14