Open 1-Bart-1 opened 7 months ago
Hello, thanks for the bug report and proposed solution. I'm not sure if we can do much from the SB3 side as it seems to be a very specific usecase.
You could add the option for using DillWrapper instead of CloudpickleWrapper as the wrapper for a SubprocVecEnv.
ons. 24. jan. 2024, 13:19 skrev Antonin RAFFIN @.***>:
Hello, thanks for the bug report and proposed solution. I'm not sure if we can do much from the SB3 side as it seems to be a very specific usecase.
— Reply to this email directly, view it on GitHub https://github.com/DLR-RM/stable-baselines3/issues/1814#issuecomment-1908016368, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBR55DEPWU5L44UIW75M53YQD33JAVCNFSM6AAAAABCIUXSPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGAYTMMZWHA . You are receiving this because you authored the thread.Message ID: @.***>
Could you please run python -c 'import stable_baselines3 as sb3; sb3.get_system_info()'
and also give the version of dill that was tested?
I downloaded stable-baselines from https://github.com/DLR-RM/stable-baselines3 at master.
pip show dill | grep Version Version: 0.3.7
On Mon, Jan 29, 2024 at 1:21 PM Antonin RAFFIN @.***> wrote:
Could you please run python -c 'import stable_baselines3 as sb3; sb3.get_system_info()' and also give the version of dill that was tested?
— Reply to this email directly, view it on GitHub https://github.com/DLR-RM/stable-baselines3/issues/1814#issuecomment-1914584093, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIBR55AZQMG6I2SA6NFCRD3YQ6H6NAVCNFSM6AAAAABCIUXSPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJUGU4DIMBZGM . You are receiving this because you authored the thread.Message ID: @.***>
🐛 Bug
When running model.learn on a SubprocVecEnv as follows:
env = make_vec_env(ENV_ID, n_envs=cpus, vec_env_cls=SubprocVecEnv, vec_env_kwargs=dict(start_method="spawn")) model = SAC(**kwargs, env=env) model.learn(N_TIMESTEPS, callback=eval_callback)
the program ends up in a deadlock. This is likely because I am using a custom environment which is running julia code by using juliacall. The solution is to change CloudpickleWrapper to a DillWrapper as follows:
in stable_baselines3.common.vec_env.base_vec_env.py ` import dill class DillWrapper: def init(self, var: Any): self.var = var
`
And use DillWrapper instead of CloudpickleWrapper in stable_baselines3.common.vec_env.subproc_vec_env.py By doing this, the problem seems to be less often, but not completely solved.
I am running the code with the following command: apptainer exec \ --env PYTHON_JULIACALL_SYSIMAGE=/cluster/home/bartva/BOAT/Simulation/Kite/trainer/.JlEnv.so \ --bind /cluster/work/bartva \ --nv \ .kite-app.sif python -u BOAT/Simulation/Kite/trainer/hyperparam-tuning.py \ --trials 500 \ --startup_trials 5 \ --evaluations 5 \ --steps 200000 \ --episodes 10 \ --cpus 0 \ --verbose 2 \
Code example
KiteEnv.py:
hyperparam-tuning.py:
Environment.jl (Has to be built as a custom package!)
Relevant log output / Error message
System Info
I am running the code in an Ubuntu Apptainer on a High Performance Cluster: https://www.hpc.ntnu.no/idun/
Checklist