search

hubbs5 / or-gym

Environments for OR and RL Research
MIT License
371 stars 93 forks source link

ERROR: ray::RolloutWorker.__init__() #19

Open bertanimre opened 2 years ago

bertanimre commented 2 years ago

Hello, I would like to use or-gym supply chain environments for my project. I am trying to learn the environments now.

While following the "Using Ray and DFO to optimize a multi-echelon supply chain" tutorial, I faced an error called "-- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.init()".

I didn't understand the reason for the error because I followed the steps without changing them. Could you check it?

Thanks in advance.

Best regards, Bertan

hubbs5 commented 2 years ago

There was a breaking change made to Ray that we're working to address. What version are you running? Can you post the traceback?

bertanimre commented 2 years ago

Hello, The Ray version is 1.10.0 I attached the screenshot of the problem.

Best regards, Bertan Error

wojnarabc commented 1 year ago

Hello, facing similar issue with ray 2.0.0 while trying to run inv-management-quickstart.ipynb with InvManagement-v0/v1. Tutorial mentioned that 1.0.0 ray version should be used (runs fine there) but this one is already more than two years old. Is it something easily adjustable on our end ? Below my traceback:

`Exception has occurred: AssertionError The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=11044, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7fe7f1a01970>) File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in init self._build_policy_map( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map self.policy_map.create_policy( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy self[policy_id] = create_policy_for_framework( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework return policy_class( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/algorithms/ppo/ppo_tf_policy.py", line 83, in init base.init( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 93, in init ) = self._init_action_fetches(timestep, explore) File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 627, in _init_action_fetches ) = self.exploration.get_exploration_action( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 84, in get_exploration_action return self._get_tf_exploration_action_op( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 91, in _get_tf_exploration_action_op stochastic_actions = tf.cond( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 94, in self.random_exploration.get_tf_exploration_action_op( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/random.py", line 138, in get_tf_exploration_action_op action = tf.cond( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/utils/exploration/random.py", line 67, in true_fn action_dist.required_model_output_shape( File "/Applications/anaconda3/envs/rl_supply_chain_updated/lib/python3.8/site-packages/ray/rllib/models/tf/tf_action_dist.py", line 188, in required_model_output_shape assert np.all(actionspace.high == high) AssertionError

During handling of the above exception, another exception occurred:

File "/Users/{user}/Projects/or-gym/examples/inv-management-quickstart.py", line 42, in agent = PPOTrainer(env=env_name,`

wojnarabc commented 1 year ago

Started to work on my end when added framework= "torch" to rl_config and modified attribute path in both pol_loss and vf_loss adding ['learner_stats'] level on fourth place. i.e. i['info']['learner']['default_policy']['learner_stats']['policy_loss']. Tested on Ray 2.1.0 version.

deter3 commented 1 year ago

@wojnarabc your solution is working now .

sumayaar commented 4 months ago

Started to work on my end when added framework= "torch" to rl_config and modified attribute path in both pol_loss and vf_loss adding ['learner_stats'] level on fourth place. i.e. i['info']['learner']['default_policy']['learner_stats']['policy_loss']. Tested on Ray 2.1.0 version.

Can you share the code instance for rl_config and how the error solved for ray 2.1.0 version?