ray-project / ray

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://ray.io
Apache License 2.0
33.35k stars 5.65k forks source link

[rllib] Initializing a trainer fails - TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` #7629

Closed CesMak closed 3 years ago

CesMak commented 4 years ago

Tensorflow fails initializing a trainer

I use rllib and have tensorflow (1.1.4) and pytorch installed in a python 3.6.9 environment. When initializing a trainer trainer = ppo.PPOTrainer(env=WitchesEnv, config= config) I get an Error.

Error:

TypeError: Using atf.Tensoras a Pythonboolis not allowed. Useif t is not None:instead ofif t:to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

Full Code

import gym, ray
from ray.rllib.agents import ppo
from gameClasses import card, deck, player, game
import json

class WitchesEnv(gym.Env):
    def __init__(self, env_config):
        print("Inside InIT WITCHES ENV")
        self.action_space = gym.spaces.Discrete(60)
        self.observation_space = gym.spaces.Discrete(180)

        # Create the game:
        self.options = {}
        self.options_file_path =  "../data/reinforce_options.json"
        with open(self.options_file_path) as json_file:
            self.options = json.load(json_file)
        self.my_game     = game(self.options)

        print("End of WitchesEnv")
        # Start the first game
        # self.reset()

    def reset(self):
        print("INSIDE RESET \n\n")
        self.my_game.reset_game()
        active_player, state, options = self.my_game.getState()
        return state

    def step(self, action):
        print("INSIDE STEP! \n\n")
        print(action)
        assert self.action_space.contains(action)
        done = 0
        rewards, round_finished = self._takeAction(action)
        if len(self.my_game.players[current_player].hand) == 0: # game finished
            done = 1
        #play until ai!
        return None#<obs>, <reward: float>, <done: bool>, <info: dict>

ray.init()
config = {
    "lr": 0.01,
    "num_workers": 0,
}
trainer = ppo.PPOTrainer(env=WitchesEnv, config= config)
#
# while True:
#     print(trainer.train())

Full Error


home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Couldn't import `requests` library. Be sure to install it on the client side.
Couldn't import `requests` library. Be sure to install it on the client side.
Traceback (most recent call last):
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/tune/web_server.py", line 16, in <module>
    import requests  # `requests` is not part of stdlib.
ModuleNotFoundError: No module named 'requests'
2020-03-16 20:10:35,850 INFO resource_spec.py:212 -- Starting Ray with 8.01 GiB memory available for workers and up to 4.02 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
2020-03-16 20:10:36,281 INFO services.py:1078 -- View the Ray dashboard at localhost:8265
2020-03-16 20:10:36,709 INFO trainer.py:420 -- Tip: set 'eager': true or the --eager flag to enable TensorFlow eager execution
2020-03-16 20:10:36,790 INFO trainer.py:580 -- Current log_level is WARN. For more information, set 'log_level': 'INFO' / 'DEBUG' or use the -v and -vv flags.
Inside InIT WITCHES ENV
Laura's hand: [5 of B, 6 of B, 5 of G, 6 of G, 7 of G, 8 of G, 10 of G, >11< of G, 13 of G, 3 of R, 9 of R, >11< of R, 13 of R, 2 of Y, 8 of Y]
Alfons's hand: [>11< of B, 13 of B, J of B, 2 of G, 4 of G, 9 of G, J of G, 1 of R, 6 of R, 14 of R, 3 of Y, 6 of Y, 10 of Y, 13 of Y, 14 of Y]
Frank's hand: [1 of B, 2 of B, 8 of B, 9 of B, 1 of G, 14 of G, 2 of R, 8 of R, 12 of R, J of R, 4 of Y, 7 of Y, >11< of Y, 12 of Y, J of Y]
Lea's hand: [3 of B, 4 of B, 7 of B, 10 of B, 12 of B, 14 of B, 3 of G, 12 of G, 4 of R, 5 of R, 7 of R, 10 of R, 1 of Y, 5 of Y, 9 of Y]
End of WitchesEnv
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
  warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/utils/from_config.py:134: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  obj = yaml.load(type_)
Traceback (most recent call last):
  File "rllib_example.py", line 82, in <module>
    trainer = a3c.A2CTrainer(env=WitchesEnv, config= config)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 86, in __init__
    Trainer.__init__(self, config, env, logger_creator)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 447, in __init__
    super().__init__(config, logger_creator)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/tune/trainable.py", line 172, in __init__
    self._setup(copy.deepcopy(self.config))
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 591, in _setup
    self._init(self.config, self.env_creator)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/agents/trainer_template.py", line 105, in _init
    self.config["num_workers"])
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/agents/trainer.py", line 658, in _make_workers
    logdir=self.logdir)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 60, in __init__
    RolloutWorker, env_creator, policy, 0, self._local_config)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/evaluation/worker_set.py", line 262, in _make_worker
    _fake_sampler=config.get("_fake_sampler", False))
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 355, in __init__
    self._build_policy_map(policy_dict, policy_config)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/evaluation/rollout_worker.py", line 820, in _build_policy_map
    policy_map[name] = cls(obs_space, act_space, merged_conf)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/policy/tf_policy_template.py", line 138, in __init__
    obs_include_prev_action_reward=obs_include_prev_action_reward)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/policy/dynamic_tf_policy.py", line 180, in __init__
    timestep=timestep)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/ray/rllib/utils/exploration/stochastic_sampling.py", line 74, in get_exploration_action
    return self._get_tf_exploration_action_op(action_dist, explore)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 414, in __call__
    self._initialize(args, kwds, add_initializers_to=initializer_map)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 357, in _initialize
    *args, **kwds))
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1349, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1652, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1545, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 715, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 307, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 705, in wrapper
    raise e.ag_error_metadata.to_exception(type(e))
TypeError: in converted code:
    relative to /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages:

    ray/rllib/utils/exploration/stochastic_sampling.py:79 _get_tf_exploration_action_op
        if explore:
    tensorflow/python/framework/ops.py:690 __bool__
        raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "

    TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
ericl commented 4 years ago

Seems like a bug. Can you provide a runnable repro script (the one above is not runnable due to from gameClasses import card, deck, player, game).

Also, would be great to verify it also happens on the latest snapshot: https://ray.readthedocs.io/en/latest/installation.html

CesMak commented 4 years ago

Hey there, yes same error happens even with

import gym, ray
from ray.rllib.agents import ppo

class WitchesEnv(gym.Env):
    def __init__(self, env_config):
        print("Inside InIT WITCHES ENV")
        self.action_space = gym.spaces.Discrete(60)
        self.observation_space = gym.spaces.Discrete(180)

    def reset(self):
        return None

    def step(self, action):
        return None#<obs>, <reward: float>, <done: bool>, <info: dict>

ray.init()
config = {
    "lr": 0.01,
    "num_workers": 0,
}
trainer = ppo.PPOTrainer(env=WitchesEnv, config= config)
#
# while True:
#     print(trainer.train())

I installed it as follows:

pip install -U ray

Requirement already up-to-date: ray in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages
Requirement already up-to-date: grpcio in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: cloudpickle in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: click in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: pyyaml in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: redis>=3.3.2 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: funcsigs in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: google in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Collecting numpy>=1.16 (from ray)
  Downloading https://files.pythonhosted.org/packages/07/08/a549ba8b061005bb629b76adc000f3caaaf881028b963c2e18f811c6edc1/numpy-1.18.2-cp36-cp36m-manylinux1_x86_64.whl (20.2MB)
    100% |████████████████████████████████| 20.2MB 75kB/s 
Requirement already up-to-date: py-spy>=0.2.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: pytest in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: protobuf>=3.8.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: jsonschema in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: colorama in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: packaging in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: filelock in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: six>=1.0.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: aiohttp in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from ray)
Requirement already up-to-date: beautifulsoup4 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from google->ray)
Requirement already up-to-date: pluggy<1.0,>=0.12 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: wcwidth in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: attrs>=17.4.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: py>=1.5.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: importlib-metadata>=0.12; python_version < "3.8" in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: more-itertools>=4.0.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from pytest->ray)
Requirement already up-to-date: setuptools in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from protobuf>=3.8.0->ray)
Requirement already up-to-date: pyrsistent>=0.14.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from jsonschema->ray)
Requirement already up-to-date: pyparsing>=2.0.2 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from packaging->ray)
Requirement already up-to-date: chardet<4.0,>=2.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: typing-extensions>=3.6.5; python_version < "3.7" in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: async-timeout<4.0,>=3.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: multidict<5.0,>=4.5 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: idna-ssl>=1.0; python_version < "3.7" in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: yarl<2.0,>=1.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from aiohttp->ray)
Requirement already up-to-date: soupsieve>=1.2 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from beautifulsoup4->google->ray)
Requirement already up-to-date: zipp>=0.5 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from importlib-metadata>=0.12; python_version < "3.8"->pytest->ray)
Requirement already up-to-date: idna>=2.0 in /home/markus/Documents/06_Software_Projects/mcts/mcts_env/lib/python3.6/site-packages (from idna-ssl>=1.0; python_version < "3.7"->aiohttp->ray)
Installing collected packages: numpy
  Found existing installation: numpy 1.18.1
    Uninstalling numpy-1.18.1:
      Successfully uninstalled numpy-1.18.1
Successfully installed numpy-1.18.2
ericl commented 4 years ago

It works for me on both TF2.0 and 1.15. Maybe upgrade to 1.15?

pengmun commented 4 years ago

I had similar issue on ray 0.8.2 with tf 1.14. It worked fine after downgrading ray to 0.8.1.

CesMak commented 4 years ago

@pengmun How did you do that ? using pip?

I tried it with: pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/{COMMIT_HASH}/ray-{RAY_VERSION}-{PYTHON_VERSION}-{PYTHON_VERSION}m-{OS_VERSION}_intel.whl

The commit from here:

pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/38ec2e7/ray-0.8.1-cp36-cp36m-manylinux1_x86_64.whl

  pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/38ec2e70524a277d5aea307f6c843065ff982da5/ray-0.8.1-cp36-cp36m-manylinux1_x86_64.whl

but both failed: Not Found for url

pengmun commented 4 years ago

@CesMak Yupe, I used pip to install the older version. You can do so by using the following commands.

pip install ray==0.8.1 pip install ray[debug]==0.8.1 pip install ray[tune]==0.8.1

CesMak commented 4 years ago

Thanks that worked.

You can close this issue

stale[bot] commented 3 years ago

Hi, I'm a bot from the Ray team :)

To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.

If there is no further activity in the 14 days, the issue will be closed!

You can always ask for help on our discussion forum or Ray's public slack channel.

stale[bot] commented 3 years ago

Hi again! The issue will be closed because there has been no more activity in the 14 days since the last message.

Please feel free to reopen or open a new issue if you'd still like it to be addressed.

Again, you can always ask for help on our discussion forum or Ray's public slack channel.

Thanks again for opening the issue!