search

hill-a / stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
http://stable-baselines.readthedocs.io/
MIT License
4.16k stars 725 forks source link

[question] After being encapsulated into a class, self_ traceback = tf_ stack.extract_ Stack() error #991

Closed borninfreedom closed 4 years ago

borninfreedom commented 4 years ago

I have write the codes into a class for convienience, but I encountered many errors. The codes are like this:

import gym
import os
import numpy as np
import tensorflow as tf
from stable_baselines import DQN,PPO2
from stable_baselines.common.vec_env import DummyVecEnv,VecVideoRecorder
from stable_baselines.common.evaluation import evaluate_policy
import matplotlib.pyplot as plt

class my_stableBaselines():

    def __init__(self,env_id='CartPole-v1',algo='DQN',policy='MlpPolicy',video_folder='logs/videos',video_length=1000):
        """

        :param env_id:
        :param algo:
        :param policy:
        :param video_folder:
        :param video_length:
        """
        available_model=['DQN','PPO','PPO2','DDPG']
        available_policy=['MlpPolicy']
        self.env_id=env_id
        self.video_folder=video_folder
        self.video_length=video_length

        if algo in available_model:
            self.algo=algo
        else:
            raise ValueError('unavailable model.')

        if policy in available_policy:
            self.policy=policy
        else:
            raise ValueError('unavailable policy.')

        model_dir='./models'
        tensorboard_dir='./tensorboard'

        print('GPU is available:',tf.test.is_gpu_available())

    def _train(self,learning_rate=1e-3,timesteps=1e4):
        """

        :param timesteps:
        :return:
        """
        self.env=gym.make(self.env_id)

        if self.algo=='PPO2':
            self.env=DummyVecEnv([lambda:self.env])
            path='./tensorboard/'+'ppo2_'+self.env_id
            self.model=PPO2(self.policy,self.env,verbose=1,tensorboard_log='./ppo2_tensorboard/')
            self.model.learn(total_timesteps=int(timesteps))
            self.model.save('./models/ppo2_'+self.env_id)

        if self.algo=='DQN':
            path = './tensorboard/' + 'dqn_' + self.env_id
            self.model=DQN(self.policy,self.env,learning_rate=learning_rate,prioritized_replay=True,verbose=1, tensorboard_log='./dqn_tensorboard/')
            self.model.learn(total_timesteps=int(timesteps))
            self.model.save('./models/dqn_'+self.env_id)

test=my_stableBaselines()
test._train()

And the errors are like this:

2020-08-29 19:13:54.308810: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll
2020-08-29 19:13:54.631765: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.632226: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.632729: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.633074: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.633495: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.633861: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.635585: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2020-08-29 19:13:54.635936: W tensorflow/stream_executor/stream.cc:2041] attempting to perform BLAS operation using StreamExecutor without BLAS support
Traceback (most recent call last):
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
    return fn(*args)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
  (0) Internal: Blas GEMM launch failed : a.shape=(1, 4), b.shape=(4, 64), m=1, n=64, k=4
     [[{{node deepq/model/action_value/fully_connected/MatMul}}]]
     [[deepq/cond/Merge/_17]]
  (1) Internal: Blas GEMM launch failed : a.shape=(1, 4), b.shape=(4, 64), m=1, n=64, k=4
     [[{{node deepq/model/action_value/fully_connected/MatMul}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-636b5e4da6fc>", line 1, in <module>
    runfile('C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py', wdir='C:/Users/born_/PycharmProjects/test_sb')
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py", line 85, in <module>
    test._train()
  File "C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py", line 60, in _train
    self.model.learn(total_timesteps=int(timesteps))
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\dqn.py", line 216, in learn
    action = self.act(np.array(obs)[None], update_eps=update_eps, **kwargs)[0]
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\build_graph.py", line 159, in act
    return _act(obs, stochastic, update_eps)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\common\tf_util.py", line 287, in <lambda>
    return lambda *args, **kwargs: func(*args, **kwargs)[0]
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\common\tf_util.py", line 330, in __call__
    results = sess.run(self.outputs_update, feed_dict=feed_dict, **kwargs)[:-1]
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 956, in run
    run_metadata_ptr)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1359, in _do_run
    run_metadata)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\client\session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InternalError: 2 root error(s) found.
  (0) Internal: Blas GEMM launch failed : a.shape=(1, 4), b.shape=(4, 64), m=1, n=64, k=4
     [[node deepq/model/action_value/fully_connected/MatMul (defined at C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
     [[deepq/cond/Merge/_17]]
  (1) Internal: Blas GEMM launch failed : a.shape=(1, 4), b.shape=(4, 64), m=1, n=64, k=4
     [[node deepq/model/action_value/fully_connected/MatMul (defined at C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]
0 successful operations.
0 derived errors ignored.
Original stack trace for 'deepq/model/action_value/fully_connected/MatMul':
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevconsole.py", line 483, in <module>
    pydevconsole.start_client(host, port)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevconsole.py", line 411, in start_client
    process_exec_queue(interpreter)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\pydevconsole.py", line 258, in process_exec_queue
    more = interpreter.add_exec(code_fragment)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_code_executor.py", line 106, in add_exec
    more = self.do_add_exec(code_fragment)
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_ipython_console.py", line 36, in do_add_exec
    res = bool(self.interpreter.add_exec(code_fragment.text))
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_ipython_console_011.py", line 483, in add_exec
    self.ipython.run_cell(line, store_history=True)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 2867, in run_cell
    raw_cell, store_history, silent, shell_futures)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 2895, in _run_cell
    return runner(coro)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\async_helpers.py", line 68, in _pseudo_sync_runner
    coro.send(None)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 3072, in run_cell_async
    interactivity=interactivity, compiler=compiler, result=result)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 3263, in run_ast_nodes
    if (await self.run_code(code, result,  async_=asy)):
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-636b5e4da6fc>", line 1, in <module>
    runfile('C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py', wdir='C:/Users/born_/PycharmProjects/test_sb')
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm 2020.2\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py", line 85, in <module>
    test._train()
  File "C:/Users/born_/PycharmProjects/test_sb/my_stableBaselines.py", line 59, in _train
    self.model=DQN(self.policy,self.env,learning_rate=learning_rate,prioritized_replay=True,verbose=1, tensorboard_log='./dqn_tensorboard/')
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\dqn.py", line 103, in __init__
    self.setup_model()
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\dqn.py", line 141, in setup_model
    double_q=self.double_q
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\build_graph.py", line 367, in build_train
    act_f, obs_phs = build_act(q_func, ob_space, ac_space, stochastic_ph, update_eps_ph, sess)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\build_graph.py", line 141, in build_act
    policy = q_func(sess, ob_space, ac_space, 1, 1, None)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\policies.py", line 224, in __init__
    layer_norm=False, **_kwargs)
  File "C:\Users\born_\anaconda3\envs\workEnv\lib\site-packages\stable_baselines\deepq\policies.py", line 112, in __init__
    action_out = tf_layers.fully_connected(action_out, num_outputs=layer_size, activation_fn=None)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\framework\python\ops\arg_scope.py", line 182, in func_with_args
    return func(*args, **current_args)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py", line 1866, in fully_connected
    outputs = layer.apply(inputs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\util\deprecation.py", line 324, in new_func
    return func(*args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 1700, in apply
    return self.__call__(inputs, *args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\layers\base.py", line 548, in __call__
    outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\engine\base_layer.py", line 854, in __call__
    outputs = call_fn(cast_inputs, *args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\autograph\impl\api.py", line 234, in wrapper
    return converted_call(f, options, args, kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\autograph\impl\api.py", line 439, in converted_call
    return _call_unconverted(f, args, kwargs, options)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\autograph\impl\api.py", line 330, in _call_unconverted
    return f(*args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\keras\layers\core.py", line 1050, in call
    outputs = gen_math_ops.mat_mul(inputs, self.kernel)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\ops\gen_math_ops.py", line 6136, in mat_mul
    name=name)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "C:\Users\born_\AppData\Roaming\Python\Python36\site-packages\tensorflow_core\python\framework\ops.py", line 1748, in __init__
    self._traceback = tf_stack.extract_stack()

It's too long. I do not know how to fix it.

Miffyli commented 4 years ago

These issues are not a place for tech support and help for bugs with custom scenarios, mainly for issues and enhancement proposals.

Quickly looking at the error it seems something is wrong with the tensorflow, perhaps the lack of libraries it tried to open is an issue. Also make sure you use TF1.x, not TF2.x. I recommend running example codes using DQN first and seeing if the problem exist there, and then move onto your custom code.

Closing issue as "no tech support".

borninfreedom commented 4 years ago

OK, thank you!

araffin commented 4 years ago

Last thing: it seems you are trying to re-invent the rl zoo (cf doc).

borninfreedom commented 4 years ago

Oh, yeah? I just want to simplify my work. I'm not familiar with RL zoo. I will have a look at it. Thank you for your reminding.