Cannot Execute Main.py - Githubissues

nilesh-hegde commented 4 years ago

Note - nvidia_smi wasn't recognised so i had to import pynvml.smi to use nvidia_smi but it now shows 'nvmlInit attribute is missing

C:\PEDRA>python main.py pygame 1.9.6 Hello from the pygame community. https://www.pygame.org/contribute.html ------------------------------- configs/config.cfg ------------------------------- [general_params] run_name: Tello_indoor env_type: Indoor env_name: indoor_long mode: train SimMode: Multirotor drone: DJIMavic ClockSpeed: 20 algorithm: DeepQLearning ip_address: 127.0.0.5 num_agents: 3 [camera_params] width: 320 height: 180 fov_degrees: 80 C:\PEDRA\util\transformations.py:1914: UserWarning: failed to import module _transformations warnings.warn('failed to import module %s' % name) --------------------------- configs/DeepQLearning.cfg --------------------------- [simulation_params] load_data: False load_data_path: DeepNet/models/Tello_indoor/VanLeer/ distributed_algo: LocalLearningLocalUpdate [RL_params] input_size: 103 num_actions: 25 train_type: e2e wait_before_train: 5000 max_iters: 150000 buffer_len: 10000 batch_size: 32 epsilon_saturation: 100000 crash_thresh: 1.3 Q_clip: True train_interval: 2 update_target_interval: 8000 gamma: 0.99 dropout_rate: 0.1 learning_rate: 2e-06 switch_env_steps: 2000000000 epsilon_model: exponential custom_load: False custom_load_path: models/trained/Indoor/indoor_long/Imagenet/e2e/drone0/drone0 communication_interval: 100 average_connectivity: 2

---------------------------------- Environment ---------------------------------- The system cannot find the path specified. Successfully loaded environment: indoor_long

------------------------------------- Drone ------------------------------------- Connected! Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

Traceback (most recent call last): File "main.py", line 87, in eval(name) File "", line 1, in File "C:\PEDRA\algorithms\DeepQLearning.py", line 37, in DeepQLearning nvidia_smi.nvmlInit() AttributeError: type object 'nvidia_smi' has no attribute 'nvmlInit'

aqeelanwar commented 4 years ago

Did you install the nvidia-ml-py3 package?

pip install nvidia-ml-py3

nilesh-hegde commented 4 years ago

I did install nvidia-ml-py3 package but I'm getting the following error. Does this have something to do with the fact that the code is running on a system with AMD?

C:\PEDRA>python main.py pygame 1.9.6 Hello from the pygame community. https://www.pygame.org/contribute.html ------------------------------- configs/config.cfg ------------------------------- [general_params] run_name: Tello_indoor env_type: Indoor env_name: indoor_long mode: train SimMode: Multirotor drone: DJIMavic ClockSpeed: 20 algorithm: DeepQLearning ip_address: 127.0.0.5 num_agents: 3 [camera_params] width: 320 height: 180 fov_degrees: 80 C:\PEDRA\util\transformations.py:1914: UserWarning: failed to import module _transformations warnings.warn('failed to import module %s' % name) --------------------------- configs/DeepQLearning.cfg --------------------------- [simulation_params] load_data: False load_data_path: DeepNet/models/Tello_indoor/VanLeer/ distributed_algo: LocalLearningLocalUpdate [RL_params] input_size: 103 num_actions: 25 train_type: e2e wait_before_train: 5000 max_iters: 150000 buffer_len: 10000 batch_size: 32 epsilon_saturation: 100000 crash_thresh: 1.3 Q_clip: True train_interval: 2 update_target_interval: 8000 gamma: 0.99 dropout_rate: 0.1 learning_rate: 2e-06 switch_env_steps: 2000000000 epsilon_model: exponential custom_load: False custom_load_path: models/trained/Indoor/indoor_long/Imagenet/e2e/drone0/drone0 communication_interval: 100 average_connectivity: 2

---------------------------------- Environment ---------------------------------- The system cannot find the path specified. Successfully loaded environment: indoor_long

------------------------------------- Drone ------------------------------------- Connected! Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

Traceback (most recent call last): File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\pynvml\nvml.py", line 731, in _load_nvml_library nvml_lib = CDLL(os.path.join(os.getenv("ProgramFiles", "C:/Program Files"), "NVIDIA Corporation/NVSMI/nvml.dll")) File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\ctypes__init.py", line 364, in init__ self._handle = _dlopen(self._name, mode) OSError: [WinError 126] The specified module could not be found

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "main.py", line 87, in eval(name) File "", line 1, in File "C:\PEDRA\algorithms\DeepQLearning.py", line 37, in DeepQLearning nvmlInit() File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\pynvml\nvml.py", line 742, in nvmlInit _load_nvml_library() File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\pynvml\nvml.py", line 736, in _load_nvml_library check_return(NVML_ERROR_LIBRARY_NOT_FOUND) File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\pynvml\nvml.py", line 366, in check_return raise NVMLError(ret) pynvml.nvml.NVMLError_LibraryNotFound: NVML Shared Library Not Found

aqeelanwar commented 4 years ago

As a matter of fact, Yes. nvidia-smi only supports NVIDIA GPUs. It won't work for AMD.

I will edit the code to check the GPU and disable nvidia-smi if NVIDIA GPU is not found. In the meanwhile you can comment out the imports and the following lines

nvidia_smi.nvmlInit()

and

gpu_memory, gpu_utilization, sys_memory = get_SystemStats(process) for i in range(0, len(gpu_memory)): tag_mem = 'GPU'+str(i)+'-Memory-GB' tag_util = 'GPU' + str(i) + 'Utilization-%' agent[name_agent].log_to_tensorboard(tag=tag_mem, group='SystemStats', value=gpu_memory[i], index=iter) agent[name_agent].log_to_tensorboard(tag=tag_util,group='SystemStats', value=gpu_utilization[i], index=iter) agent[name_agent].log_to_tensorboard(tag='Memory-GB',group='SystemStats', value=sys_memory, index=iter)

nilesh-hegde commented 4 years ago

Ok But will this affect the general working of the code?

aqeelanwar commented 4 years ago

It won't. The only difference will be that the tensorboard won't be able to display GPU stats.

aqeelanwar commented 4 years ago

I have fixed the issue. If you pull the repository, you will see the issue has been resolved. Let me know if this works for you.

nilesh-hegde commented 4 years ago

Yes the issue has been fixed. Is this the expected behaviour of the code? are the drones being trained and three drones represent ARDrone / DJIMavic/ DJIPhantom drones?

C:\PEDRA>python main.py pygame 1.9.6 Hello from the pygame community. https://www.pygame.org/contribute.html ------------------------------- configs/config.cfg ------------------------------- [general_params] run_name: Tello_indoor env_type: Indoor env_name: indoor_long mode: train SimMode: Multirotor drone: DJIMavic ClockSpeed: 20 algorithm: DeepQLearning ip_address: 127.0.0.5 num_agents: 3 [camera_params] width: 320 height: 180 fov_degrees: 80 C:\PEDRA\util\transformations.py:1914: UserWarning: failed to import module _transformations warnings.warn('failed to import module %s' % name) --------------------------- configs/DeepQLearning.cfg --------------------------- [simulation_params] custom_load: False custom_load_path: models/trained/Indoor/indoor_long/Imagenet/e2e/drone0/drone0 distributed_algo: LocalLearningLocalUpdate [RL_params] input_size: 103 num_actions: 25 train_type: e2e wait_before_train: 5000 max_iters: 150000 buffer_len: 10000 batch_size: 32 epsilon_saturation: 100000 crash_thresh: 1.3 Q_clip: True train_interval: 2 update_target_interval: 8000 gamma: 0.99 dropout_rate: 0.1 learning_rate: 2e-06 switch_env_steps: 2000000000 epsilon_model: exponential [distributed_RL params] communication_interval: 100 average_connectivity: 2

---------------------------------- Environment ---------------------------------- The system cannot find the path specified. Successfully loaded environment: indoor_long

------------------------------------- Drone ------------------------------------- Connected! Client Ver:1 (Min Req: 1), Server Ver:1 (Min Req: 1)

----------------- drone0 ----------------- Initializing DQN WARNING:tensorflow:From C:\PEDRA\network\agent.py:231: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:231: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:239: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\image_ops_impl.py:1518: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\image_ops_impl.py:1518: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. WARNING:tensorflow:From C:\PEDRA\network\network.py:127: The name tf.truncated_normal is deprecated. Please use tf.random.truncated_normal instead.

WARNING:tensorflow:From C:\PEDRA\network\network.py:127: The name tf.truncated_normal is deprecated. Please use tf.random.truncated_normal instead.

WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variables.py:2825: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variables.py:2825: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. WARNING:tensorflow:From C:\PEDRA\network\network.py:110: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow:From C:\PEDRA\network\network.py:110: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
https://github.com/tensorflow/addons
https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.flatten instead. WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\contrib\layers\python\layers\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version. Instructions for updating: Use keras.layers.flatten instead. WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\layers\core.py:332: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\layers\core.py:332: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From C:\PEDRA\network\network.py:140: The name tf.nn.relu_layer is deprecated. Please use tf.compat.v1.nn.relu_layer instead.

WARNING:tensorflow:From C:\PEDRA\network\network.py:140: The name tf.nn.relu_layer is deprecated. Please use tf.compat.v1.nn.relu_layer instead.

WARNING:tensorflow:From C:\PEDRA\network\network.py:142: The name tf.nn.xw_plus_b is deprecated. Please use tf.compat.v1.nn.xw_plus_b instead.

WARNING:tensorflow:From C:\PEDRA\network\loss_functions.py:9: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From C:\PEDRA\network\loss_functions.py:9: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where WARNING:tensorflow:From C:\PEDRA\network\agent.py:256: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:256: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:259: The name tf.InteractiveSession is deprecated. Please use tf.compat.v1.InteractiveSession instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:260: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:261: The name tf.local_variables_initializer is deprecated. Please use tf.compat.v1.local_variables_initializer instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:262: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:263: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.

Initializing Target C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py:1750: UserWarning: An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s). warnings.warn('An interactive session is already active. This can '

----------------- drone1 ----------------- Initializing DQN C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py:1750: UserWarning: An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s). warnings.warn('An interactive session is already active. This can ' Initializing Target C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py:1750: UserWarning: An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s). warnings.warn('An interactive session is already active. This can '

----------------- drone2 ----------------- Initializing DQN C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py:1750: UserWarning: An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s). warnings.warn('An interactive session is already active. This can ' Initializing Target C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py:1750: UserWarning: An interactive session is already active. This can cause out-of-memory errors in some cases. You must explicitly call InteractiveSession.close() to release resources held by the other session(s). warnings.warn('An interactive session is already active. This can ' ----------------------------------- Log files ----------------------------------- Log path: models/trained/Indoor/indoor_long/Imagenet/e2e//drone0/trainlog.txt Log path: models/trained/Indoor/indoor_long/Imagenet/e2e//drone1/trainlog.txt Log path: models/trained/Indoor/indoor_long/Imagenet/e2e//drone2/trainlog.txt

------------------------------- Simulation begins ------------------------------- WARNING:tensorflow:From C:\PEDRA\network\agent.py:337: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.

WARNING:tensorflow:From C:\PEDRA\network\agent.py:337: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.

drone0 - Level 0 - Iter: 1/0 U1-R1 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +0.3400 Last Crash = 0 t=0.308 SF = 0.5371 Reward: +0.3400 drone1 - Level 0 - Iter: 1/0 F0-L1 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +1.0000 Last Crash = 0 t=0.209 SF = 0.5263 Reward: +1.0000 drone2 - Level 0 - Iter: 1/0 U1-R1 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +1.0000 Last Crash = 0 t=0.227 SF = 0.5570 Reward: +1.0000 drone0 - Level 0 - Iter: 2/0 U1-L1 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +1.0000 Last Crash = 1 t=0.218 SF = 1.1053 Reward: +0.6600 drone1 - Level 0 - Iter: 2/0 F0-F0 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +2.0000 Last Crash = 1 t=0.221 SF = 1.1233 Reward: +1.0000 drone2 - Level 0 - Iter: 2/0 D1-F0 - Rand Eps: -0.1110 lr: 0.00000200 Ret = +2.0000 Last Crash = 1 t=0.222 SF = 1.1320 Reward: +1.0000 drone0 - Level 0 - Iter: 3/0 D2-F0 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +1.6600 Last Crash = 2 t=0.224 SF = 1.6648 Reward: +0.6600 drone1 - Level 0 - Iter: 3/0 U2-L2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +3.0000 Last Crash = 2 t=0.228 SF = 1.6894 Reward: +1.0000 drone2 - Level 0 - Iter: 3/0 U2-R2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +3.0000 Last Crash = 2 t=0.204 SF = 1.6665 Reward: +1.0000 drone0 - Level 0 - Iter: 4/0 U1-F0 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +2.3200 Last Crash = 3 t=0.228 SF = 2.2268 Reward: +0.6600 drone1 - Level 0 - Iter: 4/0 U1-L2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +4.0000 Last Crash = 3 t=0.229 SF = 2.2276 Reward: +1.0000 drone2 - Level 0 - Iter: 4/0 U1-R2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +4.0000 Last Crash = 3 t=0.228 SF = 2.2440 Reward: +1.0000 drone0 - Level 0 - Iter: 5/0 F0-L2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +2.5000 Last Crash = 4 t=0.227 SF = 2.7809 Reward: +0.1800 drone1 - Level 0 - Iter: 5/0 U1-L2 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +5.0000 Last Crash = 4 t=0.230 SF = 2.7664 Reward: +1.0000 drone2 - Level 0 - Iter: 5/0 U1-R1 - Rand Eps: -0.1109 lr: 0.00000200 Ret = +5.0000 Last Crash = 4 t=0.231 SF = 2.8014 Reward: +1.0000

nilesh-hegde commented 4 years ago

Got it Read the readme files

nilesh-hegde commented 4 years ago

In mode: Infer i'm getting the following error:

Traceback (most recent call last): File "main.py", line 94, in eval(name) File "", line 1, in File "C:\PEDRA\algorithms\DeepQLearning.py", line 86, in DeepQLearning p_z,f_z, fig_z, ax_z, line_z, fig_nav, ax_nav, nav = initialize_infer(env_cfg=env_cfg, client=client, env_folder=env_folder) File "C:\PEDRA\aux_functions.py", line 86, in initialize_infer f_z = env_cfg.floor_z/100 TypeError: unsupported operand type(s) for /: 'DotMap' and 'Int'

aqeelanwar commented 4 years ago

Can you share the environment config file? It should be at unreal_envs/indoor_long/config.cfg

nilesh-hegde commented 4 years ago

Fixed it In my config file i had it as indoor_cloud instead of indoor_long

nilesh-hegde commented 4 years ago

During Training why does a "Communicating the weights and averaging them" line pop up after certain number of iterations?

aqeelanwar commented 4 years ago

For distributed Reinforcement Learning, multiple drones needs to communicate with each other. The drones communicate every 'communication_interval' iteration. This parameter is defined in the DeepQLearning.cfg file. Multiple drones share their NN weights and average them. In case you are using only one drone, this line doesn't affect the code at all.

nilesh-hegde commented 4 years ago

Thank you But sometimes in simulation the drone is stationary at the starting point with the iterations taking place as the last crash parameter in the command line increasing reaching 1000's. Is this an expected behaviour ? I have attached a snippet of the prompt

drone0 - Level 0 - Iter: 92002/4343 D1-F0 - Rand Eps: 0.8398 lr: 0.00000200 Ret = +395.9152 Last Crash = 825 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92003/4343 D2-F0 - Pred Eps: 0.8398 lr: 0.00000200 Ret = +396.3945 Last Crash = 826 t=0.263 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92004/4343 U1-F0 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +396.8738 Last Crash = 827 t=0.430 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92005/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +397.3532 Last Crash = 828 t=0.259 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92006/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +397.8325 Last Crash = 829 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92007/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +398.3118 Last Crash = 830 t=0.261 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92008/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +398.7911 Last Crash = 831 t=0.424 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92009/4343 D2-F0 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +399.2704 Last Crash = 832 t=0.239 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92010/4343 D1-L2 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +399.7497 Last Crash = 833 t=0.427 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92011/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +400.2291 Last Crash = 834 t=0.262 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92012/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +400.7084 Last Crash = 835 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92013/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +401.1877 Last Crash = 836 t=0.264 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92014/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +401.6670 Last Crash = 837 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92015/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +402.1463 Last Crash = 838 t=0.261 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92016/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +402.6256 Last Crash = 839 t=0.428 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92017/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +403.1050 Last Crash = 840 t=0.260 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92018/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +403.5843 Last Crash = 841 t=0.427 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92019/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +404.0636 Last Crash = 842 t=0.260 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92020/4343 F0-R1 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +404.5429 Last Crash = 843 t=0.430 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92021/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +405.0222 Last Crash = 844 t=0.260 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92022/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +405.5015 Last Crash = 845 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92023/4343 D1-L2 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +405.9809 Last Crash = 846 t=0.238 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92024/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +406.4602 Last Crash = 847 t=0.424 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92025/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +406.9395 Last Crash = 848 t=0.262 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92026/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +407.4188 Last Crash = 849 t=0.425 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92027/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +407.8981 Last Crash = 850 t=0.261 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92028/4343 D1-R2 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +408.3774 Last Crash = 851 t=0.426 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92029/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +408.8568 Last Crash = 852 t=0.262 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92030/4343 D1-L2 - Rand Eps: 0.8399 lr: 0.00000200 Ret = +409.3361 Last Crash = 853 t=0.430 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92031/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +409.8154 Last Crash = 854 t=0.257 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92032/4343 D2-F0 - Pred Eps: 0.8399 lr: 0.00000200 Ret = +410.2947 Last Crash = 855 t=0.428 SF = 0.0000 Reward: +0.4793 drone0 - Level 0 - Iter: 92033/4343 D2-F0 - Pred Eps: 0.8400 lr: 0.00000200 Ret = +410.7740 Last Crash = 856 t=0.264 SF = 0.0000 Reward: +0.4793

nilesh-hegde commented 4 years ago

In the training log i deleted the above iterations and ran infer by making custom load true and adding path to drone0. the cfg file is as follows

[simulation_params] custom_load: True custom_load_path: C:\PEDRA\models\trained\Indoor\indoor_long\Imagenet\e2e\drone0 distributed_algo: LocalLearningLocalUpdate ;distributed_algo: LocalLearningGlobalUpdate ;distributed_algo: GlobalLearningGlobalUpdate-SA ;distributed_algo: GlobalLearningGlobalUpdate-MA

[RL_params] input_size: 103 num_actions: 25 train_type: e2e wait_before_train: 5000 max_iters: 150000 buffer_len: 10000 batch_size: 32 epsilon_saturation: 100000 crash_thresh: 1.3 Q_clip: True train_interval: 2 update_target_interval: 8000 gamma: 0.99 dropout_rate: 0.1 learning_rate: 2e-6 switch_env_steps: 2000000000 epsilon_model: exponential

[distributed_RL params] communication_interval: 100 average_connectivity: 2

But I'm getting the following error

Loading weights from: C:\PEDRA\models\trained\Indoor\indoor_long\Imagenet\e2e\drone0 Traceback (most recent call last): File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call return fn(*args) File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn target_list, run_metadata) File "C:\Users\User\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file C:\PEDRA\models\trained\Indoor\indoor_long\Imagenet\e2e\drone0: Unknown: NewRandomAccessFile failed to Create/Open: C:\PEDRA\models\trained\Indoor\indoor_long\Imagenet\e2e\drone0 : Access is denied.

aqeelanwar commented 4 years ago

As for the issue where the last crash keeps on increasing without any movement in the drone, the environment sometimes acts up in detecting collisions. I have fixed the issue.

nilesh-hegde commented 4 years ago

Ok Thank you

aqeelanwar / PEDRA

Cannot Execute Main.py #19