search

reinforcement-learning-kr / Unity_ML_Agents

Unity ML-agents Project Repository of RLKorea
81 stars 49 forks source link

Python Learning Error #8

Open sadlion opened 4 years ago

sadlion commented 4 years ago

책 2장 ML-Agents 3DBall 예제를 돌리는데, Unity 실행되지만 아무런 반응없이(공은 굴러가고) , python에서 학습시 연결오류가 발생합니다. [환경] Windows, Unity 2019.1.14f1, python 3.6, mlagents 0.8.1,mlagents-envs 0.8.1 [오류] env = UnityEnvironment(file_name=env_name) ==> UnityTimeOutException: The Unity environment took too long to respond. Make sure that : The environment does not need user interaction to launch The Academy's Broadcast Hub is configured correctly The Agents are linked to the appropriate Brains The environment and the Python interface have compatible versions.

image

Kyushik commented 4 years ago

UnityTimeOutException: The Unity environment took too long to respond.
위 오류의 경우 다양한 이유로 발생할 수 있기 때문에 이유를 찾기가 조금 어렵긴 합니다. 혹시 올려주신 내용이 오류 코드 전체 내용일까요? 혹시 더 내용이 있으면 첨부해주시면 좋을 것 같습니다. 일단 다음의 사항들을 확인해보시면 좋을 것 같습니다.

sadlion commented 4 years ago

Virtualenv로 환경을 꾸며서 해 보니 안되는 것 같고, Anaconda 의 conda virtualenv로 환경을 꾸며서 해보니 control 클릭시에도 agent에 의해 학습은 되는 것 같습니다. 다만, 저장할 때 convert 오류가 발생되네요. 이 부분도 체크해 보고 업데이트할 수 있으면 해 보겠습니다. (agent 버젼을 0.10.1 버젼으로 GridWorld로 모델 변경해서 테스트, 0.8.1버젼은 테스트전.)

Windows, Unity 2018.1 Anaconda python 3.7, mlagents 0.10.1,mlagents-envs 0.10.1

Reward: 0.987. Training. INFO:mlagents.trainers: first-grid: GridWorldLearning: Step: 14000. Time Elapsed: 130.803 s Mean Reward: -0.261. Std of Reward: 0.937. Training. ... Converting ./models/first-grid-0/GridWorldLearning/frozen_graph_def.pb to ./models/first-grid-0/GridWorldLearning.nn File "C:\anaconda3\envs\ml-agents-10\Scripts\mlagents-learn-script.py", line 11, in load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')() File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\learn.py", line 417, in main run_training(0, run_seed, options, Queue()) File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\learn.py", line 255, in run_training tc.start_learning(env) File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\trainer_controller.py", line 219, in start_learning self._export_graph() File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\trainer_controller.py", line 129, in _export_graph self.trainers[brain_name].export_model() File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\trainer.py", line 152, in export_model self.policy.export_model() File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\tf_policy.py", line 230, in export_model tf2bc.convert(frozen_graph_def_path, self.model_path + ".nn") File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 1552, in convert i_model, args File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 1397, in process_model process_layer(node, o_context, args) File "c:\project\unit\ml-agents-0.10.1\ml-agents\mlagents\trainers\tensorflow_to_barracuda.py", line 1220, in process_layer assert all_elements_equal(input_ranks)

sadlion commented 4 years ago

Test Environment

control 선택하지 않고 런을 실행하면 잘 동작하지만, control을 선택하고 빌드후 train시키거나, mlagents-learn 실행후 런하게 되면 다음과 같은 에러가 발생합니다. brain code는 #C 이고 train code는 python인데, 상관없나요?

-- Unity console UnityAgentsException: The Communicator was unable to connect. Please make sure the External process is ready to accept communication with Unity. MLAgents.Batcher.SendAcademyParameters (MLAgents.CommunicatorObjects.UnityRLInitializationOutput academyParameters) (at Assets/ML-Agents/Scripts/Batcher.cs:91) MLAgents.Academy.InitializeEnvironment () (at Assets/ML-Agents/Scripts/Academy.cs:345) MLAgents.Academy.Awake () (at Assets/ML-Agents/Scripts/Academy.cs:250)

-- Jupyter 상에서 python code c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents\mlagents\trainers\learn.py:141: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. trainer_config = yaml.load(data_file) INFO:mlagents.envs:Start training by pressing the Play button in the Unity Editor. Process Process-1: Traceback (most recent call last): File "C:\Anaconda3\envs\ml-agents-8\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "C:\Anaconda3\envs\ml-agents-8\lib\multiprocessing\process.py", line 93, in run self._target(*self._args, **self._kwargs) File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\subprocess_environment.py", line 53, in worker env = env_factory(worker_id) File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents\mlagents\trainers\learn.py", line 192, in create_unity_environment base_port=start_port File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\environment.py", line 76, in init aca_params = self.send_academy_parameters(rl_init_parameters_in) File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\environment.py", line 538, in send_academy_parameters return self.communicator.initialize(inputs).rl_initialization_output File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\rpc_communicator.py", line 80, in initialize "The Unity environment took too long to respond. Make sure that :\n" mlagents.envs.exception.UnityTimeOutException: The Unity environment took too long to respond. Make sure that : The environment does not need user interaction to launch The Academy's Broadcast Hub is configured correctly The Agents are linked to the appropriate Brains The environment and the Python interface have compatible versions. Traceback (most recent call last): File "C:\Anaconda3\envs\ml-agents-8\lib\multiprocessing\connection.py", line 312, in _recv_bytes nread, err = ov.GetOverlappedResult(True) BrokenPipeError: [WinError 109] The pipe has been ended

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\subprocess_environment.py", line 38, in recv response: EnvironmentResponse = self.conn.recv() File "C:\Anaconda3\envs\ml-agents-8\lib\multiprocessing\connection.py", line 250, in recv buf = self._recv_bytes() File "C:\Anaconda3\envs\ml-agents-8\lib\multiprocessing\connection.py", line 321, in _recv_bytes raise EOFError EOFError

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Anaconda3\envs\ml-agents-8\Scripts\mlagents-learn-script.py", line 11, in load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')() File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents\mlagents\trainers\learn.py", line 262, in main run_training(0, run_seed, options, Queue()) File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents\mlagents\trainers\learn.py", line 88, in run_training keep_checkpoints, lesson, env.external_brains, File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\subprocess_environment.py", line 173, in external_brains return self.envs[0].recv().payload File "c:\d_drive\30.project\dev_folder\dev_python36\rl\unity_ml_agents-master\ml-agents-0.8.1\ml-agents-envs\mlagents\envs\subprocess_environment.py", line 41, in recv raise KeyboardInterrupt KeyboardInterrupt

Kyushik commented 4 years ago

원래 control에 체크하고 환경을 빌드한 후에 학습을 시키는 것이 맞는 과정입니다! 유니티 코드는 C#이고 학습 코드는 python이지만 이 코드들이 서로 통신을 할 수 있게 도와주는 것이 ML-Agents의 역할이라 잘 작동을 해야합니다. 일단 Unity console 상에서 에러가 발생하면 환경은 열리지만 진행되지 않고 멈춰있는 상태가 됩니다. 유니티에서 발생하는 에러에 대한 확인은 developer build로 확인해보신건가요? 통신을 하기 위한 설정이 뭔가 잘못된 것 같은데 에이전트들에도 learning brain으로 잘 할당이 되어있는지 확인해보면 좋을 것 같습니다. 아니면 현재 테스트해보고 계신 유니티 환경 코드를 제 메일로 보내주시면 한번 직접 확인해보겠습니다. kyushikmin@gmail.com 입니다!