How to use example/recorders/?

liber145 commented 7 years ago

example/recorders/botaction_recorder.py states that

This is a small server that accepts connections on a websocket port and writes it to a file.

The purpose is to allow a universe-env with a built-in bot to record the actions it's taking as a demonstration. So the demonstration includes a botactions.jsonl file that gets used instead of the vnc client log. (The vnc client log is still recorded and needed to fully parse the VNC protocol.)

I want to use it to record my actions when playing games. Following the example code in ReadMe, I can start the agent, connect to the server and play games successfully. And the output in the terminal states that

universe-B62sOv-0 | [init] [2017-02-06 14:52:33,862] Launching system_diagnostics_logger.py, recorder_logdir=/tmp/demo universe-B62sOv-0 | [init] [2017-02-06 14:52:33,865] Launching reward_recorder.py, recorder_logdir=/tmp/demo universe-B62sOv-0 | [init] [2017-02-06 14:52:33,869] Launching vnc_recorder.py, recorder_logdir=/tmp/demo

There is supposed to be three log files in /tmp/demo. However, after docker exec into that running docker, I only get system_diagnostics.jsonl there.

I have a poor knowledge about the network. It is hard for me to get clear about communication actions between recorder and agent. Could anyone show me how to use /example/recorders/*_recorder.py?

Expected behavior

Get /tmp/demo/botactions.jsonl and /tmp/demo/vnc_record.jsonl in docker file system.

Actual behavior

There is only /tmp/demo/botactions.jsonl in docker file system.

Versions list

Darwin nat-tvwna-inside-visitornet101-d-3212.princeton.org 15.6.0 Darwin Kernel Version 15.6.0: Mon Jan 9 23:07:29 PST 2017; root:xnu-3248.60.11.2.1~1/RELEASE_X86_64 x86_64 Python 3.5.2 :: Anaconda 4.2.0 (x86_64) Name: universe Version: 0.21.2 Summary: Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications. Home-page: https://github.com/openai/universe Author: OpenAI Author-email: universe@openai.com License: UNKNOWN Location: /Users/liyujun/Programs/universe Requires: autobahn, docker-py, docker-pycreds, fastzbarlight, go-vncdriver, gym, Pillow, PyYAML, six, twisted, ujson

Name: gym Version: 0.7.1 Summary: The OpenAI Gym: A toolkit for developing and comparing your reinforcement learning agents. Home-page: https://github.com/openai/gym Author: OpenAI Author-email: gym@openai.com License: UNKNOWN Location: /Users/liyujun/anaconda3/lib/python3.5/site-packages Requires: pyglet, numpy, requests, six

Name: numpy Version: 1.11.1 Summary: NumPy: array processing for numbers, strings, records, and objects. Home-page: http://www.numpy.org Author: NumPy Developers Author-email: numpy-discussion@scipy.org License: BSD Location: /Users/liyujun/anaconda3/lib/python3.5/site-packages Requires:

Name: go-vncdriver Version: 0.4.19 Summary: UNKNOWN Home-page: UNKNOWN Author: UNKNOWN Author-email: UNKNOWN License: UNKNOWN Location: /Users/liyujun/anaconda3/lib/python3.5/site-packages Requires: numpy

Name: Pillow Version: 3.3.1 Summary: Python Imaging Library (Fork) Home-page: http://python-pillow.org Author: Alex Clark (Fork Author) Author-email: aclark@aclark.net License: Standard PIL License Location: /Users/liyujun/anaconda3/lib/python3.5/site-packages Requires:

tlbtlbtlb commented 7 years ago

examples/botaction_recorder.py is for recording the actions of a built-in bot playing the game, not for human demonstrations. Currently there are no released examples of such environments, but there might be someday.

The ones to use for recording human demonstrations are examples/vnc_recorder.py and examples/reward_recorder.py.

Documentation's a bit thin on these. Basically, you start a remote env (see https://github.com/openai/universe/blob/master/doc/remotes.rst) and then start both a vnc_recorder and reward_recorder. Give both of them the same --logfile-dir /tmp/demo1234. Then you connect your VNC client (TurboVNC works best) to the vnc_recorder's proxy port (typically localhost:5899) and play the game. Then you should find files, something line /tmp/demo1234/rewards.jsonl and /tmp/client.fbs and /tmp/server.fbs.

liber145 commented 7 years ago

@tlbtlbtlb Thanks a lot! I followed your steps, and I can get client.fbs and server.fbs in /tmp/demo1234/ now. But I could not find rewards.jsonl anywhere.

I start vnc_recorder and reward_recorder, then connect VNC client to the default port by mac's built-in VNC and play the game. However, there is no output in the terminal of reward_recorder. And in the terminal of vnc_recorder, it shows that

[2017-02-06 15:23:48,187] [3] Closing [2017-02-06 15:23:48,188] [3] Copying rewards.demo into this connection's log dir [2017-02-06 15:23:48,188] /tmp/demo/rewards.demo does not exist; not copying into recording directory [2017-02-06 15:23:48,188] /tmp/demo/botactions.jsonl does not exist; not copying into recording directory [2017-02-06 15:23:48,188] /tmp/demo/env_id.txt does not exist; not copying into recording directory

There are some files not existed. I guess that I may do something wrong in the above operations.

Questions:

It is OK without rewards.jsonl file?
How to read these binary files clients.fbs, sever.fbs?

tlbtlbtlb commented 7 years ago

It's misleading that these scripts are in the examples directory -- they aren't examples of anything you can actually do yet. There are major parts of this demo system we haven't finished or released, especially the code for reading and replaying the demos.

Hopefully we'll get this working soon so we can release the replay code and it'll be usable for demonstration learning. Until then, you're welcome to hack around of course and see what you can extract from the files, but we can't put much effort into supporting it.

liber145 commented 7 years ago

OK. Thanks for your advice. The Universe is a very great job! I have learned something about docker and anaconda while using Universe, and I will keep learning how to use it. 👍

openai / universe