Dockerize application

[enric1994]

Hi, I am trying to run the repo in a Docker. That can make the installation smoother. I get the following error:

root@de47041b77ca:/n64# python2 test.py 
[2019-01-25 23:40:55,630] Making new env: Smash-Discrete-pikachu-v0
validate sub
('ControllerHTTPServer started on port ', 8082)
Initially on DISPLAY unix:0
Starting xvfb with command: ['Xvfb', ':0', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']

Using DISPLAY :0
Changed to DISPLAY :0
Starting emulator with comand: ['vglrun', '-d', ':0', 'mupen64plus', '--nospeedlimit', '--nosaveoptions', '--resolution', '640x480', '--gfx', 'mupen64plus-video-rice.so', '--audio', 'dummy', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/n64/gym_mupen64plus/ROMs/smash.n64']
Traceback (most recent call last):
  File "test.py", line 19, in <module>
    main()
  File "test.py", line 5, in main
    env = gym.make('Smash-Discrete-pikachu-v0')
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 161, in make
    return registry.make(id)
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 119, in make
    env = spec.make()
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 86, in make
    env = cls(**self._kwargs)
  File "/n64/gym_mupen64plus/envs/Smash/discrete_envs.py", line 57, in __init__
    opponent_bot_level=opponent_bot_level, map=map)
  File "/n64/gym_mupen64plus/envs/Smash/smash_env.py", line 51, in __init__
    super(SmashEnv, self).__init__()
  File "/n64/gym_mupen64plus/envs/mupen64plus_env.py", line 80, in __init__
    input_driver_path=self.config['INPUT_DRIVER_PATH'])
  File "/n64/gym_mupen64plus/envs/mupen64plus_env.py", line 307, in _start_emulator
    stderr=subprocess.STDOUT)
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Close called!

You can reproduce the issue by cloning my fork: https://github.com/enric1994/gym-mupen64plus

Place the smash.n64 in the ROMs folder. Then run docker-compose up -d to start the container, then docker exec -it n64 bash to open the container command line and finally python2 test.py

Thanks

[bzier]

Hey @enric1994, sorry for the delay. I've been away from this project, and my GitHub in general, for a little while. I'll try to take a look at this soon and let you know what I find out.

[bzier]

I haven't had a chance to look at this closely yet. Been a busy week. I have had gym working with Docker for a long time. I never really finished cleaning it up, but it was functional. Have you seen my Dockerize branch? If not, take a look at it and see if you can use any of what's there. Let me know if you've already seen it, or if that doesn't help.

[enric1994]

That's great! You already Dockerized the repo! I will try it during the next days.

Million thanks!

[enric1994]

Hello!

I have tried to run the Docker branch but I got this error:

docker run -it \          
  --name test-gym-env \
  -p 5900 \
  -v $(pwd)/gym_mupen64plus/ROMs:/src/gym-mupen64plus/gym_mupen64plus/ROMs \
  n64 \
  bash

#!/bin/python
import gym, gym_mupen64plus

env = gym.make('Mario-Kart-Luigi-Raceway-v0')
env.reset()
env.render()

for i in range(88):
    (obs, rew, end, info) = env.step([0, 0, 0, 0, 0]) # NOOP until green light
    env.render()

for i in range(100):
    (obs, rew, end, info) = env.step([0, 0, 1, 0, 0]) # Drive straight
    env.render()

raw_input("Press <enter> to exit... ")

env.close()

root@8f0e0a371bbd:/src/gym-mupen64plus# python mytest.py 
[2019-03-03 22:08:36,330] Making new env: Mario-Kart-Luigi-Raceway-v0
('ControllerHTTPServer started on port ', 8082)
Initially on DISPLAY :0
Starting xvfb with command: ['Xvfb', ':0', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']

Using DISPLAY :0
Changed to DISPLAY :0
Starting emulator with comand: ['vglrun', '-d', ':0', 'mupen64plus', '--nospeedlimit', '--resolution', '640x480', '--audio', 'dummy', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/src/gym-mupen64plus/gym_mupen64plus/ROMs/marioKart.n64']
Calling mss.mss() with DISPLAY :0
 __  __                         __   _  _   ____  _             
|  \/  |_   _ _ __   ___ _ __  / /_ | || | |  _ \| |_   _ ___ 
| |\/| | | | | '_ \ / _ \ '_ \| '_ \| || |_| |_) | | | | / __|  
| |  | | |_| | |_) |  __/ | | | (_) |__   _|  __/| | |_| \__ \  
|_|  |_|\__,_| .__/ \___|_| |_|\___/   |_| |_|   |_|\__,_|___/  
             |_|         http://code.google.com/p/mupen64plus/  
Mupen64Plus Console User-Interface Version 2.5.0

UI-Console: attached to core library 'Mupen64Plus Core' version 2.5.0
UI-Console:             Includes support for Dynamic Recompiler.
UI-Console:             Includes support for MIPS r4300 Debugger.
Core: Goodname: Mario Kart 64 (E) (V1.1) [!]
Core: Name: MARIOKART64         
Core: MD5: 2BB149A583FDEFEA96805F628FE42FD9
Core: CRC: 2577C7D4 D18FAAAE
Core: Imagetype: .z64 (native)
Core: Rom size: 12582912 bytes (or 12 Mb or 96 Megabits)
Core: Version: 1446
Core: Manufacturer: Nintendo
Core: Country: Unknown (0x150)
UI-Console Status: Cheat codes disabled.
Video Warning: No version number in 'Rice-Video' config section. Setting defaults.
Video Warning: Old parameter config version detected : 0, updating to 1;
UI-Console: using Video plugin: 'Mupen64Plus OpenGL Video Plugin by Rice' v2.5.0
UI-Console: using Audio plugin: <dummy>
UI-Console: using Input plugin: 'Mupen64Plus Bot Input Plugin' v0.0.1
UI-Console: using RSP plugin: 'Hacktarux/Azimer High-Level Emulation RSP Plugin' v2.5.0
Input: Mupen64Plus Bot Input Plugin version 0.0.1 initialized.
Core Warning: No audio plugin attached.  There will be no sound output.
Video: SSE processing enabled.
Video: ROM (CRC d4c77725aeaa8fd1-50) not found in INI file
Video: Enabled hacks for game: 'MARIOKART64'
Video: Initializing OpenGL Device Context.
Core: Setting 32-bit video mode: 640x480
Video Warning: Failed to set GL_SWAP_CONTROL to 0. (it's 24)
Video Warning: Failed to set GL_BUFFER_SIZE to 32. (it's 24)
Video Warning: Failed to set GL_DEPTH_SIZE to 16. (it's 32)
Video: Using OpenGL: VMware, Inc. - llvmpipe (LLVM 6.0, 256 bits) : 3.0 Mesa 18.0.5
Core: Starting R4300 emulator: Dynamic Recompiler
Core: R4300: starting 64-bit dynamic recompiler at: 0x7fe3751036e0
Changed back to DISPLAY :0
Player row: 0
Player col: 0
Map series: 0
Map choice: 0
Reset called!
[2019-03-03 22:08:47,173] Making new env: Mario-Kart-Luigi-Raceway-v0
Traceback (most recent call last):
  File "mytest.py", line 21, in <module>
    main()
  File "mytest.py", line 7, in main
    env.render()
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 174, in render
    return self._render(mode=mode, close=close)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 341, in _render
    return self.env.render(mode, close)
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 174, in render
    return self._render(mode=mode, close=close)
  File "/src/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 155, in _render
    from gym.envs.classic_control import rendering
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/classic_control/rendering.py", line 23, in <module>
    from pyglet.gl import *
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/__init__.py", line 100, in <module>
    from pyglet.gl.lib import GLException
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/lib.py", line 143, in <module>
    from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX
  File "/usr/local/lib/python2.7/dist-packages/pyglet/gl/lib_glx.py", line 43, in <module>
    import pyglet.lib
  File "/usr/local/lib/python2.7/dist-packages/pyglet/lib.py", line 61, in <module>
    script_path = pyglet.resource.get_script_home()
  File "/usr/local/lib/python2.7/dist-packages/pyglet/__init__.py", line 384, in __getattr__
    __import__(import_name)
  File "/usr/local/lib/python2.7/dist-packages/pyglet/resource.py", line 88, in <module>
    standard_library.install_aliases()
  File "/usr/local/lib/python2.7/dist-packages/future/standard_library/__init__.py", line 485, in install_aliases
    import test
  File "/src/gym-mupen64plus/test.py", line 4, in <module>
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 161, in make
    return registry.make(id)
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 119, in make
    env = spec.make()
  File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 86, in make
    env = cls(**self._kwargs)
  File "/src/gym-mupen64plus/gym_mupen64plus/envs/MarioKart64/mario_kart_env.py", line 46, in __init__
    super(MarioKartEnv, self).__init__(mk_config['ROM_NAME'])
  File "/src/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 63, in __init__
    self.controller_server, self.controller_server_thread = self._start_controller_server()
  File "/src/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 168, in _start_controller_server
    config['ACTION_TIMEOUT'])
  File "/src/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 348, in __init__
    super(ControllerHTTPServer, self).__init__(server_address, self.ControllerRequestHandler)
  File "/usr/lib/python2.7/SocketServer.py", line 417, in __init__
    self.server_bind()
  File "/usr/lib/python2.7/BaseHTTPServer.py", line 108, in server_bind
    SocketServer.TCPServer.server_bind(self)
  File "/usr/lib/python2.7/SocketServer.py", line 431, in server_bind
    self.socket.bind(self.server_address)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
Close called!
Close called!

Then I exposed the port 5900 like this: -v 5900:5900 and a similar error appeared:

root@e308dd35c43d:/src/gym-mupen64plus# python mytest.py 
[2019-03-03 22:29:22,811] Making new env: Mario-Kart-Luigi-Raceway-v0
('ControllerHTTPServer started on port ', 8082)
Initially on DISPLAY :0
Starting xvfb with command: ['Xvfb', ':0', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']

Using DISPLAY :0
Changed to DISPLAY :0
Starting emulator with comand: ['vglrun', '-d', ':0', 'mupen64plus', '--nospeedlimit', '--resolution', '640x480', '--audio', 'dummy', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/src/gym-mupen64plus/gym_mupen64plus/ROMs/marioKart.n64']
Calling mss.mss() with DISPLAY :0
 __  __                         __   _  _   ____  _             
|  \/  |_   _ _ __   ___ _ __  / /_ | || | |  _ \| |_   _ ___ 
| |\/| | | | | '_ \ / _ \ '_ \| '_ \| || |_| |_) | | | | / __|  
| |  | | |_| | |_) |  __/ | | | (_) |__   _|  __/| | |_| \__ \  
|_|  |_|\__,_| .__/ \___|_| |_|\___/   |_| |_|   |_|\__,_|___/  
             |_|         http://code.google.com/p/mupen64plus/  
Mupen64Plus Console User-Interface Version 2.5.0

UI-Console: attached to core library 'Mupen64Plus Core' version 2.5.0
UI-Console:             Includes support for Dynamic Recompiler.
UI-Console:             Includes support for MIPS r4300 Debugger.
Core: Goodname: Mario Kart 64 (E) (V1.1) [!]
Core: Name: MARIOKART64         
Core: MD5: 2BB149A583FDEFEA96805F628FE42FD9
Core: CRC: 2577C7D4 D18FAAAE
Core: Imagetype: .z64 (native)
Core: Rom size: 12582912 bytes (or 12 Mb or 96 Megabits)
Core: Version: 1446
Core: Manufacturer: Nintendo
Core: Country: Unknown (0x150)
UI-Console Status: Cheat codes disabled.
Video Warning: No version number in 'Rice-Video' config section. Setting defaults.
Video Warning: Old parameter config version detected : 0, updating to 1;
UI-Console: using Video plugin: 'Mupen64Plus OpenGL Video Plugin by Rice' v2.5.0
UI-Console: using Audio plugin: <dummy>
UI-Console: using Input plugin: 'Mupen64Plus Bot Input Plugin' v0.0.1
UI-Console: using RSP plugin: 'Hacktarux/Azimer High-Level Emulation RSP Plugin' v2.5.0
Input: Mupen64Plus Bot Input Plugin version 0.0.1 initialized.
Core Warning: No audio plugin attached.  There will be no sound output.
Video: SSE processing enabled.
Video: ROM (CRC d4c77725aeaa8fd1-50) not found in INI file
Video: Enabled hacks for game: 'MARIOKART64'
Video: Initializing OpenGL Device Context.
Core: Setting 32-bit video mode: 640x480
Video Warning: Failed to set GL_SWAP_CONTROL to 0. (it's 24)
Video Warning: Failed to set GL_BUFFER_SIZE to 32. (it's 24)
Video Warning: Failed to set GL_DEPTH_SIZE to 16. (it's 32)
Video: Using OpenGL: VMware, Inc. - llvmpipe (LLVM 6.0, 256 bits) : 3.0 Mesa 18.0.5
Core: Starting R4300 emulator: Dynamic Recompiler
Core: R4300: starting 64-bit dynamic recompiler at: 0x7fd65f46e6e0
Changed back to DISPLAY :0
Player row: 0
Player col: 0
Map series: 0
Map choice: 0
Reset called!
Press <enter> to exit... 

Do I need any tool/package installed in my host machine to visualize the game (I have read something related with VNC and XVFB)? I have a brand new Ubuntu 18.04

Thanks for your time and sorry for the long logs :)

[bzier]

The first error you got, socket.error: [Errno 98] Address already in use, occurs when the controller server attempts to open an http listening port that is already in use. This can happen sometimes if you have an attempt that failed and the process crashed, but didn't properly clean up. Typically this doesn't happen within a docker container because you would just spin up a new one (different ports in a different container). However, it looks like you are running the commands from a bash prompt within the container, so my assumption is that you had tried it before, it didn't clean up, then your subsequent attempt failed with that error.

The second output you provided doesn't actually have any errors. That one successfully ran your mytest.py script to completion. With that container running, before hitting enter to exit, attempt to connect to it using a VNC client and you should see Mario frozen a little ways down the track. I think my README includes info about connecting with a VNC client. With Docker, you need to use the container IP address, unless you map the port to the host (as you did with -p 5900:5900). If you'd like to see Mario driving, just simply adjust the script to drive straight in a while loop instead of the for loop, and you should see him smashing into the wall down the track.

Let me know if you have questions, or if it doesn't work for you. Any feedback on the README, or docker setup is certainly welcome. Eventually I'd like to merge that branch in and make it the primary way to get up and running, but I want to make sure it is clear and easy before I do.

[enric1994]

Yey! It is working!

1. docker-compose up -d
2. Open a VNC client (like Vinagre) and connect to 127.0.0.1:5900

I updated a docker-compose with the example script to my fork: https://github.com/enric1994/gym-mupen64plus/tree/dockerize

I know that is not a really "clean" solution (exposing the entire container to the host is not a good practice) but at least it is only 2 steps.

It would be great to visualize the game in the browser instead of VNC, it can be tricky for some dummy users like me :sweat_smile: . Another option would be automatically open a window on the host system. If you know any approach to do that I can invest it.

There is another problem, Docker philosophy is running only one task. But in our container we want to run the VNC server + the python script. But the VNC server must be on the same container, right?

I will start training my Super Smash Bros characters now :smile: I am really excited about the benefits of running that inside a Docker. I can create 5 containers that train my favourite characters in parallel!!

Thanks for all your help

[kevinhughes27]

What is left to merge the docker branch into master? maybe @enric1994 could help finish getting it merge ready.

I would't worry too much about the docker concerns. Exposing the entire container is fine because there is nothing sensitive going on. If it is an easy fix or documentation thing then let's address it. For multiple processes Docker does support this just fine it is just less common and we do need them to be in the some container. Lets not mistakenly apply best practises for running sass applications in the cloud with how we are using docker for env managment and being able to easily spin this up on any machine.

[enric1994]

Yes, I would be very happy to contribute, there is a lot of stuff to do in the Dockerize branch. At the moment I am trying to figure out why Mario is completely stopped.

I get your point @kevinhughes27 , we are not in production so there is no need to worry about fancy best practices.

[kevinhughes27]

Awesome! :heart:

Hope you figure out what's up with Mario :)

[roclark]

Hey guys! I know this is an old post that I'm resurrecting, but I am stumbling a bit here and need some help.

I have a container up and running, and it generates similar output to @enric1994's output above (the second block above) and everything appears to be functional. I mapped port 5900 on the host to the container and added /tmp/.X11-unix as a volume (as seen below).

docker run --rm -it --name test-gym-env -p 5900:5900 \
    --mount source=/home/roclark/gym-mupen64plus/gym_mupen64plus/ROMs,target=/src/gym-mupen64plus/gym_mupen64plus/ROMs,type=bind \
    -v /tmp/.X11-unix:/tmp/.X11-unix roclark/gym-mupen64plus

Once up, I can see my socket on the host (the X1 socket is a separate VNC connection for remoting into the computer):

$ ls -la /tmp/.X11-unix/
total 8
drwxrwxrwt  2 root root 4096 Sep 12 17:10 .
drwxrwxrwt 14 root root 4096 Sep 12 17:33 ..
srwxrwxrwx  1 root root    0 Sep 12 17:10 X0
srwxrwxrwx  1 root root    0 Sep 12 12:32 X1

Now, when I try and connect to the socket using Vinagre, I immediately get "127.0.0.1 was closed" and no other information.

I am stuck at this point, so figured I would ask if there was anything else either of you did to get this up and running and connect to the emulator using a VNC client. Is there anything else I need to setup to get this working?

Thanks for any help!

[kevinhughes27]

No worries! as old as this thread is, it is also the cutting edge of what we were trying to do here :)

So the VNC client quits right away, is there any output from the container when you try and connect?

Can you clarify exactly what code/branch you are using and which OS?

I think we should merge the Dockerize branch since even if we are not 100% confident in it. At this point the native install is also pretty outdated and possibly broken so at least with Dockerize it will be easier to help people get going.

[roclark]

Thanks for the quick response @kevinhughes27!

So the VNC client quits right away, is there any output from the container when you try and connect?

Unfortunately no - the container keeps running with no indication that anything went wrong.

Can you clarify exactly what code/branch you are using and which OS?

I created a fork which basically just takes the Dockerfile from the dockerize branch here and merges it with the code in master to get both updates. I am running this on Ubuntu 18.04.

I think that's a great idea on merging the dockerize branch! That will certainly help new people like me get up and running.

Also, as an update - I still can't connect with the VNC client, but I attached gym's Monitoring wrapper (with some adjustments to the code here) to my environment and am able to pull a video of the test run. This scenario is probably sufficient for me as I mainly like to review recorded videos after the fact, but I noticed in the video that everything is frozen as if the state never changes (ie. the first image is static throughout the video). Is there a way to debug if the step function is actually progressing the frames?

Thanks again for the help!

[roclark]

I manually checked the observation returned from step() and noticed that it stayed static throughout the test, which likely explains why my video appears frozen. I wonder if this is similar to what @enric1994 mentioned above with Mario being stopped, or if the environment was changing, but Mario just chose not to move? Either way, looks like I won't be able to verify the VNC issue until I get the environment to advance after each step.

Thanks again for the help here, I greatly appreciate it!

[bzier]

You should be able to run a script like this to know that Mario is supposed to move (as opposed to an AI agent choosing to stay still). This should help in determining if it is an issue with the agent or the environment.

I haven't worked on this project in quite a while, but really wanted to get the docker branch cleaned up and merged in, to be the 'normal' way to run this. If you are able to make progress on this, please do provide updates here. I'm happy to review and merge a PR.

[roclark]

Thanks for the help @bzier! I actually have been using that script as a reference to see if I could get him to move forwards, but he’s still stuck. Looking at the recorded video (and the actual pixel array from the observation), Lakitu isn’t moving at the beginning, and the start light doesn’t change, so I am pretty sure this is the environment being stuck somehow, but am unsure how to move forward with troubleshooting. As a related question, I am not very experienced with Xvfb. I noticed in /dev/shm that I have a virtual frame buffer file after starting the script, but it doesn’t seem to be updating at all. For example, if I modify the script to run for 5 minutes, the file will still be the original one created at the beginning of execution. Is this normal behavior, or should the virtual frame buffer update with the new screens?

I think we all know the feeling of jumping around on projects! 😄 I certainly do that a lot. I appreciate the great work and help here! If I get it running and if any changes are necessary, I am happy to create a PR so that others may benefit.

Thanks again everyone!

[roclark]

I apologize for the spam, but wanted to add a bit more context to the conversation.

I built a container targeting the dockerize branch in this repo (ie. without the latest commits to master that I've been testing with) and am still experiencing the same issue where the screen appears frozen, so we can rule out any of the more recent commits to master being incompatible here.

Also, I tested this with Super Smash Bros. and saw the same behavior (stuck on the calibration screen) so this shouldn't be related to a ROM either.

Currently, I'm still suspicious about the Xvfb issue I mentioned above. Meanwhile, I will look at how future observations are pulled and see if anything is amiss there. I don't have much experience with emulators, so I'm a little out of my element, but I appreciate the help!

[kevinhughes27]

Keep us posted!

Also since you've already rebased and are running the dockerize branch do you want open a PR? We can put a note in the readme that docker setup is in beta still. I don't think anything in the branch breaks the ability to run without docker but I haven't thoroughly checked. I really think it will help to get everyone on the same master branch to continue with debugging.

[roclark]

I can certainly create a PR with the merge, but I also would feel better waiting until it's properly functional - but that's just me. If the group here desires, I will be happy to go forward with the PR though!

I've spent a healthy amount of time looking into this over the weekend and have scratched a few more things off the list, but am still not out of the water yet. Here's what I've found:

1. I can now connect to the emulator over VNC on my host - I had to run startx, Xvfb, and x11vnc before calling the test script (after changing the config to not use Xvfb and manually setting the image offset to 0,0). Still can't figure out why it isn't working when the package directly calls Xvfb though.

2. I verified that there isn't an issue with running the emulator itself in the container as I am able to view a live output (with no inputs mind you) with the following command inside the container:

   /usr/games/mupen64plus --configdir /config/mupen64plus \
   --fullscreen gym_mupen64plus/ROMs/marioKart.n64

   After connecting to the emulator via VNC, I can see the Mario Kart 64 title screen and the animated title video when you don't press start after a few seconds, so the emulator appears to work just fine inside the container.

3. I also verified that I can grab screenshots of the emulator targeting the correct screen while using mss (which this project appears to use upon diving into the code) by using a very dumb script which grabs an image from the screen and saves it as a PNG. The saved PNG matches with what was most recently visible on the emulator, so the process of grabbing screenshots checks out as well.

With these checks, I feel the most likely culprit at this point is the code in the actual environment for this project. We know the code is able to navigate the menus just fine as we get into the race start screen without an issue after calling reset(). I wonder if there is some locking mechanism or screen freeze that occurs right after finishing the reset() call which prevents the screen (and in-turn virtual buffer) from updating. Unfortunately, since I haven't fully familiarized myself with the code in this repository yet, I don't know where to look as to why this would be happening. I know priorities and obligations have likely shifted, but would it be possible for one of you to help look into why it would be frozen? I will keep digging, but can't say if I will be successful or not.

I really enjoy this project and think it's a wonderful contribution to the community, and I am happy to help expand it if possible! Thank you to everyone who put the work in to get this running!

[bzier]

Thanks for all the info @roclark. I will try to take a look at it tonight if I can and let you know what I come up with.

As far as locking/blocking goes, the environment itself waits for the HTTP get request from the emulator for the controller data. If the emulator doesn't send this request, or if the environment's controller server doesn't receive it, it will block. However, I believe there is a timeout configured for how long it waits.

I don't suspect that's the problem since it is able to successfully navigate the menus, but it's worth mentioning/understanding.

BTW, do you have the log output from the container that you could share? This is usually fairly verbose, so if you do, please use a paste bin or Dropbox or something rather than cluttering this thread with it.

Thanks

[roclark]

Hey @bzier, thanks for the response! I actually managed to get it working earlier this morning! Turns out, the render() call was causing issues. Removing all calls to render() in the test script enabled it to complete as expected and Mario drove straight down the track and crashed repeatedly into the first wall until the script ended.

I don't have a very technical explanation as to why render() isn't working, but my assumption is that directing the output of mupen64plus to Xvfb is already occurring, so the secondary call to render() causes the buffer and/or the emulator to appear stuck.

At this point, I think things are looking good! There are a few more things I would want to cleanup in the container (ie. further testing with Xvfb using the script as opposed to my workaround and upgrading to a newer version of Ubuntu where Python 3 is default) on my end, but I will be happy to create a PR once ready!

Also, another question - I used the sample script that you provided and initially had Mario wait 84 frames before accelerating. After looking at the output, he appears to sit at the starting line for a couple seconds before actually moving down the track. I'm wondering, was 84 frames originally set to correspond with the exact frame that the light turns green, or was it just some known frame after the light is green, but not necessarily immediate? My concern is if it was originally set to be exactly 84 frames that the new changes I made somehow desynchronize the emulator and the code, or that the frames are offset. Do you see anything concerning here?

Thanks again to everyone for the work on this! Now that everything's running, I'm excited to get some testing in when I get a chance! I will keep this updated when I complete the Docker changes.

[enric1994]

Amazing work @roclark ! Can't wait to start a Reinforcement Learning project on SSB!

[bzier]

@roclark, yes the render issue totally makes sense. It was rendering the image of the current frame over top of the emulator. As a result, all subsequent calls to observe were a screenshot of the render and not of the emulator screen. Mario was moving the whole time, but you couldn't see it :)

The example script should not have the call to render when running in docker. Probably one of several things that still needs to be updated before merging that docker branch.

@enric1994, FYI, I have been able to successfully train an AI on Mario, but never on SSB. My guess is that more work needs to be done on the reward function to make it work, but I honestly haven't spent any time on it. I wrote most everything on Mario (or stole it from @kevinhughes27 😉), but someone else wrote and contributed all the SSB stuff. I only really verified that it ran successfully and provided some feedback on the code/organization. I let my Mario agent train on SSB for a while with no indication of successful learning. Could be a different model is necessary, or simply more training time, but my guess is that the reward function needs tuning.

I'm excited to have you guys active in this. It's been a fun project, but shelved for a while. Keep us posted on progress 👍

[kevinhughes27]

Great work! I am glad to hear you got it working :smile:

Looking forward to that PR and hearing what you get up to :rocket:

[roclark]

Thanks everyone for the feedback! As most of you have likely been notified, I created PR #69 which should wrap this up nicely. Overall, there were only very minor tweaks to the Dockerfile that @bzier already created as well as some documentation modifications.

Glad that we could work together to get this resolved! I'm loving this project and would be happy to make further updates if necessary!

[roclark]

Also, as an idea, are there any thoughts on pushing an image to Docker Hub? That would make it even quicker/easier for new users to get the latest image and start running some tests.

[kevinhughes27]

I don't see why not, push it!

The only concern I have is around not distributing ROM files since that could get you into trouble

[bzier]

The ROM files are not included in the image build. The ROMs directory is currently exposed as a volume from the container. I've typically used a bind mount to mount a directory from my host with my ROMs inside. So should be 'safe' I would think.

[kevinhughes27]

Sorry my comment wasn't super clear - Yup the current setup is totally good. I just to want to make sure we continue to do so if we publish an image

[roclark]

As a follow-up, I am inexperienced with some of these topics - are there any issues with including a save file in an image? The Super Smash Bros. save file (which is included in this repository, so I would assume we are OK) is copied as one of the steps, so it will be distributed along with the image. Any concerns there?

Also, since this is @bzier's baby, I want to make sure he has credit and it's posted under his name on Docker Hub. I'd be happy posting it under my account, but want to make sure credit is directed properly. 😃 We can track this with another issue as well, but I would assume we'd want to have a more formal process of versioning/releasing too, unless it already exists or would be too much of a hassle.

[kevinhughes27]

Save file is fine as far as I know!

[bzier]

The save file shouldn't be a problem. It's already included in the repo here anyway.

I haven't looked into or thought much about the versioning, releasing, docker publishing, etc. I'll try to take a look at this in the next week or two. Need to see if there's an easy way to set it up for CI/CD with a key for pushing to docker hub under my account, without exposing my credentials. I haven't published an image to docker hub before, so just need to work through the "right way" to do it.

[bzier]

Closed with the merge of #69 😀👍

[enric1994]

@roclark How can I try the new Dockerized version? I am building and running the container using the provided command but when connecting with the VNC client to 127.0.0.1:5900 it says connection closed.

I can verify that the game is running because I added some prints, and I also mapped the port 5900 to 5900 in my host.

What am I missing? Do I have to run x11vnc in the container?

[bzier]

Hey @enric1994,

Yes, you will need to run x11vnc in the container if you want to connect with vnc. The instructions should be similar to this. You can connect to a shell in your container with something like: docker exec -it <container_name> bash

At some point, I had this happening automatically with an entrypoint script, but I guess this was just locally because I don't see it anywhere here in the repo. I have some other thoughts now on ways this 'should' work, which I will think more about, but for now, the above should do the trick. Let us know

[enric1994]

@bzier Yey! I just added this and it is working perfectly: subprocess.call("x11vnc -display :1 -localhost -forever -viewonly &", shell=True) For some reason, it only works when I use display 1.

@roclark By the way, I created a Docker Compose (again) that simplify the process of verifying the environment with verifyEnv.py, so you only need to run docker compose up and start the VNC client (I think we should add a step by step guide to setup and connect the VNC client, it's a bit confusing at the moment). I think is very important to make the demo as easy as possible, so the new users with no Linux background can start using the repo in less than 2 minutes.

Feel free to try it in my fork. I can create a PR if you want

[kevinhughes27]

Definitely open a PR with docker-compose and the example script! This will also help if we setup a CI server #71

[bzier]

I have some thoughts about the long term docker compose architecture, but for now, please feel free to put in a PR. I haven't been spending time on this for a while, so anything that makes it easier for people is a step in the right direction 👍

[bzier]

Also, another question - I used the sample script that you provided and initially had Mario wait 84 frames before accelerating. After looking at the output, he appears to sit at the starting line for a couple seconds before actually moving down the track. I'm wondering, was 84 frames originally set to correspond with the exact frame that the light turns green, or was it just some known frame after the light is green, but not necessarily immediate? My concern is if it was originally set to be exactly 84 frames that the new changes I made somehow desynchronize the emulator and the code, or that the frames are offset. Do you see anything concerning here?

@roclark, just realized I never responded to this question you had asked. The 88 frame NOOP in the example script did correspond to the exact frame the light turned green at some point. I ultimately baked in the frame skip mechanism, which (I think) is 5 frames. So every step in the gym environment corresponds to 5 emulator frames. I think this is configurable if I remember correctly. I thought I had accounted for the frame skip and adjusted the NOOP in the example script, but perhaps I didn't. Either way, the intent is to provide an example of how to use it, and verify that things are up and running. I'm not too worried about it being a few seconds off. If you see any other indications of things being out of sync, we can address it with another issue, but my guess is that the frame count is just a bit off and no big deal.

[roclark]

Sorry I've been MIA! Glad you were able to resolve your issue @enric1994! I think the docker-compose idea is great and am excited to try it out!

Also, thanks for the explanation @bzier! After diving deeper in the code, the frame skipping makes sense. This would also align with what I found during my tests. It appeared that the light turned green after approximately 17 frames (88 / 5 = 17.6), so that would account for the difference. With that being the case, I think we are good here! Just wanted to check that we weren't getting out-of-sync, but that doesn't sound like it's the case.