Unable to access the X Display, is $DISPLAY set properly?

[frogoscar]

Hello,

I have recorded my game (luigi_raceway) and trained the data.csv to generate the model_weights.h5 file.

When I run "python play.py", it shows the followings:

Using TensorFlow backend.
[2017-10-09 07:48:53,027] Making new env: Mario-Kart-Royal-Raceway-v0
('ControllerHTTPServer started on port ', 8082)
initial_disp: :1
Starting xvfb with command: ['Xvfb', ':1', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
(EE) 
Fatal server error:
(EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE) 
Changed DISPLAY to: :1
Starting emulator with comand: ['vglrun', 'mupen64plus', '--resolution', '640x480', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/home/exe/gym-mupen64plus/gym_mupen64plus/ROMs/marioKart.n64']
Calling wx.App() with DISPLAY: :1
Unable to access the X Display, is $DISPLAY set properly?
Close called!

I have searched the Google, but can not find correct answers.

How can I get access to X Display ? Thanks a lot.

[frogoscar]

The problem seems to be that I have X Server at

:0

which I know from running command

xdpyinfo

, but

Starting xvfb with command: ['Xvfb', ':1', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']

wants to connect to X Server :1

So it failed.

it also did

Changed DISPLAY to: :1

But it does not find :1 because it is :0 actually.

How could I let it run with

Starting xvfb with command: ['Xvfb', ':0', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']

[bzier]

X Display :0 is your primary X11 display. This is what is used by default to display your desktop environment (i.e. all of your windows). The gym environment is attempting to establish a virtual display using XVFB (X Virtual Frame Buffer) to run the N64 emulator. Because most people are running a desktop environment on display :0, I defaulted the configuration to use display :1. In other words, the environment is attempting to spawn a new X display on :1

Based on the error message you provided, it looks like the display :1 is already in use. This can happen if you are running a (non-standard) setup with multiple X displays (e.g. multi-monitor with each monitor having its own X server... probably not the case), or more likely, if you had attempted to run the environment and had it crash at some point where it didn't clean up properly. Unfortunately, there are a few cases where the environment dies and doesn't close the Xvfb process, leaving the X display in use. Try running ps | grep -i xvfb and see if it shows a running Xvfb process. If so, you will need to kill it. That should free up the X display for you to try again. If you are actually using X Display :1 for something, you can change the config.yml file to reference a different display (e.g. :3).

Hopefully that makes some sense. Let me know if you have any questions about any of that and I can try to explain it better.

[frogoscar]

@bzier

Thank you so much for your quick answer and kind help.

I am using Ubuntu 16.04 in VirtualBox. Is it changing something when Ubuntu is in Virtual Machine ?

I ran ps | grep xvfb, it shows nothing.

I changed the file config.yml to let $DISPLAY to be :3, but it does not work :

Using TensorFlow backend.
[2017-10-09 23:34:35,274] Making new env: Mario-Kart-Luigi-Raceway-v0
('ControllerHTTPServer started on port ', 8082)
initial_disp: :0
Starting xvfb with command: ['Xvfb', ':3', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
(EE) 
Fatal server error:
(EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE) 
Changed DISPLAY to: :3
Starting emulator with comand: ['vglrun', 'mupen64plus', '--resolution', '640x480', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/home/exe/gym-mupen64plus/gym_mupen64plus/ROMs/MarioKart64']
Calling wx.App() with DISPLAY: :3
Unable to access the X Display, is $DISPLAY set properly?
Close called!

Usage: vglrun [options] [--] <OpenGL app> [OpenGL app arguments]

VirtualGL options (see documentation for a more comprehensive list)

-c <c>    : proxy = Send 3D images uncompressed using X11 Transport
                    [default for local X connections]
            jpeg = Compress 3D images using JPEG/send using VGL Transport
                   [default for remote X connections]
            rgb = Encode 3D images as RGB/send using VGL Transport
            xv = Encode 3D images as YUV420P/send using XV Transport
            yuv = Encode 3D images as YUV420P/send using the VGL Transport
                  and display on the client using X Video
            [If an image transport plugin is being used, then <c> can be any
             number >= 0 (default=0).]

-nodl     : Don't interpose the dlopen() function.  dlopen() is normally
            interposed to force applications that use dlopen() to load libGL
            to load VirtualGL instead.  For the more common case of
            applications that link directly with libGL, disabling the dlopen()
            interposer makes VirtualGL less intrusive, since it will no longer
            load libGL until the application actually uses that library.

-d <d>    : <d> = the X Display to use for 3D rendering [default = :0.0]

-fps <f>  : Limit client/server frame rate to <f> frames/sec

-gamma <g>: Set gamma correction factor to <g> (see docs)

-ge       : Fool application into thinking that LD_PRELOAD is unset

-ms <s>   : Force VirtualGL to use OpenGL multisampling with <s>
            samples (<s> = 0 forces multisampling to be disabled)

-np <n>   : Use <n> CPUs to perform image compression [default = 1]

+/-pr     : Enable/disable performance profiling output [default = disabled]

-q <q>    : Compression quality [1 <= <q> <= 100]
            [default = 95 for JPEG/VGL Transport.  Has no effect with
             X11 Transport]

-samp <s> : Chrominance subsampling factor
            <s> = gray, 1x, 2x, 4x
            [default = 1x for JPEG/VGL Transport.  Has no effect with
             X11 Transport or RGB encoding]

+/-s      : Enable/disable SSL encryption of VGL Transport or custom
            image transport, if applicable.
            [default = disabled.  Has no effect on the VGL Transport unless
             VirtualGL was built with OpenSSL support]

+/-sp     : Turn on/off frame spoiling [default = enabled]

-st <s>   : left = Send only left eye buffer of stereo application
            right = Send only right eye buffer of stereo application
            quad = Use quad-buffered stereo if available, otherwise use
                   red/cyan (anaglyphic) stereo [default]
            rc = Always use red/cyan (anaglyphic) stereo
            gm = Always use green/magenta (anaglyphic) stereo
            by = Always use blue/yellow (anaglyphic) stereo
            i = Always use interleaved (passive) stereo
            tb = Always use top/bottom (passive) stereo
            ss = Always use side-by-side (passive) stereo

+/-sync   : Enable/disable strict 2D/3D synchronization [default = disabled]

+/-tr     : Enable/disable function call tracing (generates a lot of output)
            [default = disabled]

-trans <t>: Use transport plugin contained in library libvgltrans_<t>.so

+/-v      : Enable/disable verbose VirtualGL messages [default = disabled]

+xcb/-xcb : Enable/disable XCB interposer [default = enabled]

+wm/-wm   : Enable/disable window manager mode (for running compiz, etc.)

-64       : (Solaris only) Preload VirtualGL only into 64-bit apps
-32       : (Solaris only) Preload VirtualGL only into 32-bit apps

I don't know why did it show the

Usage: vglrun [options] [--] <OpenGL app> [OpenGL app arguments]

VirtualGL options (see documentation for a more comprehensive list)

does it mean that the usage of vglrun is not correct ?

PS : The result when I ran xdpyinfo is :

name of display:    :0
version number:    11.0
vendor string:    The X.Org Foundation
vendor release number:    11903000
X.Org version: 1.19.3
maximum request size:  16777212 bytes
motion buffer size:  256
bitmap unit, bit order, padding:    32, LSBFirst, 32
image byte order:    LSBFirst
number of supported pixmap formats:    7
supported pixmap formats:
    depth 1, bits_per_pixel 1, scanline_pad 32
    depth 4, bits_per_pixel 8, scanline_pad 32
    depth 8, bits_per_pixel 8, scanline_pad 32
    depth 15, bits_per_pixel 16, scanline_pad 32
    depth 16, bits_per_pixel 16, scanline_pad 32
    depth 24, bits_per_pixel 32, scanline_pad 32
    depth 32, bits_per_pixel 32, scanline_pad 32
keycode range:    minimum 8, maximum 255
focus:  window 0x2600007, revert to Parent
number of extensions:    28
    BIG-REQUESTS
    Composite
    DAMAGE
    DOUBLE-BUFFER
    DPMS
    DRI2
    GLX
    Generic Event Extension
    MIT-SCREEN-SAVER
    MIT-SHM
    Present
    RANDR
    RECORD
    RENDER
    SECURITY
    SGI-GLX
    SHAPE
    SYNC
    X-Resource
    XC-MISC
    XFIXES
    XFree86-DGA
    XFree86-VidModeExtension
    XINERAMA
    XInputExtension
    XKEYBOARD
    XTEST
    XVideo
default screen number:    0
number of screens:    1

screen #0:
  dimensions:    1272x734 pixels (336x194 millimeters)
  resolution:    96x96 dots per inch
  depths (7):    24, 1, 4, 8, 15, 16, 32
  root window id:    0x281
  depth of root window:    24 planes
  number of colormaps:    minimum 1, maximum 1
  default colormap:    0x20
  default number of colormap cells:    256
  preallocated pixels:    black 0, white 16777215
  options:    backing-store WHEN MAPPED, save-unders NO
  largest cursor:    64x64
  current input event mask:    0xda0003

PS again : The result when I ran sudo pip install -e . in your project gym-mupen64plus :

The directory '/home/exe/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/exe/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Obtaining file:///home/exe/gym-mupen64plus
Requirement already satisfied: gym>=0.2.3 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: numpy>=1.12.0 in /home/exe/.local/lib/python2.7/site-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: PyYAML>=3.12 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: requests>=2.0 in /usr/lib/python2.7/dist-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Requirement already satisfied: six in /home/exe/.local/lib/python2.7/site-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Requirement already satisfied: pyglet>=1.2.0 in /usr/local/lib/python2.7/dist-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Installing collected packages: gym-mupen64plus
  Found existing installation: gym-mupen64plus 0.0.1
    Uninstalling gym-mupen64plus-0.0.1:
      Successfully uninstalled gym-mupen64plus-0.0.1
  Running setup.py develop for gym-mupen64plus
Exception:
Traceback (most recent call last):
  File "/home/exe/.local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/home/exe/.local/lib/python2.7/site-packages/pip/commands/install.py", line 350, in run
    isolated=options.isolated_mode,
  File "/home/exe/.local/lib/python2.7/site-packages/pip/commands/install.py", line 436, in get_lib_location_guesses
    scheme = distutils_scheme('', *args, **kwargs)
  File "/home/exe/.local/lib/python2.7/site-packages/pip/locations.py", line 140, in distutils_scheme
    d = Distribution(dist_args)
  File "/home/exe/.local/lib/python2.7/site-packages/setuptools/dist.py", line 336, in __init__
    for ep in pkg_resources.iter_entry_points('distutils.setup_keywords'):
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 735, in iter_entry_points
    entries = dist.get_entry_map(group)
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2759, in get_entry_map
    self._get_metadata('entry_points.txt'), self
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2489, in parse_map
    for group, lines in data:
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3086, in split_sections
    for line in yield_lines(s):
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2349, in yield_lines
    for ss in strs:
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2684, in _get_metadata
    if self.has_metadata(name):
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1481, in has_metadata
    return self.egg_info and self._has(self._fn(self.egg_info, name))
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1841, in _has
    return zip_path in self.zipinfo or zip_path in self._index()
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1721, in zipinfo
    return self._zip_manifests.load(self.loader.archive)
  File "/home/exe/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1661, in load
    mtime = os.stat(path).st_mtime
OSError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/gym_mupen64plus-0.0.1-py2.7.egg'

Thank you very much!

[bzier]

@frogoscar So there are a lot of variables at play here, and you provided a lot of information and a few different error messages. I'd like to focus on one at a time and see where we can get. Before we worry about what is going on with the X Display, let's make sure that the gym environment is installed correctly. Then we can start the process of elimination on some of those other issues.

Also, to keep things in one place and reduce confusion, I'd like to keep the discussion here in this issue and let's leave issue 18 on the gym repo alone, since that one is already closed and that error was specifically to do with wxPython. If we bump into that again, we can refer to and update that issue as necessary, but for now, let's step away from it.

---

Ok, so to start with, let's get the gym environment up. As mentioned, the dependency for wxPython is outdated. In the gym-mupen64plus project, if you haven't already, please update setup.py with wxPython>=4.0.0b2 in place of the existing wx>=3.0.3 line.

You mentioned that you ran sudo pip install -e . which is what gave the last error message. Is there a reason you needed to use sudo? Can you try simply pip install -e . without sudo and let me know what happens? If you do need to run sudo, it looks like there is a warning/error about directory ownership. As it suggests, I would try sudo -H pip install -e ., which may help.

Also, just to double-check, make sure you are running that command from within the gym-mupen64plus directory. The trailing . at the end of that command is specifying to install the python package in the current directory, so just make sure you're in the right place. That may be obvious, but it is worth pointing out anyway.

---

To address a couple of your other questions from above (I'd still like to focus on one thing at a time, but don't want to ignore these):

• Is it changing something when Ubuntu is in Virtual Machine? I don't think so. I personally haven't run this in a VM, but I also haven't seen anything come up that would lead me to believe that it would make a difference. I am curious to know what the output of this command is:

  ls -la /tmp | grep X[0-9]-lock

  That will list all of the lock files for different X servers running on your machine. One reason Xvfb can fail to run is if something hasn't freed the lock. I expect at least .X0-lock to be listed for your primary X Display (your desktop environment), but if others show up too, that may be the (or a) cause of the problem.

• Does it mean that the usage of vglrun is not correct? The vglrun command looks correct to me. It matches the command that is invoked on my system. However, that is certainly what the error looks like it is saying. I'm hoping that because Xvfb is failing to start that it results in that (deceptive) error message. I have nothing to base that on, but again, one problem at a time.

[frogoscar]

@bzier

Thank you very much for your kindness :

➜  ~ ls -la /tmp | grep X[0-9]-lock
zsh: no matches found: X[0-9]-lock
➜  ~ ls -la /tmp                   
total 44
drwxrwxrwt 11 root root 4096 oct.  11 08:52 .
drwxr-xr-x 24 root root 4096 oct.   7 17:31 ..
-rw-------  1 exe  exe     0 oct.  11 08:51 config-err-nixcKm
drwxrwxrwt  2 root root 4096 oct.  11 08:49 .font-unix
drwx------  2 exe  exe  4096 oct.  11 08:52 gnome-software-N22J7Y
drwxrwxrwt  2 root root 4096 oct.  11 08:51 .ICE-unix
drwx------  3 root root 4096 oct.  11 08:51 systemd-private-2ba36dfa4a8d4abdb5ec3980a2d15b68-colord.service-b5ojp4
drwx------  3 root root 4096 oct.  11 08:51 systemd-private-2ba36dfa4a8d4abdb5ec3980a2d15b68-rtkit-daemon.service-ifZEs9
drwxrwxrwt  2 root root 4096 oct.  11 08:49 .Test-unix
drwx------  2 exe  exe  4096 oct.  11 08:51 tracker-extract-files.1000
drwxrwxrwt  2 root root 4096 oct.  11 08:51 .X11-unix
drwxrwxrwt  2 root root 4096 oct.  11 08:49 .XIM-unix

[frogoscar]

@bzier

I changed the setup.py, from

diff --git a/setup.py b/setup.py
index d68f42f..0ba931f 100644
--- a/setup.py
+++ b/setup.py
@@ -6,4 +6,4 @@ setup(name='gym_mupen64plus',
                         'numpy>=1.12.0',
                         'PyYAML>=3.12',
                         'termcolor>=1.1.0',
-                        'wx>=3.0.3'])
+                        'wxPython>=4.0.0b2'])

I am running pip install -e . in the folder gym-mupen64plus , it is installing wxPython4.0.0b2 (it takes time), we will see if it succeed or not.

Thanks a lot for your kindly help!

[frogoscar]

@bzier The result of pip install -e . is the same as before, still error when installing wxPython4.0.0b2 :

../../../../sip/cpp/sip_html2cmodule.cpp:391:4: error: ‘wxWebView’ in namespace ‘::’ does not name a type
    ::wxWebView* sipVH__html2_23(sip_gilstate_t sipGILState, sipVirtErrorHandlerFunc sipErrorHandler, sipSimpleWrapper *sipPySelf, PyObject *sipMethod,  ::wxWindow*parent, ::wxWindowID id,const  ::wxString& url,const  ::wxPoint& pos,const  ::wxSize& size,long style,const  ::wxString& name)
      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:401:4: error: ‘wxWebView’ in namespace ‘::’ does not name a type
    ::wxWebView* sipVH__html2_22(sip_gilstate_t sipGILState, sipVirtErrorHandlerFunc sipErrorHandler, sipSimpleWrapper *sipPySelf, PyObject *sipMethod)
      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:592:55: error: ‘wxWEBVIEW_FIND_BACKWARDS’ was not declared in this scope
       {sipName_WEBVIEW_FIND_BACKWARDS, static_cast<int>(wxWEBVIEW_FIND_BACKWARDS), 5},
                                                         ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:593:53: error: ‘wxWEBVIEW_FIND_DEFAULT’ was not declared in this scope
       {sipName_WEBVIEW_FIND_DEFAULT, static_cast<int>(wxWEBVIEW_FIND_DEFAULT), 5},
                                                       ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:594:57: error: ‘wxWEBVIEW_FIND_ENTIRE_WORD’ was not declared in this scope
       {sipName_WEBVIEW_FIND_ENTIRE_WORD, static_cast<int>(wxWEBVIEW_FIND_ENTIRE_WORD), 5},
                                                           ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:595:62: error: ‘wxWEBVIEW_FIND_HIGHLIGHT_RESULT’ was not declared in this scope
       {sipName_WEBVIEW_FIND_HIGHLIGHT_RESULT, static_cast<int>(wxWEBVIEW_FIND_HIGHLIGHT_RESULT), 5},
                                                                ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:596:56: error: ‘wxWEBVIEW_FIND_MATCH_CASE’ was not declared in this scope
       {sipName_WEBVIEW_FIND_MATCH_CASE, static_cast<int>(wxWEBVIEW_FIND_MATCH_CASE), 5},
                                                          ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:597:50: error: ‘wxWEBVIEW_FIND_WRAP’ was not declared in this scope
       {sipName_WEBVIEW_FIND_WRAP, static_cast<int>(wxWEBVIEW_FIND_WRAP), 5},
                                                    ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:598:53: error: ‘wxWEBVIEW_NAV_ERR_AUTH’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_AUTH, static_cast<int>(wxWEBVIEW_NAV_ERR_AUTH), 8},
                                                       ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:599:60: error: ‘wxWEBVIEW_NAV_ERR_CERTIFICATE’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_CERTIFICATE, static_cast<int>(wxWEBVIEW_NAV_ERR_CERTIFICATE), 8},
                                                              ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:600:59: error: ‘wxWEBVIEW_NAV_ERR_CONNECTION’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_CONNECTION, static_cast<int>(wxWEBVIEW_NAV_ERR_CONNECTION), 8},
                                                             ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:601:58: error: ‘wxWEBVIEW_NAV_ERR_NOT_FOUND’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_NOT_FOUND, static_cast<int>(wxWEBVIEW_NAV_ERR_NOT_FOUND), 8},
                                                            ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:602:54: error: ‘wxWEBVIEW_NAV_ERR_OTHER’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_OTHER, static_cast<int>(wxWEBVIEW_NAV_ERR_OTHER), 8},
                                                        ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:603:56: error: ‘wxWEBVIEW_NAV_ERR_REQUEST’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_REQUEST, static_cast<int>(wxWEBVIEW_NAV_ERR_REQUEST), 8},
                                                          ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:604:57: error: ‘wxWEBVIEW_NAV_ERR_SECURITY’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_SECURITY, static_cast<int>(wxWEBVIEW_NAV_ERR_SECURITY), 8},
                                                           ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:605:63: error: ‘wxWEBVIEW_NAV_ERR_USER_CANCELLED’ was not declared in this scope
       {sipName_WEBVIEW_NAV_ERR_USER_CANCELLED, static_cast<int>(wxWEBVIEW_NAV_ERR_USER_CANCELLED), 8},
                                                                 ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:606:55: error: ‘wxWEBVIEW_RELOAD_DEFAULT’ was not declared in this scope
       {sipName_WEBVIEW_RELOAD_DEFAULT, static_cast<int>(wxWEBVIEW_RELOAD_DEFAULT), 9},
                                                         ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:607:56: error: ‘wxWEBVIEW_RELOAD_NO_CACHE’ was not declared in this scope
       {sipName_WEBVIEW_RELOAD_NO_CACHE, static_cast<int>(wxWEBVIEW_RELOAD_NO_CACHE), 9},
                                                          ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:608:51: error: ‘wxWEBVIEW_ZOOM_LARGE’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_LARGE, static_cast<int>(wxWEBVIEW_ZOOM_LARGE), 10},
                                                     ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:609:53: error: ‘wxWEBVIEW_ZOOM_LARGEST’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_LARGEST, static_cast<int>(wxWEBVIEW_ZOOM_LARGEST), 10},
                                                       ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:610:52: error: ‘wxWEBVIEW_ZOOM_MEDIUM’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_MEDIUM, static_cast<int>(wxWEBVIEW_ZOOM_MEDIUM), 10},
                                                      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:611:51: error: ‘wxWEBVIEW_ZOOM_SMALL’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_SMALL, static_cast<int>(wxWEBVIEW_ZOOM_SMALL), 10},
                                                     ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:612:50: error: ‘wxWEBVIEW_ZOOM_TINY’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_TINY, static_cast<int>(wxWEBVIEW_ZOOM_TINY), 10},
                                                    ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:613:57: error: ‘wxWEBVIEW_ZOOM_TYPE_LAYOUT’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_TYPE_LAYOUT, static_cast<int>(wxWEBVIEW_ZOOM_TYPE_LAYOUT), 11},
                                                           ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:614:55: error: ‘wxWEBVIEW_ZOOM_TYPE_TEXT’ was not declared in this scope
       {sipName_WEBVIEW_ZOOM_TYPE_TEXT, static_cast<int>(wxWEBVIEW_ZOOM_TYPE_TEXT), 11},
                                                         ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:671:37: error: ‘wxWebViewBackendDefault’ was not declared in this scope
       {sipName_WebViewBackendDefault, wxWebViewBackendDefault, 'N'},
                                       ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:672:32: error: ‘wxWebViewBackendIE’ was not declared in this scope
       {sipName_WebViewBackendIE, wxWebViewBackendIE, 'N'},
                                  ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:673:36: error: ‘wxWebViewBackendWebKit’ was not declared in this scope
       {sipName_WebViewBackendWebKit, wxWebViewBackendWebKit, 'N'},
                                      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:674:36: error: ‘wxWebViewDefaultURLStr’ was not declared in this scope
       {sipName_WebViewDefaultURLStr, wxWebViewDefaultURLStr, 'N'},
                                      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:675:30: error: ‘wxWebViewNameStr’ was not declared in this scope
       {sipName_WebViewNameStr, wxWebViewNameStr, 'N'},
                                ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:682:35: error: ‘wxEVT_WEBVIEW_ERROR’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_ERROR, wxEVT_WEBVIEW_ERROR},
                                     ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:683:36: error: ‘wxEVT_WEBVIEW_LOADED’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_LOADED, wxEVT_WEBVIEW_LOADED},
                                      ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:684:39: error: ‘wxEVT_WEBVIEW_NAVIGATED’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_NAVIGATED, wxEVT_WEBVIEW_NAVIGATED},
                                         ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:685:40: error: ‘wxEVT_WEBVIEW_NAVIGATING’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_NAVIGATING, wxEVT_WEBVIEW_NAVIGATING},
                                          ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:686:39: error: ‘wxEVT_WEBVIEW_NEWWINDOW’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_NEWWINDOW, wxEVT_WEBVIEW_NEWWINDOW},
                                         ^
  ../../../../sip/cpp/sip_html2cmodule.cpp:687:43: error: ‘wxEVT_WEBVIEW_TITLE_CHANGED’ was not declared in this scope
       {sipName_wxEVT_WEBVIEW_TITLE_CHANGED, wxEVT_WEBVIEW_TITLE_CHANGED},
                                             ^
  Waf: Leaving directory `/tmp/pip-build-JY1hpo/wxPython/build/waf/2.7/gtk3'
  Build failed
   -> task in '_html2' failed (exit status 1):
    {task 140008285381776: cxx sip_html2cmodule.cpp -> sip_html2cmodule.cpp.17.o}
  ['/usr/bin/g++', '-fPIC', '-pthread', '-pthread', '-UNDEBUG', '-g', '-pthread', '-fno-strict-aliasing', '-fwrapv', '-fstack-protector-strong', '-fno-strict-aliasing', '-fwrapv', '-fstack-protector-strong', '-fno-strict-aliasing', '-I/tmp/pip-build-JY1hpo/wxPython/build/wxbld/gtk3/lib/wx/include/gtk3-unicode-3.0', '-I/tmp/pip-build-JY1hpo/wxPython/ext/wxWidgets/include', '-I/tmp/pip-build-JY1hpo/wxPython/build/waf/2.7/gtk3/sip/siplib', '-I/tmp/pip-build-JY1hpo/wxPython/sip/siplib', '-I/tmp/pip-build-JY1hpo/wxPython/build/waf/2.7/gtk3/src', '-I/tmp/pip-build-JY1hpo/wxPython/src', '-I/usr/include/python2.7', '-I/usr/include/x86_64-linux-gnu/python2.7', '-DPYTHONDIR="/usr/local/lib/python2.7/dist-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python2.7/dist-packages"', '-DHAVE_PYTHON_H=1', '-DHAVE_WX=1', '-DHAVE_WXADV=1', '-DHAVE_WXSTC=1', '-DHAVE_WXHTML=1', '-DHAVE_WXGL=1', '-DHAVE_WXWEBVIEW=1', '-DHAVE_WXXML=1', '-DHAVE_WXXRC=1', '-DHAVE_WXRICHTEXT=1', '-DHAVE_WXMEDIA=1', '-DHAVE_WXRIBBON=1', '-DHAVE_WXPROPGRID=1', '-DHAVE_WXAUI=1', '-D_FILE_OFFSET_BITS=64', '-DWXUSINGDLL', '-D__WXGTK__', '-D_FORTIFY_SOURCE=2', '-DNDEBUG', '-D_FORTIFY_SOURCE=2', '../../../../sip/cpp/sip_html2cmodule.cpp', '-c', '-o', 'sip/cpp/sip_html2cmodule.cpp.17.o']
  Command '"/usr/bin/python" /tmp/pip-build-JY1hpo/wxPython/bin/waf-1.7.15-p1 --wx_config=/tmp/pip-build-JY1hpo/wxPython/build/wxbld/gtk3/wx-config --gtk3 --python="/usr/bin/python" --out=build/waf/2.7/gtk3 configure build ' failed with exit code 1.

Thanks a lot!

Don't know why wxPyhon is so hard to install.

[bzier]

Take a look here. It says this error may be the result of missing the libwebkitgtk-dev library. As I mentioned here, you may need to install some additional dependencies. You said you tried that, but do double-check that this one got installed.

What is the output if you run this:

sudo apt-get install libwebkitgtk-dev

[frogoscar]

@bzier

But I have mentioned in that post that I have done this :

➜  ~ sudo apt-get install libwebkitgtk-dev
[sudo] password for exe: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libwebkitgtk-dev is already the newest version (2.4.11-0ubuntu0.1).
0 upgraded, 0 newly installed, 0 to remove and 144 not upgraded.

But if I remove the wx>=3.0.3 in setup.py of your gym-mupen64plus, and then run sudo pip install -e . (pip install -e .does not work because it needs writing to installation folder), it shows that gym-mupen64plus 0.0.1 is installed successfully :

➜  gym-mupen64plus git:(master) ✗ sudo pip -v install -e .
[sudo] password for exe: 
The directory '/home/exe/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/exe/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Obtaining file:///home/exe/gym-mupen64plus
  Running setup.py (path:/home/exe/gym-mupen64plus/setup.py) egg_info for package from file:///home/exe/gym-mupen64plus
    Running command python setup.py egg_info
    running egg_info
    writing requirements to gym_mupen64plus.egg-info/requires.txt
    writing gym_mupen64plus.egg-info/PKG-INFO
    writing top-level names to gym_mupen64plus.egg-info/top_level.txt
    writing dependency_links to gym_mupen64plus.egg-info/dependency_links.txt
    reading manifest file 'gym_mupen64plus.egg-info/SOURCES.txt'
    writing manifest file 'gym_mupen64plus.egg-info/SOURCES.txt'
  Source in /home/exe/gym-mupen64plus has version 0.0.1, which satisfies requirement gym-mupen64plus==0.0.1 from file:///home/exe/gym-mupen64plus
Requirement already satisfied: gym>=0.2.3 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: numpy>=1.12.0 in /home/exe/.local/lib/python2.7/site-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: PyYAML>=3.12 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python2.7/dist-packages (from gym-mupen64plus==0.0.1)
Requirement already satisfied: requests>=2.0 in /usr/lib/python2.7/dist-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Requirement already satisfied: six in /home/exe/.local/lib/python2.7/site-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Requirement already satisfied: pyglet>=1.2.0 in /usr/local/lib/python2.7/dist-packages (from gym>=0.2.3->gym-mupen64plus==0.0.1)
Installing collected packages: gym-mupen64plus
  Found existing installation: gym-mupen64plus 0.0.1
    Not sure how to uninstall: gym-mupen64plus 0.0.1 - Check: /home/exe/gym-mupen64plus
    Can't uninstall 'gym-mupen64plus'. No files were found to uninstall.
  Running setup.py develop for gym-mupen64plus
    Running command /usr/bin/python -c "import setuptools, tokenize;__file__='/home/exe/gym-mupen64plus/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps
    running develop
    running egg_info
    writing requirements to gym_mupen64plus.egg-info/requires.txt
    writing gym_mupen64plus.egg-info/PKG-INFO
    writing top-level names to gym_mupen64plus.egg-info/top_level.txt
    writing dependency_links to gym_mupen64plus.egg-info/dependency_links.txt
    reading manifest file 'gym_mupen64plus.egg-info/SOURCES.txt'
    writing manifest file 'gym_mupen64plus.egg-info/SOURCES.txt'
    running build_ext
    Creating /usr/local/lib/python2.7/dist-packages/gym-mupen64plus.egg-link (link to .)
    gym-mupen64plus 0.0.1 is already the active version in easy-install.pth

    Installed /home/exe/gym-mupen64plus
Successfully installed gym-mupen64plus
Cleaning up...

[frogoscar]

@bzier

Today TensorKart starts successfully (amazing) with python play.py

And it seems that the model_weights.h5 file is giving the input to MarioKart (Mario-Kart-Luigi-Raceway-v0) , but it failed after several input :

➜  TensorKart git:(master) ✗ python play.py
Using TensorFlow backend.
[2017-10-12 07:37:31,092] Making new env: Mario-Kart-Luigi-Raceway-v0
('ControllerHTTPServer started on port ', 8082)
initial_disp: :0
Starting xvfb with command: ['Xvfb', ':1', '-screen', '0', '640x480x24', '-fbdir', '/dev/shm']
Changed DISPLAY to: :1
Starting emulator with comand: ['vglrun', 'mupen64plus', '--resolution', '640x480', '--input', '/usr/local/lib/mupen64plus/mupen64plus-input-bot.so', '/home/exe/gym-mupen64plus/gym_mupen64plus/ROMs/marioKart.n64']
Calling wx.App() with DISPLAY: :1
 __  __                         __   _  _   ____  _             
|  \/  |_   _ _ __   ___ _ __  / /_ | || | |  _ \| |_   _ ___ 
| |\/| | | | | '_ \ / _ \ '_ \| '_ \| || |_| |_) | | | | / __|  
| |  | | |_| | |_) |  __/ | | | (_) |__   _|  __/| | |_| \__ \  
|_|  |_|\__,_| .__/ \___|_| |_|\___/   |_| |_|   |_|\__,_|___/  
             |_|         http://code.google.com/p/mupen64plus/  
Mupen64Plus Console User-Interface Version 2.5.0

UI-Console: attached to core library 'Mupen64Plus Core' version 2.5.0
UI-Console:             Includes support for Dynamic Recompiler.
UI-Console:             Includes support for MIPS r4300 Debugger.
Core: Goodname: Mario Kart 64 (E) (V1.1) [!]
Core: Name: MARIOKART64         
Core: MD5: 2BB149A583FDEFEA96805F628FE42FD9
Core: CRC: 2577C7D4 D18FAAAE
Core: Imagetype: .z64 (native)
Core: Rom size: 12582912 bytes (or 12 Mb or 96 Megabits)
Core: Version: 1446
Core: Manufacturer: Nintendo
Core: Country: Unknown (0x150)
UI-Console Status: Cheat codes disabled.
UI-Console: using Video plugin: 'Mupen64Plus OpenGL Video Plugin by Rice' v2.5.0
UI-Console: using Audio plugin: 'Mupen64Plus SDL Audio Plugin' v2.5.0
UI-Console: using Input plugin: 'Mupen64Plus Bot Input Plugin' v0.0.1
UI-Console: using RSP plugin: 'Hacktarux/Azimer High-Level Emulation RSP Plugin' v2.5.0
Input: Mupen64Plus Bot Input Plugin version 0.0.1 initialized.
Video: SSE processing enabled.
Video: ROM (CRC d4c77725aeaa8fd1-50) not found in INI file
Video: Enabled hacks for game: 'MARIOKART64'
Video: Initializing OpenGL Device Context.
Core: Setting 32-bit video mode: 640x480
Video Warning: Failed to set GL_SWAP_CONTROL to 0. (it's 24)
Video Warning: Failed to set GL_BUFFER_SIZE to 32. (it's 24)
Video Warning: Failed to set GL_DEPTH_SIZE to 16. (it's 32)
Video: Using OpenGL: VMware, Inc. - Gallium 0.4 on llvmpipe (LLVM 4.0, 256 bits) : 3.0 Mesa 17.0.7
Audio: Initializing SDL audio subsystem...
Core: Starting R4300 emulator: Dynamic Recompiler
Core: R4300: starting 64-bit dynamic recompiler at: 0x7efe3acea6e0
Changed DISPLAY to: :0
('Frame ', '10', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '80', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '120', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '125', ': ', '[0, -80, 0, 0, 0]')
('Frame ', '130', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '132', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '134', ': ', '[0, 0, 1, 0, 0]')
('Player row: ', '0')
('Player col: ', '0')
('Frame ', '160', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '162', ': ', '[0, 0, 1, 0, 0]')
('Map series: ', '0')
('Map choice: ', '0')
('Frame ', '202', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '230', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '232', ': ', '[0, 0, 1, 0, 0]')
Reset called!
Traceback (most recent call last):
  File "play.py", line 62, in <module>
    obs = env.reset()
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 104, in reset
    return self._reset()
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 49, in _reset
    return self.env.reset()
  File "/usr/local/lib/python2.7/dist-packages/gym/core.py", line 104, in reset
    return self._reset()
  File "/home/exe/gym-mupen64plus/gym_mupen64plus/envs/MarioKart64/mario_kart_env.py", line 52, in _reset
    return super(MarioKartEnv, self)._reset()
  File "/home/exe/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 128, in _reset
    return self._observe()
  File "/home/exe/gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py", line 95, in _observe
    bmp = wx.Bitmap(config['SCR_W'], config['SCR_H'])
  File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_gdi.py", line 639, in __init__
    _gdi_.Bitmap_swiginit(self,_gdi_.new_Bitmap(*args, **kwargs))
TypeError: String or Unicode type required
Close called!
Sending SHUTDOWN response

(play.py:3515): GLib-GObject-CRITICAL **: g_type_instance_get_private: assertion 'instance != NULL && instance->g_class != NULL' failed

(play.py:3515): Gdk-CRITICAL **: IA__gdk_gc_set_subwindow: assertion 'GDK_IS_GC (gc)' failed

(play.py:3515): GLib-GObject-CRITICAL **: g_type_instance_get_private: assertion 'instance != NULL && instance->g_class != NULL' failed

(play.py:3515): Gdk-CRITICAL **: IA__gdk_gc_set_subwindow: assertion 'GDK_IS_GC (gc)' failed

(play.py:3515): GLib-GObject-CRITICAL **: g_type_instance_get_private: assertion 'instance != NULL && instance->g_class != NULL' failed

(play.py:3515): Gdk-CRITICAL **: IA__gdk_gc_set_subwindow: assertion 'GDK_IS_GC (gc)' failed

(play.py:3515): GLib-GObject-CRITICAL **: g_type_instance_get_private: assertion 'instance != NULL && instance->g_class != NULL' failed

(play.py:3515): Gdk-CRITICAL **: IA__gdk_gc_set_subwindow: assertion 'GDK_IS_GC (gc)' failed
../src/gtk/dcclient.cpp(250): assert "Assert failure" failed in wxFreePoolGC(): Wrong GC
../src/gtk/dcclient.cpp(250): assert "Assert failure" failed in wxFreePoolGC(): Wrong GC
../src/gtk/dcclient.cpp(250): assert "Assert failure" failed in wxFreePoolGC(): Wrong GC
../src/gtk/dcclient.cpp(250): assert "Assert failure" failed in wxFreePoolGC(): Wrong GC

There is a "TypeError: String or Unicode type required".

My wxPython is of version 3.0.2 since 4.0.0b2 can not be installed :

Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>> wx.__version__
'3.0.2.0'

Thank you very much!

[bzier]

Yeah, I knew you had installed those dependencies, but just thought it was best to confirm it.

Unfortunately those first few frames are just the environment navigating through the menu. When it calls reset() is the first attempt to get a screenshot, which uses wx to do it. Since that still isn't installed correctly, it isn't going to move beyond that. So it's disappointing, but those first few inputs aren't actually the AI.

At this point, I may have to defer you to the wxPython project. I'm really not sure why it won't build for you if you have the necessary dependencies. In my experience, when it has failed to build, installing those had been the key. I hate to leave you hanging, but I'm not sure what else to suggest now.

[bzier]

Take a look at #11. It describes the same error with wx 3 that you're seeing now. I would still recommend pursuing fixing the wxPython 4 build, but in the meantime the suggested fix in #11 might get things moving. You'll probably have to do that (or something similar) in the _observe () method of gym-mupen64plus/gym_mupen64plus/envs/mupen64plus_env.py

[frogoscar]

@bzier

Thank you very much.

I will continue to try.

PS : I have done that fix from #11 for record.py

[frogoscar]

@bzier @kevinhughes27

Hello, I have succeed to launch the game with python play.py (after I successfully installed wxPython4.0.0a1), yeah!

But the Mario does not move (See : Screenshot), and the console keeps printing :

Changed DISPLAY to: :0
Emulator closed with code: 0
('Frame ', '10', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '80', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '120', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '125', ': ', '[0, -80, 0, 0, 0]')
('Frame ', '130', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '132', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '134', ': ', '[0, 0, 1, 0, 0]')
('Player row: ', '0')
('Player col: ', '0')
('Frame ', '160', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '162', ': ', '[0, 0, 1, 0, 0]')
('Map series: ', '0')
('Map choice: ', '0')
('Frame ', '202', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '230', ': ', '[0, 0, 1, 0, 0]')
('Frame ', '232', ': ', '[0, 0, 1, 0, 0]')
Reset called!
env ready!
2017-10-13 16:48:51.963713: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-13 16:48:51.963736: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-13 16:48:51.963750: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-10-13 16:48:51.963755: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-13 16:48:51.963759: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-10-13 16:48:51.972514: E tensorflow/stream_executor/cuda/cuda_driver.cc:406] failed call to cuInit: CUDA_ERROR_UNKNOWN
2017-10-13 16:48:51.972544: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:145] kernel driver does not appear to be running on this host (exe-ercom): /proc/driver/nvidia/version does not exist
actor ready!
beginning episode loop
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/exe/TensorKart/utils.py", line 84, in _monitor_controller
    events = get_gamepad()
  File "/usr/local/lib/python2.7/dist-packages/inputs.py", line 2712, in get_gamepad
    raise UnpluggedError("No gamepad found.")
UnpluggedError: No gamepad found.

AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
AI: [0, 0, 0, 0, 0]
...

When I use python record.py to record when I play the Mario-Kart-Luigi-Raceway-v0, I used my Keyboard (Shift + Up to keep moving. Left and Right to turn left and right), is it right ?

I think the problem is that I did not use real JoyStick but the Keyboard, so when I record the game playing, it is always [0,0,0,0,0] in the samples/xxx/data.csv file :

samples/2017-10-13_17:15:24/img_0.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_1.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_2.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_3.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_4.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_5.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_6.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_7.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_8.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_9.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_10.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_11.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_12.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_13.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_14.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_15.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_16.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_17.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_18.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_19.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_20.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_21.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_22.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_23.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_24.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_25.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_26.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_27.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_28.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_29.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_30.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_31.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_32.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_33.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_34.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_35.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_36.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_37.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_38.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_39.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_40.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_41.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_42.png,0,0,0,0,0
samples/2017-10-13_17:15:24/img_43.png,0,0,0,0,0

See : Screenshot

But I do not have JoyStick at home, how to do ?

Thanks a lot!

[bzier]

That's great to hear that you got wxPython working! Out of curiosity, and for the sake of others who come along later, what was the trick/solution?

As far as the inputs go, you are correct in your assessment as far as why the AI is not acting. It was trained on data that indicated it should always leave the controller untouched. Maybe @kevinhughes27 can chime in here with any suggestions too, since he wrote record.py and utils.py, but my thought is that you would need to modify the XboxController class in utils.py to support reading from the keyboard. Or better yet, create a controller abstraction with XboxController and KeyboardController as concrete implementations. Essentially what it is doing is transforming the gamepad input into a representation of the N64 controller anyway. It just needs some mapping from an arbitrary input source to the N64 controller output.

After that, obviously you'd need to record new races in order to pick up the controller data and retrain your model with good data.

[frogoscar]

@bzier

I am also able to install wxPython4.0.0b2 on my personal computer.

done
  Removing source in /tmp/pip-build-N9dzDd/wxPython
Successfully installed wxPython-4.0.0b2
Cleaning up...
➜  gtk-build python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import wx
>>> wx.__version__
'4.0.0b2'

I think it is because :

• Firstly I installed wxWidgets3.0.3 (install from building from source code)
• Then I remove all the stuff of wxPython3.0.2
• Finally I install wxPython4.0.0b2

I will detail this later.

[frogoscar]

@bzier

Yes, I think you are right. I thought about completing a little the code by modifying the XboxController class or write another class who receives from Keyboard rather than Gamepad.

Hope @kevinhughes27 the author has some ideas about that. :)

If it is done, it is cool for those who have no Gamepad.

[kevinhughes27]

a KeyboardController controller class sounds like a good idea to me. There is also a data set viewer util you can use to make sure the data you've recorded looks good. I used this quite a bit when I trained my model.

[bzier]

@frogoscar, it seems you're past the issues with the X Display referenced in the title and first couple posts of this issue. Are you ok if we close the issue now? If you encounter further issues with the keyboard input or other problems along the way, you could open a new issue... or if this issue comes back, just comment or re-open it.

---

One thing that I discovered that I wanted it mention is that I had given an invalid command earlier. For anyone else who comes along later, I wanted to make sure it was correct. Grep is case-sensitive by default, so the command to look for a running Xvfb process should have been:

Correct case for the process name: ps | grep Xvfb

or tell grep to ignore case: ps | grep -i xvfb

I am going to edit the previous post as well to reduce confusion.

[frogoscar]

@bzier Yes, I think we could close this issue. Thanks a lot for your help.

[bzier]

Not a problem, and glad you were able to get it working.