OSX installation

[nrocketmann]

I am using a mac: OSX version 10.13.2 and I'm trying to install gymfc. I followed the install instructions, and brew installed Gazebo and Ogre. I am doing this in an anaconda3 environment with python3.5.

When I entered the command: sudo python3 -m gymfc.controllers.iris_pid_eval --env-id=AttFC_GyroErr-MotorVel_M4_Ep-v0

I got this error:

Exception occured connecting to Gazebo retrying .... [Errno 61] Connect call failed ('127.0.0.1', 11345)
Gazebo multi-robot simulator, version 8.6.0
Copyright (C) 2012 Open Source Robotics Foundation.
Released under the Apache 2 License.
http://gazebosim.org

[Msg] Waiting for master.
[Msg] Connected to gazebo master @ http://127.0.0.1:11345
[Msg] Publicized address: 10.113.38.81
[Err] [RenderEngine.cc:663] EXCEPTION: unable to find OpenGL rendering system. OGRE is probably installed incorrectly. Double check the OGRE cmake output, and make sure OpenGL is enabled.

[Err] [RenderingIface.cc:38] Failed to load the Rendering engine subsystem
unable to find OpenGL rendering system. OGRE is probably installed incorrectly. Double check the OGRE cmake output, and make sure OpenGL is enabled.
[Err] [gazebo.cc:87] Unable to load sensors
[Err] [Server.cc:298] Unable to load gazebo
libc++abi.dylib: Pure virtual function called!
Exception occured connecting to Gazebo retrying .... [Errno 61] Connect call failed ('127.0.0.1', 11345)
Exception occured connecting to Gazebo retrying .... [Errno 61] Connect call failed ('127.0.0.1', 11345)
Exception occured connecting to Gazebo retrying .... [Errno 61] Connect call failed ('127.0.0.1', 11345)
Exception occured connecting to Gazebo retrying .... [Errno 61] Connect call failed ('127.0.0.1', 11345)
Traceback (most recent call last):
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/gazebo_env.py", line 287, in _connect
    self.manager = await pygazebo.connect((self.host, self.gz_port)) 
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 381, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/tasks.py", line 310, in _wakeup
    future.result()
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/pygazebo/pygazebo.py", line 562, in handle_connect
    future.result()
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/pygazebo/pygazebo.py", line 252, in callback_impl
    future.result()  # check for exception
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/tasks.py", line 242, in _step
    result = coro.throw(exc)
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/selector_events.py", line 451, in sock_connect
    return (yield from fut)
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 381, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/tasks.py", line 310, in _wakeup
    future.result()
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/selector_events.py", line 481, in _sock_connect_cb
    raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionRefusedError: [Errno 61] Connect call failed ('127.0.0.1', 11345)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/controllers/iris_pid_eval.py", line 180, in <module>
    main(args.env_id, args.seed)
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/controllers/iris_pid_eval.py", line 163, in main
    env = gym.make(env_id)
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/site-packages/gym/envs/registration.py", line 167, in make
    return registry.make(id)
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/site-packages/gym/envs/registration.py", line 119, in make
    env = spec.make()
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/site-packages/gym/envs/registration.py", line 86, in make
    env = cls(**self._kwargs)
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/attitude_control_env.py", line 67, in __init__
    super(GyroErrorESCVelocityFeedbackEnv, self).__init__(motor_count = motor_count, world=world)
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/gazebo_env.py", line 145, in __init__
    self._start_sim()
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/gazebo_env.py", line 240, in _start_sim
    self.loop.run_until_complete(self._connect())
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/base_events.py", line 467, in run_until_complete
    return future.result()
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/futures.py", line 294, in result
    raise self._exception
  File "/Users/nameerhirschkind/anaconda3/envs/machine-learning/lib/python3.5/asyncio/tasks.py", line 240, in _step
    result = coro.send(None)
  File "/Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/gazebo_env.py", line 293, in _connect
    if not self.manager:
AttributeError: 'GyroErrorESCVelocityFeedbackEnv' object has no attribute 'manager'

Any idea on how to fix this? Thanks in advance!

[wil3]

Hi @nrocketmann the error you are seeing is from Gazebo, it appears you do not have the proper dependencies. You can confirm this independent of GymFC from the command line just start Gazebo by typing, gazebo. The only OS I have tested it on is Ubuntu so I dont know how to solve your specific OS problem. My advice would be to try and see how to get Gazebo running on OSX, possibly report an issue to that project is you believe it is a bug. If this is to much of a hassle install Ubuntu in a VM like VirtualBox and run it from in there. I have also heard of success with using Gazebo in Docker. If you can get Gazebo running and you come across a bug in GymFC please let me know. Thanks!

[nrocketmann]

Thanks wil3. The strange thing is that gazebo ran fine when I ran it from terminal. It may be that I have multiple versions of it or something. I'll run it in virtualbox then with an ubuntu instance. Thanks!

[wil3]

Hi @nrocketmann, did you follow these install instructions? http://gazebosim.org/tutorials?tut=install_on_mac they make it seem like it should be pretty straight forward. If you follow the curl link it redirects to the bitbucket repo, you wouldnt want to use that script though because it is for Gazebo 9, looking through the commit history this one is for Gazebo 8, https://bitbucket.org/osrf/release-tools/raw/49a91748d2ce8521b34178609de25cbd089442c2/one-line-installations/gazebo.sh so you could do,

curl -ssL https://bitbucket.org/osrf/release-tools/raw/49a91748d2ce8521b34178609de25cbd089442c2/one-line-installations/gazebo.sh | sh

There would be overhead running in a VM so if possible it would be best to run on the host. Maybe try starting from scratch uninstalling all previous versions? However if you are planning to use for research doing RL you'll want a server, training can take days (eg DDPG).

[nrocketmann]

Hi @wil3, Thanks for the tips and advice. I used gazebo8 the first time, which I thought might have been the problem. I tried what you said, and it does indeed look really simple, but I get the same error now, and when I tried gazebo --version It says I'm running 8.6. Also, when I type gazebo, it works fine and launches the editor. Thanks in advance! Nameer

[wil3]

Hi @nrocketmann, You know what, this is my mistake I do think this is an issue with GymFC sorry about that. I forgot it is setting global variables to load Gazebo here, https://github.com/wil3/gymfc/blob/master/gymfc/envs/gazebo_env.py#L218-L223 so its really only compatible with Ubuntu at the moment. GymFC must source the setup file to set global variables, Its currently setting the variables in this file so if your paths don't match it wont work. This is also related to issue #1.

Looking closer at Gazebo, it seem to generate setup.sh at install and I don't even know what its doing in Windows (maybe an equivalent bat file). Can you locate your equivalent setup.sh file and let me know where it is located? On Ubuntu it is at /usr/share/gazebo-8/setup.sh. I need to work on the best fix for this to be cross-platform. In the meantime to get it working you can use the file paths in your setup.sh for the paths in gazebo_env.py to see if it will work.

[nrocketmann]

Thanks for the advice! I changed the paths and that does seem to have fixed the previous error, however, I'm now encountering another one:

[Err] [Plugin.hh:180] Failed to load plugin libQuadcopterWorldPlugin.dylib: dlopen(libQuadcopterWorldPlugin.dylib, 9): image not found

I tried copying the gymfc assets and plugins into the appropriate gazebo directories, which helped fix an ealier FileNotFoundError but it still seems unable to find the libQuadcopterWorldPlugin.dylib file. I saw in the build directory in gymfc plugins a file with the same name with a .so extension instead of .dylib. Are these related? If so, how could I fix the issue?

Thanks! Nameer

[nrocketmann]

Alright, I figured out what the problem is. Gazebo for mac wants .dylib files for plugins, which is a rough equivalent to the .so file on linux (correct me if I'm wrong here, I don't know much about this). But the libQuadcopterWorldPlugin file in gymfc is in the .so format. I renamed it to a .dylib file, so gazebo could actually find it, but naturally, it couldn't read the file: /usr/local/Cellar/gazebo8/8.6.0/lib/gazebo-8/plugins/build/libQuadcopterWorldPlugin.dylib: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x03

Any ideas on how to convert the .so to a .dylib and have it do the same thing? Thanks!

[wil3]

Great to hear that error went away. What startup script did you get the correct paths on OSX from? I want to get this fixed.

However don't move anything out of GymFC. The export variables are to tell Gazebo to look for the assets in GymFC before starting. If you getting a filenotfounderror there's something wrong, post the full error you are getting so we can get that fixed.

Yep that error is because the plugin has been compiled for Ubuntu. You'll need to compile it for Mac. Gazebo uses cmake to build the plugins, the instructions for Ubuntu are here, https://github.com/wil3/gymfc/tree/master/gymfc/envs/assets/gazebo/plugins just delete the build directory first. I'm using cmake 3.5.2. Ill see if its possible to do this compilation during the install of the GymFC module in the future.

Hope that helps. Let me know how it works out.

[nrocketmann]

I used the setup.sh from gazebo-8, which was located in: /usr/local/share/gazebo-8 and just replaced all the paths in gazebo_env.py with static ones from the setup file.

As far as the file not found error, I'm not really sure what caused it; it very well may have been something I did with the paths. I kept all of the gymfc files intact, just copied some of them to gazebo to make sure I don't get any path or filenotfound errors.

As far as the .dylib thing, that's what I figured. I'll install cmake and see if I can get it to work with that.

[nrocketmann]

I got the latest cmake (3.12) and tried to compile with the given instructions. I first had to get zmq using these: https://github.com/JustinTulloss/zeromq.node/issues/283 instructions and install uuid, which was easy. Now, I'm getting these other errors that I don't really understand:

[ 50%] Building CXX object CMakeFiles/QuadcopterWorldPlugin.dir/QuadcopterWorldPlugin.cpp.o
In file included from /users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.cpp:49:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/sensors/sensors.hh:27:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/sensors/WirelessTransmitter.hh:22:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/physics/physics.hh:2:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/physics/Actor.hh:26:
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:159:40: error: 
      reference to 'Spline' is ambiguous
      private: mutable ignition::math::Spline *positionSpline;
                                       ^
/usr/local/include/ignition/math4/ignition/math/Spline.hh:38:33: note: 
      candidate found by name lookup is 'ignition::math::v4::Spline'
    class IGNITION_MATH_VISIBLE Spline
                                ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:28:11: note: 
      candidate found by name lookup is 'ignition::math::Spline'
    class Spline;
          ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:162:40: error: 
      reference to 'RotationSpline' is ambiguous
      private: mutable ignition::math::RotationSpline *rotationSpline;
                                       ^
/usr/local/include/ignition/math4/ignition/math/RotationSpline.hh:34:34: note: 
      candidate found by name lookup is 'ignition::math::v4::RotationSpline'
    class IGNITION_MATH_VISIBLE  RotationSpline
                                 ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:29:11: note: 
      candidate found by name lookup is 'ignition::math::RotationSpline'
    class RotationSpline;
          ^
/users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.cpp:332:29: error: 
      no matching conversion for functional-style cast from 'double' to
      'boost::posix_time::milliseconds' (aka
      'subsecond_duration<boost::posix_time::time_duration, 1000>')
  ...boost::this_thread::sleep(boost::posix_time::milliseconds(msPeriod));
                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/include/boost/date_time/time_duration.hpp:270:30: note: candidate
      constructor (the implicit copy constructor) not viable: no known
      conversion from 'double' to 'const
      boost::date_time::subsecond_duration<boost::posix_time::time_duration,
      1000>' for 1st argument
  class BOOST_SYMBOL_VISIBLE subsecond_duration : public base_duration
                             ^
/usr/local/include/boost/date_time/time_duration.hpp:270:30: note: candidate
      constructor (the implicit move constructor) not viable: no known
      conversion from 'double' to
      'boost::date_time::subsecond_duration<boost::posix_time::time_duration,
      1000>' for 1st argument
/usr/local/include/boost/date_time/time_duration.hpp:286:59: note: candidate
      template ignored: disabled by 'enable_if' [with T = double]
                                typename boost::enable_if<boost::is_inte...
                                                          ^
3 errors generated.
make[2]: *** [CMakeFiles/QuadcopterWorldPlugin.dir/QuadcopterWorldPlugin.cpp.o] Error 1
make[1]: *** [CMakeFiles/QuadcopterWorldPlugin.dir/all] Error 2
make: *** [all] Error 2

[wil3]

Looks like its an issue with Boost, what version are you using?

[nrocketmann]

Hi, sorry it took so long to get back. I was getting the zeromq error again and this time fixed it this way: https://github.com/caskformula/homebrew-caskformula/issues/10 . Now I can reproduce the error I mentioned above. It looks like I have boost version 1.67.0. I installed it both via brew and macports. Brew also says boost is linked. Should I install a different version of it?

[wil3]

1.58, not sure why your install is using a newer version. The sleep API changed in newer versions, see if rolling back to 1.58 fixes it.

[nrocketmann]

I tried installing boost 1.58 with macports, and 1.57 with brew (brew doesn't have 1.58), and still got the same error. Maybe cmake can't find boost? I'm not really sure. I also don't know what paths cmake uses, so I can't change those.

[wil3]

Hey, sorry I dont have the knowledge of OSX to know details on how packages are installed but that error is due to a conflicting API versions. My suggestion would be to look into your Gazebo installation and what boost version it is using. When building the plugin with cmake it is referencing the Gazebo libraries so whatever ones its using in your install that's what the plugin will use. If you are making changes made sure to delete your build directory and start over with cmake to make sure nothing is cached.

[nrocketmann]

Yeah, I have been clearing the build directory each time. Alright, now the boost part of the error is gone. I think brew installing boost did work, I just didn't notice that the error had changed slightly:

Scanning dependencies of target QuadcopterWorldPlugin
[ 50%] Building CXX object CMakeFiles/QuadcopterWorldPlugin.dir/QuadcopterWorldPlugin.cpp.o
In file included from /Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.cpp:49:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/sensors/sensors.hh:27:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/sensors/WirelessTransmitter.hh:22:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/physics/physics.hh:2:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/physics/Actor.hh:26:
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:159:40: error: 
      reference to 'Spline' is ambiguous
      private: mutable ignition::math::Spline *positionSpline;
                                       ^
/usr/local/include/ignition/math4/ignition/math/Spline.hh:38:33: note: 
      candidate found by name lookup is 'ignition::math::v4::Spline'
    class IGNITION_MATH_VISIBLE Spline
                                ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:28:11: note: 
      candidate found by name lookup is 'ignition::math::Spline'
    class Spline;
          ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:162:40: error: 
      reference to 'RotationSpline' is ambiguous
      private: mutable ignition::math::RotationSpline *rotationSpline;
                                       ^
/usr/local/include/ignition/math4/ignition/math/RotationSpline.hh:34:34: note: 
      candidate found by name lookup is 'ignition::math::v4::RotationSpline'
    class IGNITION_MATH_VISIBLE  RotationSpline
                                 ^
/usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/Animation.hh:29:11: note: 
      candidate found by name lookup is 'ignition::math::RotationSpline'
    class RotationSpline;
          ^
In file included from /Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.cpp:53:
In file included from /Users/nameerhirschkind/PycharmProjects/drone/gymfc/gymfc/envs/assets/gazebo/plugins/QuadcopterWorldPlugin.hh:21:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/common.hh:12:
In file included from /usr/local/Cellar/gazebo8/8.6.0/include/gazebo-8/gazebo/common/CommonIface.hh:22:
/opt/local/include/boost/uuid/sha1.hpp:13:9: warning: This header is
      implementation detail and provided for backwards compatibility.
      [-W#pragma-messages]
#pragma message("This header is implementation detail and provided for b...
        ^
1 warning and 2 errors generated.
make[2]: *** [CMakeFiles/QuadcopterWorldPlugin.dir/QuadcopterWorldPlugin.cpp.o] Error 1
make[1]: *** [CMakeFiles/QuadcopterWorldPlugin.dir/all] Error 2
make: *** [all] Error 2

It looks to me like a problem with the ignition math library: https://ignitionrobotics.org/libs/math Any ideas? Thanks!

[wil3]

Could be another version conflict, compare the library versions to those installed on on Gazebo 8 on Ubuntu to what you have installed on your system. What have you tried so far?

[weixingsun]

Hello, My install completed but when I run controller, I have error : missing mpi4py sun$ python3 -m gymfc.controllers.iris_pid_eval --env-id=AttFC_GyroErr-MotorVel_M4_Ep-v0 Traceback (most recent call last): File "/anaconda3/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/anaconda3/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/sun/Desktop/OpenAI/gymfc/gymfc/controllers/iris_pid_eval.py", line 7, in from mpi4py import MPI ModuleNotFoundError: No module named 'mpi4py'

[wil3]

Hi @weixingsun when you do pip3 freeze | grep mpi4py what is the output?

[wil3]

Sorry this issue should have been closed, I don't have any plans to support OSX. Unless your Gazebo paths match the startup in https://github.com/wil3/gymfc/blob/master/gymfc/envs/gazebo_env.py#L242 it isn't going to work. Version 2 will tie into Gazebo without hardcoded paths making it more flexible but I don't know if its enough. If you do get it working submitting a pull request for any documentation and changes would be great.

[ssdasgupta]

@nrocketmann , did you solve the problem finally?

[nrocketmann]

@ssdasgupta, I'm afraid I didn't. I haven't looked at this in about a year now. Best of luck if you're going down the same path.