DanielTakeshi
commented
3 years ago Attempt 2: Follow-up on Ubuntu 16.04 Machine and Compiling Docker
I tried to reproduce all the instructions above, on an Ubuntu 16.04 machine instead of an Ubuntu 18.04 machine. I followed all the steps the same way as above, however this time there is a slightly different error. After compiling PyFlex and exiting Docker, I activate the softgym env and need to again refresh the environment variables (please let me know if this is not what you do):
seita@triton1:~/softgym (master) $ conda activate softgym
(softgym) seita@triton1:~/softgym (master) $ python examples/random_env.py --env_name ClothFlatten
Traceback (most recent call last):
File "examples/random_env.py", line 5, in <module>
from softgym.registered_env import env_arg_dict, SOFTGYM_ENVS
ModuleNotFoundError: No module named 'softgym'
(softgym) seita@triton1:~/softgym (master) $ export PYFLEXROOT=${PWD}/PyFlex
(softgym) seita@triton1:~/softgym (master) $ export PYTHONPATH=${PYFLEXROOT}/bindings/build:$PYTHONPATH
(softgym) seita@triton1:~/softgym (master) $ export LD_LIBRARY_PATH=${PYFLEXROOT}/external/SDL2-2.0.4/lib/x64:$LD_LIBRARY_PATHThen I run but now there is a GLIBC error:
(softgym) seita@triton1:~/softgym (master) $ python examples/random_env.py --env_name ClothFlatten
Traceback (most recent call last):
File "examples/random_env.py", line 5, in <module>
from softgym.registered_env import env_arg_dict, SOFTGYM_ENVS
File "/home/seita/softgym/softgym/registered_env.py", line 1, in <module>
from softgym.envs.pour_water import PourWaterPosControlEnv
File "/home/seita/softgym/softgym/envs/pour_water.py", line 4, in <module>
import pyflex
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /home/seita/softgym/PyFlex/bindings/build/pyflex.cpython-36m-x86_64-linux-gnu.so)
(softgym) seita@triton1:~/softgym (master) $ ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu11.2) 2.23
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
(softgym) seita@triton1:~/softgym (master) $ Other stuff from that machine, with paths, CUDA, etc.
(softgym) seita@triton1:~/softgym (master) $ echo $LD_LIBRARY_PATH
/home/seita/softgym/PyFlex/external/SDL2-2.0.4/lib/x64:/usr/local/cuda/lib64:/usr/local/cuda-9.0/lib64:/home/seita/.mujoco/mjpro150/bin:/home/seita/.mujoco/mujoco200/bin:/usr/lib/nvidia-410
(softgym) seita@triton1:~/softgym (master) $
(softgym) seita@triton1:~/softgym (master) $ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176
(softgym) seita@triton1:~/softgym (master) $ nvidia-smi
Fri Feb 12 14:38:41 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104 Driver Version: 410.104 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 00000000:02:00.0 Off | N/A |
| 23% 27C P8 8W / 250W | 2MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 TITAN Xp Off | 00000000:03:00.0 On | N/A |
| 23% 31C P8 10W / 250W | 39MiB / 12192MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 1 31669 G /usr/lib/xorg/Xorg 36MiB |
+-----------------------------------------------------------------------------+
(softgym) seita@triton1:~/softgym (master) $ It's version 2.23 on this machine, whereas the Ubuntu 18 version in my post above has the correct 2.27 version. However, according to this:
https://stackoverflow.com/questions/59145051/glibc-2-27-not-found-ubuntu-16-04
in Ubuntu 16.04, this is the correct version, and upgrading it can be very tricky and not straightforward. In fact, the answer there even seems to suggest to switch to Ubuntu 18.04, which confuses me even more since I believe Ubuntu 16.04 is what the SoftGym paper is normally using.
Does this issue make sense?
Xingyu-Lin
yufeiwang63
FranBesq
Hello! Here are my attempts at installing SoftGym.
Contents:
Background
I have an Ubuntu 18.04 machine and I am attempting to get softgym to work. The machine has CUDA 10.0:
My directory structure is like this: conda environments are installed in
/home/seita/miniconda3/(I use miniconda) and softgym is cloned to/home/seita/softgym/. I am following these instructions in parallel:To start, after cloning a fresh copy of the repository, let's pull the docker image and make sure I have things updated. Here's what the output looks like:
Remark: I have a
.bashrcsetting that tells me the branch of a repo in parentheses, so it says(master)right after the "softgym" text in the command line.Creating the Conda Environment
First before entering docker, I create the conda environment. The instructions in this fork suggest making the conda environment inside docker, however what that does is it will make the conda environment created by the
rootuser and thus I won't be able to runconda install <xyz>commands outside of docker. In addition, installing it the way the README suggests will get the package versions set up correctly:Now that environment is saved in
/home/seita/miniconda3/envs/softgym/Entering Docker and Adjusting Paths
Next, let's go into docker to compile PyFlex. That is the only reason to use Docker. I run this command:
Explanation: the first
-vwill mount/home/seita/softgym(i.e., where I cloned softgym) to/workspace/softgyminside the docker. So inside docker, I can change directory to/workspace/softgymand it will look like as if I am inside/home/seita/softgym. A similar thing happens with the second mounting command for miniconda. In fact I'm using the same exact directory before and after the colon, which means the directory structure is the same inside docker. The other commands are just copied from what you have.Running the command means I am in a docker container as a "root" user. I go to the softgym directory. Next the current README here just says I need to adjust the PATH variable, and then run a
prepare_1.0.shscript. This other reference omits theprepare_1.0.shscript by just manually performing the same commands by adjustingPYFLEXROOT,PYTHONPATH, andLD_LIBRARY_PATH. Here are the environment variables at the beginning when I enter docker:Now let's follow the current README and adjust the path, then run the prepare script which will assign to three environment variables. The prepare script also activates the softgym conda environment that we created earlier outside of docker. We need to adjust the path so that the
. activate softgymcommand will work:Now we see that we are in the softgym conda environment, and furthermore, that the environment variables are updated:
Now we have to install the
pybind11package as stated in the README [output omitted as it's pretty standard]:Compiling PyFlex in Docker
Now let's try to compile PyFlex. The current instructions in this repo say to do this:
. ./prepare_1.0.sh && ./compile_1.0.sh. We've already done the first, let's do the second part to compile, and I could have merged the commands together by doingconda install pybind11outside of docker.This looks like it proceeded well, and outside docker I can see the Pyflex compiled module in
softgym/PyFlex/bindings/build/.Note: upon re-reading this post, I actually realize there is a slight typo, I did
./compile_1.0.shinstead of. ./compile_1.0.shwith the leading period. However I don't think this affects things and I re-did it with the period there and it gave the same output.Using the Compiled Code
Now we are outside of docker. I refresh via the
. ~/.bashrc, activate the same conda environment and attempt to run an example:Normally since this is the name of the package, I would expect to be able to do
pip install -e .or somewthere, but there is nosetup.pyfile so that must not be the issue. Following the other linked references at the top, perhaps we have to refresh the environment variables? Here's what I had before:Then I update them, but then that leads to a segmentation fault.
So now I am not sure how to proceed. The libraries seem to be installed:
(using the one-liner solution to check if a package is installed or not)
@Xingyu-Lin @yufeiwang63 If any of you two have time to reproduce this setup I am wondering how you managed to get around the segmentation fault? I know it might take a while but I wonder if any of you are able to start with a "clean" machine and then go through the full installation steps to get it working?