search

aws / studio-lab-examples

Example notebooks for working with SageMaker Studio Lab. Sign up for an account at the link below!
https://studiolab.sagemaker.aws
Apache License 2.0
655 stars 193 forks source link

How to get root access in SageMaker Studio Lab #118

Closed ar8372 closed 2 years ago

ar8372 commented 2 years ago

Hi, I am trying to install some libraries in Studio Lab which requires root privileges.

Below I have run whoami to check if I am root user. (I am not as it should print 'root' in case of root user) whoami_image

Below you can see the error on running sudo: -> bash: sudo: command not found sudo_cmd

I followed this link to install sudo. On running su - , It asks for the password, but we don't have any password for Studio Lab. password

Can anyone tell how to get root access or a way to install libraries which require root access/(or packages which installs using sudo). Please let me know if my query is not clear.

icoxfog417 commented 2 years ago

Thank you for trying Studio Lab. Now Studio Lab does not allow sudo (similar issue: https://github.com/aws/studio-lab-examples/issues/40#issuecomment-1005305538). We can use pip and conda instead. Please refer the following issue.

What software do you try to install? Some libraries will be available in conda-forge . Here is the sample of search.

https://anaconda.org/search?q=gym

ar8372 commented 2 years ago

Thanks for the reply, I will try to explain the issue. I am trying to run bipedal robot from Open-ai gym. 

!pip install gym
!apt-get update
!apt-get -qq -y install xvfb freeglut3-dev ffmpeg> /dev/null
!apt-get install xvfb
!pip install pyvirtualdisplay 
!pip -q install pyglet
!pip -q install pyopengl
!apt-get install swig
!pip install box2d box2d-kengz
!pip install pybullet

These are the libraries which I need to install for the code to work. It works fine on google-colab: (screenshot below) colab_gym_ss

But it throws error when I run it on SageMaker Studio Lab: (screenshot below) sagemaker1

Error Log (I have made gym_install.sh file which installs everything described above, I am running it below.): 

(AKR_env) studio-lab-user@default:~/sagemaker-studiolab-notebooks/GM_/ARS_src$ sh gym_install.sh
Requirement already satisfied: gym in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (0.24.1)
Requirement already satisfied: gym-notices>=0.0.4 in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (from gym) (0.0.7)
Requirement already satisfied: numpy>=1.18.0 in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (from gym) (1.22.4)
Requirement already satisfied: cloudpickle>=1.2.0 in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (from gym) (2.1.0)
Reading package lists... Done
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
Requirement already satisfied: pyvirtualdisplay in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (3.0)
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?
Collecting box2d
  Using cached Box2D-2.3.2.tar.gz (427 kB)
  Preparing metadata (setup.py) ... done
Collecting box2d-kengz
  Using cached Box2D-kengz-2.3.3.tar.gz (425 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: box2d, box2d-kengz
  Building wheel for box2d (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Using setuptools (version 62.3.3).
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/Box2D.py -> build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D
      creating build/lib.linux-x86_64-cpython-310/Box2D/b2
      copying library/Box2D/b2/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D/b2
      running build_ext
      building 'Box2D._Box2D' extension
      swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
      swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
      error: command 'swig' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for box2d
  Running setup.py clean for box2d
  Building wheel for box2d-kengz (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Using setuptools (version 62.3.3).
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/Box2D.py -> build/lib.linux-x86_64-cpython-310/Box2D
      creating build/lib.linux-x86_64-cpython-310/Box2D/b2
      copying library/Box2D/b2/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D/b2
      running build_ext
      building 'Box2D._Box2D' extension
      swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
      swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
      error: command 'swig' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for box2d-kengz
  Running setup.py clean for box2d-kengz
Failed to build box2d box2d-kengz
Installing collected packages: box2d-kengz, box2d
  Running setup.py install for box2d-kengz ... error
  error: subprocess-exited-with-error

  × Running setup.py install for box2d-kengz did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      Using setuptools (version 62.3.3).
      running install
      /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D
      copying library/Box2D/Box2D.py -> build/lib.linux-x86_64-cpython-310/Box2D
      creating build/lib.linux-x86_64-cpython-310/Box2D/b2
      copying library/Box2D/b2/__init__.py -> build/lib.linux-x86_64-cpython-310/Box2D/b2
      running build_ext
      building 'Box2D._Box2D' extension
      swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
      swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
      error: command 'swig' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> box2d-kengz

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
Requirement already satisfied: pybullet in /home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages (3.2.5)
(AKR_env) studio-lab-user@default:~/sagemaker-studiolab-notebooks/GM_/ARS_src$

To be specific I am getting error in this line:

(AKR_env) studio-lab-user@default:~/sagemaker-studiolab-notebooks/GM_/ARS_src$ apt-get install xvfb
E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), are you root?

So as mentioned by you I tried to find xvfb in conda-forge, but couldn't find it. There are some wrapper xvfb on conda-forge but installing them didn't help with the error. 

(AKR_env) studio-lab-user@default:~/sagemaker-studiolab-notebooks/GM_/ARS_src$ python check.py
Traceback (most recent call last):
  File "/home/studio-lab-user/sagemaker-studiolab-notebooks/GM_/ARS_src/check.py", line 9, in <module>
    display = Display(visible=0, size=(1024, 768))
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages/pyvirtualdisplay/display.py", line 54, in __init__
    self._obj = cls(
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages/pyvirtualdisplay/xvfb.py", line 44, in __init__
    AbstractDisplay.__init__(
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages/pyvirtualdisplay/abstractdisplay.py", line 85, in __init__
    helptext = get_helptext(program)
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/site-packages/pyvirtualdisplay/util.py", line 13, in get_helptext
    p = subprocess.Popen(
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/subprocess.py", line 966, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/studio-lab-user/.conda/envs/AKR_env/lib/python3.10/subprocess.py", line 1842, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'Xvfb'
(AKR_env) studio-lab-user@default:~/sagemaker-studiolab-notebooks/GM_/ARS_src$

To reproduce above error run below code (check.py) :->

import os
import numpy as np
import gym
from gym import wrappers
import pyvirtualdisplay
from pyvirtualdisplay import Display

if __name__ == "__main__":
    display = Display(visible=0, size=(1024, 768))
icoxfog417 commented 2 years ago

Thank you for sharing the error message. Studio Lab does not allow apt install so that the gym_install.sh does not work straightly. We have to prepare the environment by conda and pip.

At first we can install swig from conda. We can not install xvfb from conda, is this necessary to run the code?

ar8372 commented 2 years ago

Basically I have to capture the video output using Display of pyvirtualdisplay library. Everything else works. As you can see display = Display(visible=0, size=(1024, 768)) this line throws error FileNotFoundError: [Errno 2] No such file or directory: 'Xvfb' , so I am trying to install Xvfb.

Thanks,

icoxfog417 commented 2 years ago

@ar8372 Sorry for the late reply. I raised the #124 to work OpenAI Gym in Studio Lab. Please comment to #124 if you have additional information. We need your insight to solve the issue. If you do not mind, please close this issue to suppress the duplication of issues.