search

aws-samples / sagemaker-run-notebook

Tools to run Jupyter notebooks as jobs in Amazon SageMaker - ad hoc, on a schedule, or in response to events
Apache License 2.0
142 stars 70 forks source link

Installing on current AWS Sagemaker JupyterLab #52

Closed papierGaylard closed 2 years ago

papierGaylard commented 2 years ago

Hi there,

I'm wondering if anyone has had success installing this and scheduling a notebook using AWS Sagemaker Jupyter Lab?

Following the Quick Start, I've created the using command run notebook create stack from my local machine, using my was credentials. It creates and the stack says 'successful'.

Next, I run the run-notebook create-container, which runs the AWS codebuild job, which runs successfully.

Then, I load into my AWS Sagemaker jupyter lab notebook instance and I try to run the command install-run-notebook.sh.

Then I run the command and it successfully finished.

However when I refresh the browser, restart the client or shut down start the notebook instance I can't get the extension to pop up.

Any ideas?

papierGaylard commented 2 years ago

Here's an update with my log output from the 'install-run-notebook.sh':

sh-4.2$ bash install-run-notebook.sh 
Collecting https://github.com/aws-samples/sagemaker-run-notebook/releases/download/v0.20.0/sagemaker_run_notebook-0.20.0.tar.gz
  Downloading https://github.com/aws-samples/sagemaker-run-notebook/releases/download/v0.20.0/sagemaker_run_notebook-0.20.0.tar.gz (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.7/64.7 KB 1.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: boto3>=1.10.44 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from sagemaker-run-notebook==0.20.0) (1.21.42)
Requirement already satisfied: botocore<1.25.0,>=1.24.42 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (1.24.42)
Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (0.5.2)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (1.0.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from botocore<1.25.0,>=1.24.42->boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (1.26.8)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from botocore<1.25.0,>=1.24.42->boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (2.8.2)
Requirement already satisfied: six>=1.5 in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.25.0,>=1.24.42->boto3>=1.10.44->sagemaker-run-notebook==0.20.0) (1.16.0)
yarn config v1.15.2
success Set "cache-folder" to "/tmp/yarncache".
Done in 0.03s.
[LabBuildApp] Searching ['/home/ec2-user/SageMaker', '/home/ec2-user/.jupyter', '/home/ec2-user/.local/etc/jupyter', '/home/ec2-user/anaconda3/envs/JupyterSystemEnv/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[LabBuildApp] Looking for jupyter_config in /etc/jupyter
[LabBuildApp] Looking for jupyter_config in /usr/local/etc/jupyter
[LabBuildApp] Looking for jupyter_config in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/etc/jupyter
[LabBuildApp] Loaded config file: /home/ec2-user/anaconda3/envs/JupyterSystemEnv/etc/jupyter/jupyter_config.json
[LabBuildApp] Looking for jupyter_config in /home/ec2-user/.local/etc/jupyter
[LabBuildApp] Looking for jupyter_config in /home/ec2-user/.jupyter
[LabBuildApp] Looking for jupyter_config in /home/ec2-user/SageMaker
[LabBuildApp] WARNING | Config option `kernel_spec_manager_class` not recognized by `LabBuildApp`.
[LabBuildApp] Looking for JupyterLab_config in /etc/jupyter
[LabBuildApp] Looking for JupyterLab_config in /usr/local/etc/jupyter
[LabBuildApp] Looking for JupyterLab_config in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/etc/jupyter
[LabBuildApp] Looking for JupyterLab_config in /home/ec2-user/.local/etc/jupyter
[LabBuildApp] Looking for JupyterLab_config in /home/ec2-user/.jupyter
[LabBuildApp] Looking for JupyterLab_config in /home/ec2-user/SageMaker
[LabBuildApp] JupyterLab 1.2.21
[LabBuildApp] Building in /home/ec2-user/anaconda3/envs/JupyterSystemEnv/share/jupyter/lab
[LabBuildApp] Yarn configuration loaded.
[LabBuildApp] Node v14.18.3

[LabBuildApp] Building jupyterlab assets (build:prod)
[LabBuildApp] WARNING | 
"sagemaker_run_notebook@0.20.0" is not compatible with the current JupyterLab
Conflicting Dependencies:
JupyterLab              Extension        Package
>=1.2.11 <1.3.0         >=2.0.0 <3.0.0   @jupyterlab/application
>=1.2.11 <1.3.0         >=2.0.0 <3.0.0   @jupyterlab/apputils
>=1.2.12 <1.3.0         >=2.0.0 <3.0.0   @jupyterlab/notebook
>=16.8.4 <16.9.0        >=16.9.0 <16.10.0react

[LabBuildApp] > node /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/jupyterlab/staging/yarn.js install --non-interactive
\[LabBuildApp] yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
success Already up-to-date.
Done in 0.74s.

[LabBuildApp] > node /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer --fail
|[LabBuildApp] yarn run v1.15.2
$ /home/ec2-user/anaconda3/envs/JupyterSystemEnv/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer --fail
Done in 0.48s.

[LabBuildApp] > node /home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.7/site-packages/jupyterlab/staging/yarn.js run build:prod
|[LabBuildApp] yarn run v1.15.2
$ ensure-max-old-space webpack --config webpack.prod.config.js
Hash: 7b221e4bd819e82e79e9afbdb2b950813f5fe9516df6485fbe4237f9154e
Version: webpack 4.46.0
Child
    Hash: 7b221e4bd819e82e79e9
    Time: 51318ms
    Built at: 05/18/2022 4:04:36 PM
                                                              Asset      Size  Chunks                                Chunk Names
               2.f4b81e2bc1963983ecc5.js.map?v=f4b81e2bc1963983ecc5  1.05 MiB       2  [emitted] [dev]               
                   2.f4b81e2bc1963983ecc5.js?v=f4b81e2bc1963983ecc5   869 KiB       2  [emitted] [immutable]  [big]  
               3.80d8e6cc8397ffebf731.js.map?v=80d8e6cc8397ffebf731   500 KiB       3  [emitted] [dev]               
                   3.80d8e6cc8397ffebf731.js?v=80d8e6cc8397ffebf731   425 KiB       3  [emitted] [immutable]  [big]  
               4.b9d6e5fd43159ea216bf.js.map?v=b9d6e5fd43159ea216bf   914 KiB       4  [emitted] [dev]               
                   4.b9d6e5fd43159ea216bf.js?v=b9d6e5fd43159ea216bf   750 KiB       4  [emitted] [immutable]  [big]  
               5.996e6d254152c9a96d99.js.map?v=996e6d254152c9a96d99   889 KiB       5  [emitted] [dev]               
                   5.996e6d254152c9a96d99.js?v=996e6d254152c9a96d99   720 KiB       5  [emitted] [immutable]  [big]  
               6.64052a7445962e5fc5af.js.map?v=64052a7445962e5fc5af   249 KiB       6  [emitted] [dev]               
                   6.64052a7445962e5fc5af.js?v=64052a7445962e5fc5af   209 KiB       6  [emitted] [immutable]         
                               674f50d287a8c48dc19ba404d20fe713.eot   162 KiB          [emitted]                     
               7.07aebb4cee80d7092240.js.map?v=07aebb4cee80d7092240  5.69 KiB       7  [emitted] [dev]               
                   7.07aebb4cee80d7092240.js?v=07aebb4cee80d7092240  4.27 KiB       7  [emitted] [immutable]         
                               912ec66d7572ff821749319396470bde.svg   434 KiB          [emitted]              [big]  
                             af7ae505a9eed503f8b8e6982036873e.woff2  75.4 KiB          [emitted]                     
                               b06871f281fee6b241d60582ae9369b9.ttf   162 KiB          [emitted]                     
                              fee66e712a8a08eef5805a46892932ad.woff  95.7 KiB          [emitted]                     
                                                         index.html  1.79 KiB          [emitted]                     
            main.20ee82e362ceba29f352.js.map?v=20ee82e362ceba29f352  83.6 KiB       0  [emitted] [dev]               main
                main.20ee82e362ceba29f352.js?v=20ee82e362ceba29f352  64.1 KiB       0  [emitted] [immutable]         main
    vendors~main.9492379580ec0d04812a.js.map?v=9492379580ec0d04812a  13.4 MiB       1  [emitted] [dev]               vendors~main
        vendors~main.9492379580ec0d04812a.js?v=9492379580ec0d04812a  11.5 MiB       1  [emitted] [immutable]  [big]  vendors~main
    Entrypoint main [big] = vendors~main.9492379580ec0d04812a.js?v=9492379580ec0d04812a vendors~main.9492379580ec0d04812a.js.map?v=9492379580ec0d04812a main.20ee82e362ceba29f352.js?v=20ee82e362ceba29f352 main.20ee82e362ceba29f352.js.map?v=20ee82e362ceba29f352
    [0] multi whatwg-fetch ./build/index.out.js 40 bytes {0} [built]
    [/KN4] ./node_modules/@jupyterlab/filebrowser-extension/lib/index.js + 6 modules 127 KiB {1} [optional] [built]
        |    7 modules
    [31N0] ./node_modules/@jupyterlab/csvviewer-extension/lib/index.js + 5 modules 66.1 KiB {1} [optional] [built]
        |    6 modules
    [5pV8] ./node_modules/@jupyterlab/mathjax2-extension/lib/index.js + 1 modules 4.04 KiB {1} [optional] [built]
        |    2 modules
    [8943] ./node_modules/@jupyterlab/mainmenu-extension/lib/index.js + 10 modules 37 KiB {1} [optional] [built]
        |    11 modules
    [9fgM] ./build/imports.css 1.06 KiB {0} [built]
    [ANye] ./build/index.out.js 45.1 KiB {0} [built]
    [NHPb] ./node_modules/@jupyterlab/console-extension/lib/index.js + 6 modules 66.9 KiB {1} [optional] [built]
        |    7 modules
    [QP8U] ./node_modules/@jupyterlab/fileeditor-extension/lib/index.js + 1 modules 33.1 KiB {1} [optional] [built]
        |    2 modules
    [QbIU] ./node_modules/@jupyterlab/running-extension/lib/index.js + 1 modules 9.6 KiB {1} [optional] [built]
        |    2 modules
    [RMrj] ./node_modules/@jupyterlab/inspector-extension/lib/index.js + 1 modules 8.5 KiB {1} [optional] [built]
        |    2 modules
    [U33M] ./node_modules/@jupyterlab/logconsole-extension/lib/index.js + 6 modules 45.1 KiB {1} [optional] [built]
        |    7 modules
    [VYmV] ./node_modules/@jupyterlab/completer-extension/lib/index.js + 7 modules 61.6 KiB {1} [optional] [built]
        |    8 modules
    [ZPDT] ./node_modules/@jupyterlab/extensionmanager-extension/lib/index.js + 6 modules 43.4 KiB {1} [optional] [built]
        |    7 modules
    [dp0r] ./node_modules/@jupyterlab/git/lib/index.js + 113 modules 544 KiB {1} [optional] [built]
        |    114 modules
        + 1937 hidden modules

    WARNING in @material-ui/core
      Multiple versions of @material-ui/core found:
        3.9.4 ./~/sagemaker_session_manager/~/@material-ui/core from ./~/sagemaker_session_manager/~/@material-ui/core/styles/createMuiTheme.js
        4.12.4 ./~/@material-ui/core from ./~/@material-ui/core/esm/styles/index.js

    WARNING in @material-ui/utils
      Multiple versions of @material-ui/utils found:
        3.0.0-alpha.3 ./~/sagemaker_session_manager/~/@material-ui/utils from ./~/sagemaker_session_manager/~/@material-ui/utils/index.es.js
        4.11.3 ./~/@material-ui/utils from ./~/@material-ui/utils/esm/index.js

    WARNING in bn.js
      Multiple versions of bn.js found:
        4.12.0 ./~/bn.js from ./~/create-ecdh/browser.js
        5.2.0 ./~/browserify-rsa/~/bn.js from ./~/browserify-rsa/index.js

    WARNING in dom-helpers
      Multiple versions of dom-helpers found:
        3.4.0 ./~/dom-helpers from ./~/dom-helpers/util/hyphenateStyle.js
        5.2.1 ./~/@material-ui/core/~/dom-helpers from ./~/@material-ui/core/~/react-transition-group/esm/CSSTransition.js

    WARNING in entities
      Multiple versions of entities found:
        1.1.2 ./~/htmlparser2/~/entities from ./~/htmlparser2/lib/Tokenizer.js
        2.2.0 ./~/entities from ./~/entities/lib/decode.js

    WARNING in react-transition-group
      Multiple versions of react-transition-group found:
        2.9.0 ./~/react-transition-group from ./~/@blueprintjs/core/lib/esm/components/overlay/overlay.js
        4.4.2 ./~/@material-ui/core/~/react-transition-group from ./~/@material-ui/core/~/react-transition-group/esm/CSSTransition.js

    WARNING in readable-stream
      Multiple versions of readable-stream found:
        2.3.7 ./~/readable-stream from ./~/stream-browserify/index.js
        3.6.0 ./~/hash-base/~/readable-stream from ./~/hash-base/~/readable-stream/readable-browser.js

    WARNING in safe-buffer
      Multiple versions of safe-buffer found:
        5.1.2 ./~/safe-buffer from ./~/randombytes/browser.js
        5.2.1 ./~/hash-base/~/safe-buffer from ./~/hash-base/index.js

    WARNING in warning
      Multiple versions of warning found:
        3.0.0 ./~/sagemaker_session_manager/~/jss/~/warning from ./~/sagemaker_session_manager/~/jss/lib/SheetsManager.js
        4.0.3 ./~/warning from ./~/react-popper/lib/esm/Reference.js

    Check how you can resolve duplicate packages: 
    https://github.com/darrenscerri/duplicate-package-checker-webpack-plugin#resolving-duplicate-packages-in-your-bundle

    WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
    This can impact web performance.
    Assets: 
      912ec66d7572ff821749319396470bde.svg (434 KiB)
      vendors~main.9492379580ec0d04812a.js?v=9492379580ec0d04812a (11.5 MiB)
      2.f4b81e2bc1963983ecc5.js?v=f4b81e2bc1963983ecc5 (869 KiB)
      3.80d8e6cc8397ffebf731.js?v=80d8e6cc8397ffebf731 (425 KiB)
      4.b9d6e5fd43159ea216bf.js?v=b9d6e5fd43159ea216bf (750 KiB)
      5.996e6d254152c9a96d99.js?v=996e6d254152c9a96d99 (720 KiB)

    WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
    Entrypoints:
      main (11.5 MiB)
          vendors~main.9492379580ec0d04812a.js?v=9492379580ec0d04812a
          main.20ee82e362ceba29f352.js?v=20ee82e362ceba29f352

    Child html-webpack-plugin for "index.html":
         1 asset
        Entrypoint undefined = index.html
        [KTNU] ./node_modules/html-loader!./templates/partial.html 703 bytes {0} [built]
        [YuTi] (webpack)/buildin/module.js 497 bytes {0} [built]
        [aS2v] ./node_modules/html-webpack-plugin/lib/loader.js!./templates/template.html 1.27 KiB {0} [built]
        [yLpj] (webpack)/buildin/global.js 472 bytes {0} [built]
            + 1 hidden module
Child
    Hash: afbdb2b950813f5fe951
    Time: 2362ms
    Built at: 05/18/2022 4:03:47 PM
        Asset       Size  Chunks             Chunk Names
    index.css   85.9 KiB       0  [emitted]  index
     index.js  932 bytes       0  [emitted]  index
    Entrypoint index = index.css index.js
       4 modules
    Child mini-css-extract-plugin node_modules/css-loader/dist/cjs.js!node_modules/@jupyterlab/theme-dark-extension/style/index.css:
        Entrypoint mini-css-extract-plugin = *
           56 modules
Child
    Hash: 6df6485fbe4237f9154e
    Time: 2357ms
    Built at: 05/18/2022 4:03:47 PM
        Asset       Size  Chunks             Chunk Names
    index.css   83.3 KiB       0  [emitted]  index
     index.js  932 bytes       0  [emitted]  index
    Entrypoint index = index.css index.js
       4 modules
    Child mini-css-extract-plugin node_modules/css-loader/dist/cjs.js!node_modules/@jupyterlab/theme-light-extension/style/index.css:
        Entrypoint mini-css-extract-plugin = *
           57 modules
Done in 53.25s.

nohup: ignoring input and appending output to ‘nohup.out’
papierGaylard commented 2 years ago

UPDATE: If anyone else is looking to automate their jupyter notebook scripts in was JUPYTER LAB. Then follow the instructions for JUPYTER LAB and not AWS Sagemaker Studio!

(for me) The base install on AWS uses jupyter lab v1, so go to the V0.19.0 and work from there. Restart your instance once you have the lifecycle policy attached (you also need to shut down and edit the instance once done so to update the life cycle config).

tomfaulhaber commented 2 years ago

v0.22.0 is now released with JupyterLab 3.x support.