visgl / deck.gl

WebGL2 powered visualization framework
https://deck.gl
MIT License
12.26k stars 2.08k forks source link

JupyterLab @deck.gl/jupyter-widget extension build fails #4045

Closed fedden closed 4 years ago

fedden commented 4 years ago

Description

Hey folks - awesome library, really loving using it.

I am trying to install the jupyter-lab extension that comes with pydeck, but it crashes on the command:

jupyter lab build

Repro Steps

The first thing I do is clone deck.gl, which is incidentally on branch master. Next as per the instructions found here I change directory to the widget directory, and attempt to build it as per the instrctions from the README.

(dev) ➜  ~ git clone https://github.com/uber/deck.gl/
Cloning into 'deck.gl'...
remote: Enumerating objects: 31, done.
remote: Counting objects: 100% (31/31), done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 54715 (delta 2), reused 8 (delta 0), pack-reused 54684
Receiving objects: 100% (54715/54715), 164.52 MiB | 6.76 MiB/s, done.
Resolving deltas: 100% (38993/38993), done.
Checking connectivity... done.
(dev) ➜  ~ cd deck.gl/modules/jupyter-widget
(dev) ➜  jupyter-widget git:(master) jupyter labextension install @jupyter-widgets/jupyterlab-manager@1.0.3 --no-build
(dev) ➜  jupyter-widget git:(master) jupyter labextension install . --no-build
(dev) ➜  jupyter-widget git:(master) jupyter lab build
[LabBuildApp] JupyterLab 1.1.4
[LabBuildApp] Building in /home/tollie/miniconda3/envs/mogees/share/jupyter/lab
[LabBuildApp] Building jupyterlab assets (build:dev:minimize)
An error occured.
RuntimeError: JupyterLab failed to build
See the log file for details:  /tmp/jupyterlab-debug-o4jc1of3.log

As you can see the command jupyter lab build fails, and dumps a log to /tmp/jupyterlab-debug-o4jc1of3.log. The following is the contents of the log:

[LabBuildApp] Building jupyterlab assets (build:dev:minimize)
[LabBuildApp] > /home/tollie/miniconda3/envs/mogees/bin/npm pack /home/tollie/deck.gl/modules/jupyter-widget
[LabBuildApp] npm notice
npm notice 📦  @deck.gl/jupyter-widget@8.0.0-beta.2
npm notice === Tarball Contents ===
npm notice 1.3kB package.json
npm notice 136B  README.md
npm notice 2.3kB src/create-deck.js
npm notice 348B  src/deck-bundle.js
npm notice 1.1kB src/index.js
npm notice 833B  src/plugin.js
npm notice 220B  src/ssr-safe-mapbox.js
npm notice 303B  src/version.js
npm notice 3.5kB src/widget-tooltip.js
npm notice 4.8kB src/widget.js
npm notice === Tarball Details ===
npm notice name:          @deck.gl/jupyter-widget
npm notice version:       8.0.0-beta.2
npm notice filename:      deck.gl-jupyter-widget-8.0.0-beta.2.tgz
npm notice package size:  5.5 kB
npm notice unpacked size: 14.9 kB
npm notice shasum:        15b9ff4a06e5b3d6afcc6ba6e4fedcde35fcf5c0
npm notice integrity:     sha512-Ax8nqWObMB2zk[...]CceLcwuiFDO6Q==
npm notice total files:   10
npm notice
deck.gl-jupyter-widget-8.0.0-beta.2.tgz

[LabBuildApp] > node /home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/staging/yarn.js install --non-interactive
[LabBuildApp] yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@1.2.9: The platform "linux" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning "@deck.gl/jupyter-widget > @deck.gl/json@8.0.0-beta.2" has unmet peer dependency "@deck.gl/core@^8.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/geo-layers@8.0.0-beta.2" has unmet peer dependency "@deck.gl/core@^8.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers@8.0.0-beta.2" has unmet peer dependency "@deck.gl/core@^8.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/layers@8.0.0-beta.2" has unmet peer dependency "@deck.gl/core@^8.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/aggregation-layers@8.0.0-beta.2" has unmet peer dependency "@deck.gl/core@^8.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers > @luma.gl/experimental@8.0.0-beta.4" has unmet peer dependency "@loaders.gl/gltf@^2.0.0-beta".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers > @luma.gl/experimental@8.0.0-beta.4" has unmet peer dependency "@luma.gl/engine@^7.1.0".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers > @luma.gl/experimental@8.0.0-beta.4" has unmet peer dependency "@luma.gl/gltools@^7.1.0".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers > @luma.gl/experimental@8.0.0-beta.4" has unmet peer dependency "@luma.gl/shadertools@^7.1.0".
warning "@deck.gl/jupyter-widget > @deck.gl/mesh-layers > @luma.gl/experimental@8.0.0-beta.4" has unmet peer dependency "@luma.gl/webgl@^7.1.0".
[5/5] Building fresh packages...
success Saved lockfile.
Done in 30.72s.

[LabBuildApp] > node /home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer
[LabBuildApp] yarn run v1.15.2
$ /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer
Done in 0.66s.

[LabBuildApp] > node /home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/staging/yarn.js run build:dev:minimize
[LabBuildApp] yarn run v1.15.2
$ jlpm run build:dev
$ jlpm run build
$ webpack
ModuleNotFoundError: Module not found: Error: Can't resolve '../dist/index' in '/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/src'
    at factory.create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/webpack/lib/Compilation.js:925:10)
    at factory (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:401:22)
    at resolver (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:130:21)
    at asyncLib.parallel (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:224:22)
    at /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/neo-async/async.js:2830:7
    at /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/neo-async/async.js:6877:13
    at normalResolver.resolve (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/webpack/lib/NormalModuleFactory.js:214:25)
    at doResolve (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:184:12)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn0 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at resolver.doResolve (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:37:5)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn0 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn0 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:27:1)
    at resolver.doResolve (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:42:38)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn42 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:16:1)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn0 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:27:1)
    at resolver.doResolve (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:42:38)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn1 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:16:1)
    at hook.callAsync (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/Resolver.js:238:5)
    at _fn0 (eval at create (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at fs.stat (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/DirectoryExistsPlugin.js:22:13)
    at process.nextTick (/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:73:15)
    at processTicksAndRejections (internal/process/task_queues.js:79:9)
resolve '../dist/index' in '/home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/src'
  using description file: /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/package.json (relative path: ./src)
    Field 'browser' doesn't contain a valid alias configuration
    using description file: /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/package.json (relative path: ./dist/index)
      no extension
        Field 'browser' doesn't contain a valid alias configuration
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index doesn't exist
      .wasm
        Field 'browser' doesn't contain a valid alias configuration
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index.wasm doesn't exist
      .mjs
        Field 'browser' doesn't contain a valid alias configuration
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index.mjs doesn't exist
      .js
        Field 'browser' doesn't contain a valid alias configuration
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index.js doesn't exist
      .json
        Field 'browser' doesn't contain a valid alias configuration
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index.json doesn't exist
      as directory
        /home/tollie/miniconda3/envs/mogees/share/jupyter/lab/staging/node_modules/@deck.gl/jupyter-widget/dist/index doesn't exist
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):

[LabBuildApp]   File "/home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/debuglog.py", line 47, in debug_logging
    yield

[LabBuildApp]   File "/home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/labapp.py", line 96, in start
    core_config=self.core_config)

[LabBuildApp]   File "/home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/commands.py", line 378, in build
    command=command, clean_staging=clean_staging)

[LabBuildApp]   File "/home/tollie/miniconda3/envs/mogees/lib/python3.7/site-packages/jupyterlab/commands.py", line 583, in build
    raise RuntimeError(msg)

[LabBuildApp] RuntimeError: JupyterLab failed to build

[LabBuildApp] Exiting application: JupyterLab

Environment (please complete the following information):

Logs

fedden commented 4 years ago

Hey just incase it helps, I've made a short dockerfile that reproduces the issue. Save the following to a file called Dockerfile:

FROM continuumio/miniconda3:4.7.12
RUN conda install -y -c conda-forge jupyterlab
RUN conda install -y -c conda-forge nodejs
RUN git clone https://github.com/uber/deck.gl/
WORKDIR /deck.gl/modules/jupyter-widget
RUN jupyter labextension install @jupyter-widgets/jupyterlab-manager@1.0.3 --no-build
RUN jupyter labextension install . --no-build
RUN jupyter lab build
CMD ["bash"]

You could run the following command in the same directory that the dockerfile is in to build and reporduce the error described above!

docker build -t test .
Pessimistress commented 4 years ago

You may need to run yarn bootstrap at deck.gl root first.

fedden commented 4 years ago

Thanks for the hint, that sorted it!

Just for readers: Running yarn bootstrap failed due to jq not being installed, but then running: sudo apt-get install jq

Sorted that issue, and after that the widget built!

Thanks for the help @Pessimistress!

fedden commented 4 years ago

What I've noticed now is after calling the show() method of my pydeck.Deck instance, the output for the notebook cell just hangs with:

Loading widget...

If I then inspect the console in the browser, I can see this output when executing the cell:

Screenshot 2019-12-18 at 23 46 39

I'd be appreciative if you have any thoughts here

fedden commented 4 years ago

I think (from my limited understanding of the jupyter ecosystem and tooling) that the deck.gl extension is installed:

(dev) ➜  ~ jupyter labextension list
JupyterLab v1.1.4
Known labextensions:
   app dir: /home/tollie/miniconda3/envs/mogees/share/jupyter/lab
        @deck.gl/jupyter-widget v8.0.0-beta.2  enabled  OK*
        @jupyter-widgets/jupyterlab-manager v1.0.3  enabled  OK

   local extensions:
        @deck.gl/jupyter-widget: /home/tollie/deck.gl/modules/jupyter-widget
ajduberstein commented 4 years ago

Are you installing pydeck from its source in your Dockerfile? You seem to be building the frontend widget but pydeck doesn't appear present.

ajduberstein commented 4 years ago

pydeck 0.2.0 was released, which should solve this issue.

As an example, this Dockerfile works–

FROM jupyter/minimal-notebook

RUN pip install jupyterlab && \ 
    jupyter serverextension enable --py jupyterlab --sys-prefix && \
    pip install pydeck && \
    jupyter labextension install @jupyter-widgets/jupyterlab-manager && \
    jupyter labextension install @deck.gl/jupyter-widget

EXPOSE 8888
CMD ["jupyter", "lab"]

To build and run:

docker build -t pydeckDemo:latest .
docker run -p 8888:8888 pydeckDemo:latest jupyter lab --ip 0.0.0.0