voila-dashboards / voici

Voici turns any Jupyter Notebook into a static web application
https://voici.readthedocs.io/
Other
143 stars 8 forks source link

VoiciBuildApp error when installing packages in xeus-python-kernel environment #83

Closed paulmaxus closed 5 months ago

paulmaxus commented 1 year ago

Description

When building the application an error is raised while attempting to install packages into the xeus-python-kernel conda environment.

Reproduce

(see complete CLI output under Context) Voici seems to build the app without any issues. Then, when the xeus kernel environment is created under C:\tmp\xeus-python-kernel\envs\xeus-python-kernel, packages are being downloaded and installed but then I run into this error: [VoiciBuildApp] ERROR | [lite] [post_build] [jupyterlite-xeus-python] [ERR] [Errno 2] No such file or directory: '\\tmp\\xeus-python-kernel\\envs\\xeus-python-kernel\\Scripts/pygmentize-script.py' (complete output below)

My only guess is that this is related to the xeus-python-kernel environment being created globally, while the conda installation is local.

Context

Command Line Output

```shell
$ voici build --contents notebooks
static:jupyter-lite.json
.  pre_status:static:jupyter-lite.json
    tarball:         jupyterlite-app-0.1.0.tgz 7MB
    output:          C:\Users\user\project\_output
    lite dir:        C:\Users\user\project
    apps:            voici
    sourcemaps:      True
    unused packages: True
archive:archive
contents:contents
lite:jupyter-lite.json
mathjax:status
mimetypes:jupyter-lite.json
serve:contents
settings:overrides
translation:translation
.  status:archive:archive
[VoiciBuildApp] No archive (yet): project-jupyterlite.tgz
.  status:contents:contents
    contents: 1 files
.  status:lite:jupyter-lite.json
[VoiciBuildApp]     jupyter-lite.(json|ipynb): 0 files
.  status:mathjax:status
     jupyter-server-mathjax: None
     configured MathJax dir: C:\Users\user\project\traitlets.Undefined
     effective MathJax dir: None
.  status:mimetypes:jupyter-lite.json
    filetypes:         26
.  status:serve:contents
    url: http://127.0.0.1:8000/
    server: tornado
    headers:
.  status:settings:overrides
    overrides.json: 0
.  status:translation:translation
    translation files: 2 files
static:output_dir
-- pre_init:static:output_dir
static:unpack
-- init:static:unpack
[VoiciBuildApp] WARNING | [static] app 'voici' is not one of: {'lab', 'repl', 'retro'}
static:prune:C:\Users\user\project\_output\lab
static:prune:C:\Users\user\project\_output\repl
static:prune:C:\Users\user\project\_output\retro
.  post_init:static:prune:C:\Users\user\project\_output\lab
.  post_init:static:prune:C:\Users\user\project\_output\repl
.  post_init:static:prune:C:\Users\user\project\_output\retro
federated_extensions:copy:ext:ipysigma
federated_extensions:copy:ext:jupyterlab_pygments
federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
federated_extensions:copy:ext:@jupyterlite/xeus-python-kernel
federated_extensions:copy:ext:@voila-dashboards/jupyterlab-preview
-- pre_build:federated_extensions:copy:ext:ipysigma
-- pre_build:federated_extensions:copy:ext:jupyterlab_pygments
-- pre_build:federated_extensions:copy:ext:@jupyter-widgets/jupyterlab-manager
-- pre_build:federated_extensions:copy:ext:@jupyterlite/xeus-python-kernel
-- pre_build:federated_extensions:copy:ext:@voila-dashboards/jupyterlab-preview
contents:copy:test.ipynb
translation:copy
-- build:contents:copy:test.ipynb
-- build:translation:copy
contents:contents:.
federated_extensions:patch
jupyterlite-xeus-python:xeus:copy:ext:@jupyterlite/xeus-python-kernel
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: C:\tmp\xeus-python-kernel\envs\xeus-python-kernel

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate C:\tmp\xeus-python-kernel\envs\xeus-python-kernel
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Collecting package metadata (current_repodata.json): done
Solving environment: /
Warning: >10 possible package resolutions (only showing differing packages):
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, co
nda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  - conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, conda-forge/noarch::six-1.16.0-pyh6c4a22f_0, https://repo.mamba.pm/conda
-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, ht
tps://repo.mamba.pm/conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  - conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::backports.functools_lru_cache-
1.6.4-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::executing-
1.2.0-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  - conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, https://repo.mamba.pm/conda-fo
rge/noarch::executing-1.2.0-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, conda-forge/noarch::six-1.16.0-pyh6c4a22f_0, https:/
/repo.mamba.pm/conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0
  - conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0, conda-forge/noarch::six-1.16.0-pyh6c4a22f_0, https://repo.mamba.pm/conda-forge/noarch::backports.functools_lru_cache-1.6.4
-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::six-1.16.0-pyh6c4a22f_0, https://repo.mamba.pm/conda-forge/noarch::executing-1.2.0
-pyhd8ed1ab_0, https://repo.mamba.pm/conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0
  - conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0, conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0, conda-forge/noarch::six-1.16.0-pyh6c4a22f_0, https:/
/repo.mamba.pm/conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0
  ... and othedone

## Package Plan ##

  environment location: C:\tmp\xeus-python-kernel\envs\xeus-python-kernel

  added / updated specs:
    - python=3.10
    - xeus-lite
    - xeus-python=0.15.7

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    setuptools-64.0.0          |  py310h8bed8af_0         758 KB  https://repo.mamba.pm/emscripten-forge
    ------------------------------------------------------------
                                           Total:         758 KB

The following NEW packages will be INSTALLED:

  asttokens          conda-forge/noarch::asttokens-2.2.1-pyhd8ed1ab_0
  backcall           conda-forge/noarch::backcall-0.2.0-pyh9f0ad1d_0
  backports          conda-forge/noarch::backports-1.0-pyhd8ed1ab_3
  backports.functoo~ conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0
  decorator          conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0
  emscripten-abi     emscripten-forge/noarch::emscripten-abi-3.1.27-hb0f4dca_6
  executing          conda-forge/noarch::executing-1.2.0-pyhd8ed1ab_0
  ipython            emscripten-forge/emscripten-32::ipython-7.32.0-py310h8bed8af_1
  jedi               conda-forge/noarch::jedi-0.18.2-pyhd8ed1ab_0
  matplotlib-inline  conda-forge/noarch::matplotlib-inline-0.1.6-pyhd8ed1ab_0
  numpy              emscripten-forge/emscripten-32::numpy-1.24.3-py310h6d2fff6_0
  parso              conda-forge/noarch::parso-0.8.3-pyhd8ed1ab_0
  pexpect            conda-forge/noarch::pexpect-4.8.0-pyh1a96a4e_2
  pickleshare        conda-forge/noarch::pickleshare-0.7.5-py_1003
  pip                conda-forge/noarch::pip-23.1.2-pyhd8ed1ab_0
  prompt-toolkit     conda-forge/noarch::prompt-toolkit-3.0.38-pyha770c72_0
  ptyprocess         conda-forge/noarch::ptyprocess-0.7.0-pyhd3deb0d_0
  pure_eval          conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0
  pygments           conda-forge/noarch::pygments-2.15.1-pyhd8ed1ab_0
  pyparsing          conda-forge/noarch::pyparsing-3.1.0-pyhd8ed1ab_0
  python             emscripten-forge/emscripten-32::python-3.10.2-h_hash_26_cpython
  requests-wasm-pol~ emscripten-forge/emscripten-32::requests-wasm-polyfill-0.3.0-h8bed8af_0
  setuptools         emscripten-forge/emscripten-32::setuptools-64.0.0-py310h8bed8af_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  stack_data         conda-forge/noarch::stack_data-0.6.2-pyhd8ed1ab_0
  traitlets          conda-forge/noarch::traitlets-5.9.0-pyhd8ed1ab_0
  wcwidth            conda-forge/noarch::wcwidth-0.2.6-pyhd8ed1ab_0
  wheel              conda-forge/noarch::wheel-0.40.0-pyhd8ed1ab_0
  xeus-lite          emscripten-forge/emscripten-32::xeus-lite-1.0.1-hf0acc6b_0
  xeus-python        emscripten-forge/emscripten-32::xeus-python-0.15.7-h4d17c68_0
  xeus-python-shell  conda-forge/noarch::xeus-python-shell-0.5.0-pyhd8ed1ab_0
  xeus-python-shell~ conda-forge/noarch::xeus-python-shell-raw-0.5.0-pyhd8ed1ab_0

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 89
compressed package into 4178
compressed in 7 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 10786149
compressed package into 5682024
compressed in 221 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 83
compressed package into 4172
compressed in 10 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 34000
compressed package into 22095
compressed in 15 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 83
compressed package into 4172
compressed in 7 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 16989
compressed package into 14392
compressed in 14 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 67051
compressed package into 42028
compressed in 20 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 15510210
compressed package into 8169834
compressed in 252 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 268518
compressed package into 152873
compressed in 125 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 10134
compressed package into 10521
compressed in 31 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 37210
compressed package into 26796
compressed in 51 ms

Remember to build the main file with `-sFORCE_FILESYSTEM` so that it includes support for loading this file package
compressing package of size 24296
compressed package into 17358
compressed in 42 ms

[VoiciBuildApp] ERROR | [lite] [post_build] [jupyterlite-xeus-python] [ERR] [Errno 2] No such file or directory: '\\tmp\\xeus-python-kernel\\envs\\xeus-python-kernel\\Scripts/p
ygmentize-script.py'
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\doit_cmd.py", line 294, in run
    return command.parse_execute(args)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\cmd_base.py", line 150, in parse_execute
    return self.execute(params, args)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\cmd_base.py", line 570, in execute
    return self._execute(**exec_params)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\cmd_run.py", line 265, in _execute
    return runner.run_all(self.control.task_dispatcher())
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\runner.py", line 254, in run_all
    self.run_tasks(task_dispatcher)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\runner.py", line 213, in run_tasks
    node = task_dispatcher.generator.send(node)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\control.py", line 629, in _dispatcher_generator
    next_step = node.step()
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\control.py", line 336, in step
    return next(self.generator)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\control.py", line 345, in _func
    for value in decorated(*args, **kwargs):
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\control.py", line 473, in _add_task
    new_tasks = generate_tasks(to_load, task_gen, ref.__doc__)
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\loader.py", line 390, in generate_tasks
    for task_dict, x_doc in flat_generator(gen_result, gen_doc):
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\doit\loader.py", line 27, in flat_generator
    for item in gen:
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_core\manager.py", line 145, in _delayed_gather
    for task in _gather():
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_core\manager.py", line 138, in _gather
    raise error
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_core\manager.py", line 128, in _gather
    for task in getattr(addon, attr)(self):
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_xeus_python\env_build_addon.py", line 79, in post_build
    env_prefix = build_and_pack_emscripten_env(
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_xeus_python\build.py", line 246, in build_and_pack_emscripten_env
    raise e
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\jupyterlite_xeus_python\build.py", line 211, in build_and_pack_emscripten_env
    split_pack_environment(
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\empack\file_packager.py", line 184, in split_pack_environment
    has_any_files = split_filter_pkg(
  File "C:\Users\user\AppData\Local\miniconda3\lib\site-packages\empack\filter_env.py", line 76, in split_filter_pkg
    shutil.copy(os.path.join(env_prefix, file), dest_fpath)
  File "C:\Users\user\AppData\Local\miniconda3\lib\shutil.py", line 417, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Users\user\AppData\Local\miniconda3\lib\shutil.py", line 254, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '\\tmp\\xeus-python-kernel\\envs\\xeus-python-kernel\\Scripts/pygmentize-script.py'
```

martinRenou commented 1 year ago

I'm not a Windows user, but this path \\tmp\\xeus-python-kernel\\envs\\xeus-python-kernel\\Scripts/pygmentize-script.py does not look correct for Windows, or is it?

@derthorsten do you know why this path could look like this and if that could lead to issues?

paulmaxus commented 1 year ago

Isn't this issue a xeus-python issue, i.e. related to https://github.com/jupyterlite/xeus-python-kernel/issues/124

martinRenou commented 1 year ago

It is indeed a xeus-python-kernel issue IMO. Probably that it does not work well yet on windows.

glw commented 8 months ago

Im having a similar issue when building the demo app for Voici in windows. Calling voici notebooks/ leads to an error, that I think is very similar to the one above. WindowsPath('/tmp/xeus-python-kernel/envs/xeus-python-kernel'),

Here's some more context:

File "C:\Users\user\miniconda3\envs\voiciV2\lib\subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\user\\miniconda3\\condabin\\conda.BAT', 'install', '--yes', '--prefix', WindowsPath('/tmp/xeus-python-kernel/envs/xeus-python-kernel'), '-c', 'https://repo.mamba.pm/emscripten-forge', '-c', 'https://repo.mamba.pm/conda-forge', 'python=3.10', 'xeus-lite', 'xeus-python=0.15.9', 'ipywidgets=8.0.6', 'jupyterlab_widgets=3.0.7']' returned non-zero exit status 1. 
martinRenou commented 8 months ago

Can you check that you are using the latest empack and jupyterlite-xeus-python versions?

glw commented 8 months ago

Hi Im using the version below I believe all of them are the latest.

xeus-python=0.15.8 xeus-python-shell=0.5.0 jupyterlite-xeus-python=0.9.6 empack=3.1.4

glw commented 8 months ago

Ive gone ahead and create a new environment since I was having trouble recalling how I originally built the last one. Unfortunately Im running into the same issue. Additionally I would point out this error as well.

InvalidArchiveError('Error with archive C:\\Users\\user\\miniconda3\\pkgs\\python-3.10.2-h_hash_28_cpython.tar.bz2. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: seeking backwards is not allowed')

Seems that cpython file downloaded here may have issues.

The full ouput is here

trungleduc commented 5 months ago

This should be fixed in 0.6.0, xeus-python-kernel is replaced by jupyterlite_xeus. Please feel free to re-open if needed.