Conda install dependencies incomplete

sterlinm commented 3 years ago

Looks like a really cool project! Not a huge deal, but I think the conda recipe is missing a few required packages that are needed to run the Palmer Penguins example dashboard. I haven't tested it for the other examples.

ALL software version info

I'm doing this on a mac using conda and mamba (I tried with conda after it didn't work with mamba).

$ mamba --version
mamba 0.3.7
conda 4.8.3

Description of expected behavior and the observed behavior

I'd expect that I could create a new conda environment with only python 3 installed, run conda install -c pyviz lumen and then run one of the example dashboards with lumen serve dashboard.yaml --show. When I do that it starts to serve the dashboard but I get errors regarding missing packages and the dashboard doesn't load (though it opens a new browser tab).

Complete, minimal, self-contained example code that reproduces the issue

I'm using the penguins dashboard.yml file.

Here's the environment.yml file I expect to work.

name: lumen
channels:
  - pyviz
  - conda-forge
dependencies:
  - python=3.8
  - lumen

I then run the following:

mamba env create --file environment.yml
conda activate lumen
lumen serve dashboard.yaml --show

Stack traceback and/or browser JavaScript console output

Error running application handler <lumen.command.YamlHandler object at 0x7fc7575667f0>: No module named 'hvplot'
File "base.py", line 366, in __init__:
import hvplot.pandas # noqa Traceback (most recent call last):
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/bokeh/application/handlers/code_runner.py", line 197, in run
    exec(self._code, module.__dict__)
  File "dashboard.yaml", line 1, in <module>
    config:
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/dashboard.py", line 166, in __init__
    self._materialize_specification()
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/dashboard.py", line 245, in _materialize_specification
    self.targets[:] = [
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/dashboard.py", line 246, in <listcomp>
    Target.from_spec(
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/target.py", line 391, in from_spec
    return cls(filters=filters, source=source, **params)
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/target.py", line 129, in __init__
    self._update_views()
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/target.py", line 261, in _update_views
    key, card = self._get_card(
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/target.py", line 195, in _get_card
    card, views = None, self._materialize_views(view_filters)
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/target.py", line 176, in _materialize_views
    view = View.from_spec(view_spec, self.source, filters)
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/views/base.py", line 143, in from_spec
    view = view_type(
  File "/Users/msterling/miniconda3/envs/lumen/lib/python3.8/site-packages/lumen/views/base.py", line 366, in __init__
    import hvplot.pandas # noqa
ModuleNotFoundError: No module named 'hvplot'

Solution

After some trial and error, it works fine if I include dask, aiohttp, pyarrow, and hvplot explicitly.

Here's a working environment file.

name: lumen
channels:
  - pyviz
  - conda-forge
dependencies:
  - python=3.8
  - lumen
  - dask
  - aiohttp
  - pyarrow
  - hvplot

yougis commented 3 years ago

you right some dependencies are not define in the conda package. you should also need intake and intake-parquet as soon as you will use other examples (NYC and Precip)

jbednar commented 3 years ago

We could solve this by shipping each example with an associated anaconda-project.yml file that defines the environment, but I'd propose a simpler solution of just moving all the main examples into the same directory and providing one anaconda-project.yml (or environment.yml) file that covers all of them. I don't think such an environment will be unwieldy, and then the directory can be published as live apps on examples.pyviz.org so people can try them out online.

maximlt commented 3 years ago

A list of dependencies I had to install to run the examples:

intake-parquet
aiohttp
datashader
altair
spatialpandas (to make the lasso select tool work)

jbednar commented 3 years ago

My suggestion above still seems worthwhile, but I'd also wonder whether each Lumen .yml file should include its own environment specification in a form that can be given to conda and/or anaconda-project. That way the .yml file will be a complete specification for what's required, and e.g. the Lumen builder could keep track of which optional components are used in a given app so that the .yml spec covers not just how to build it but what needs to be in the environment when doing so.

holoviz / lumen