search

nebari-dev / nebari

🪴 Nebari - your open source data science platform
https://nebari.dev
BSD 3-Clause "New" or "Revised" License
267 stars 88 forks source link

[BUG] - Version 2023.11.1 conda-store using mixed channels #2121

Open rsignell opened 7 months ago

rsignell commented 7 months ago

Describe the bug

I upgraded to Nebari version 2023.11.1 and built a new environment (called pangeo) in the global namespace using conda-store.

I checked versions of bokeh and holoviz after the environment built and was surprised to find a mix of packages from conda-store and pip:

00:48 $ conda env list
# conda environments:
#
global-odc               /home/conda/global/envs/global-odc
global-pangeo         *  /home/conda/global/envs/global-pangeo
nebari-git-dask          /home/conda/nebari-git/envs/nebari-git-dask
base                     /opt/conda
default                  /opt/conda/envs/default

(global-pangeo) rsignell:~ 
00:49 $ conda activate global-pangeo
(global-pangeo) rsignell:~ 
00:49 $ conda list | grep -E "bokeh|nebari-dask|hvplot|geoviews|panel|datashader"
bokeh                     3.3.1                    pypi_0    pypi
datashader                0.16.0                   pypi_0    pypi
geoviews                  1.11.0                   pypi_0    pypi
geoviews-core             1.11.0             pyha770c72_0    conda-forge
hvplot                    0.9.0                    pypi_0    pypi
jupyter-bokeh             3.0.7                    pypi_0    pypi
jupyter-panel-proxy       0.1.0                    pypi_0    pypi
jupyter_bokeh             3.0.7              pyhd8ed1ab_0    conda-forge
nebari-dask               2023.11.1            hd8ed1ab_0    conda-forge
panel                     1.3.1                    pypi_0    pypi

This is despite my specifying only the conda-forge channel in my environment:

channels:
  - conda-forge
dependencies:
  - python=3.10
  - adlfs
  - aiobotocore
  - birdy
  - black
  - bokeh
  - boto3
  - cdsapi
  - cf_xarray
  - cfgrib
  - cfunits
  - climpred
  - coiled
  - curl>7.79.0
  - dask-geopandas
  - nebari-dask=2023.11.1
  - datashader
  - datacube
  - dask-geopandas
  - depfinder
  - distributed
  - earthaccess
  - earthdata
  - erddapy
  - fastparquet
  - flox
  - folium
  - fsspec
  - gcsfs
  - gdal
  - gdptools
  - geocube
  - geogif
  - geolinks
  - geopandas
  - geopy
  - geoviews
  - graphviz
  - h5netcdf
  - h5py
  - hologridgen
  - htop
  - hvplot
  - imagecodecs
  - intake-geopandas
  - intake-stac
  - intake-xarray
  - intake-parquet
  - ipykernel
  - ipyleaflet
  - ipywidgets
  - ipyleaflet
  - ipywidgets
  - isort
  - jinja2
  - jupyter_bokeh
  - jupyter-panel-proxy
  - jupyterlab_code_formatter
  - jupytext
  - jq
  - kerchunk
  - leafmap
  - lxml
  - lz4
  - mamba
  - metpy
  - nbgitpuller
  - nbstripout
  - nco
  - netcdf4 == 1.6.0
  - numba
  - numcodecs
  - odc-algo
  - odc-stac
  - openpyxl
  - osmnx
  - owslib
  - pandas == 1.5.3
  - panel
  - pangeo-forge-recipes
  - papermill
  - param
  - pip
  - pint-xarray
  - planetary-computer
  - pyarrow
  - pyepsg
  - pygeohydro
  - pydaymet
  - pynhd
  - py3dep
  - pygeoogc
  - pygeoutils
  - pynco
  - pyogrio
  - pyvista
  - async_retriever
  - pystac
  - pystac-client
  - python-graphviz
  - python-snappy
  - pyyaml
  - python-gist
  - rasterio
  - rechunker
  - requests
  - rich
  - rio-cogeo
  - rioxarray
  - scikit-image
  - s3fs
  - seawater
  - selenium
  - siphon
  - spatialpandas
  - stackstac
  - toolz
  - ujson
  - unzip
  - utide
  - vim
  - wgrib2
  - xagg
  - xarray-spatial
  - xarray_leaflet
  - xbitinfo-python
  - xesmf
  - xoak
  - xrviz
  - xskillscore
  - zip
  - zstandard
  - xmip
  - intake-esm
  - xarray-datatree
  - h5pyd
  - noaa-coops
  - kbatch
  - xstac
  - zarr
  - jupyter-book
  - ghp-import
  - jsonschema-with-format-nongpl
  - webcolors
  - xarrayutils
  - pip:
      - sciencebasepy
      - sparse @ git+https://github.com/jcmgray/sparse@master

Not sure why this is happening, and I don't think this behavior occured with previous Nebari versions (the ESIP deployment at v2023.10.1 doesn't have this issue)

Expected behavior

Expected behaviour is if only conda-forge channel is specified, packages will be installed from conda-forge, not pip.

OS and architecture in which you are running Nebari

AWS

How to Reproduce the problem?

Create an environment with the above yaml.

Command output

No response

Versions and dependencies used.

Kubenetes 1.26 Nebari 2023.11.1

Compute environment

AWS

Integrations

No response

Anything else?

No response

dharhas commented 6 months ago

This is an ongoing issue. We need to get to the bottom of this.

related - https://github.com/conda-incubator/conda-store/issues/359