holoviz / datashader

Quickly and accurately render even the largest data.
http://datashader.org
BSD 3-Clause "New" or "Revised" License
3.32k stars 366 forks source link

transfer_functions.shade() CUDARuntimeError cuDF 0.12 #879

Closed gumdropsteve closed 4 years ago

gumdropsteve commented 4 years ago

ALL software version info (this library, plus any other relevant software, e.g. bokeh, python, notebook, OS, browser, etc)

Bokeh: 1.4.0 BlazingSQL / cuDF: 0.12 Datashader: 0.10 Python: 3.7

Click here to see environment details

     ***OS Information***
     DISTRIB_ID=Ubuntu
     DISTRIB_RELEASE=18.04
     DISTRIB_CODENAME=bionic
     DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
     NAME="Ubuntu"
     VERSION="18.04.3 LTS (Bionic Beaver)"
     ID=ubuntu
     ID_LIKE=debian
     PRETTY_NAME="Ubuntu 18.04.3 LTS"
     VERSION_ID="18.04"
     HOME_URL="https://www.ubuntu.com/"
     SUPPORT_URL="https://help.ubuntu.com/"
     BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
     PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
     VERSION_CODENAME=bionic
     UBUNTU_CODENAME=bionic
     Linux ip-172-31-3-108 4.15.0-1058-aws #60-Ubuntu SMP Wed Jan 15 22:35:20 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

     ***GPU Information***
     Mon Feb 17 21:59:19 2020
     +-----------------------------------------------------------------------------+
     | NVIDIA-SMI 440.48.02    Driver Version: 440.48.02    CUDA Version: 10.2     |
     |-------------------------------+----------------------+----------------------+
     | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
     | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
     |===============================+======================+======================|
     |   0  Tesla T4            Off  | 00000000:00:1E.0 Off |                    0 |
     | N/A   46C    P0    28W /  70W |  14636MiB / 15109MiB |      0%      Default |
     +-------------------------------+----------------------+----------------------+

     +-----------------------------------------------------------------------------+
     | Processes:                                                       GPU Memory |
     |  GPU       PID   Type   Process name                             Usage      |
     |=============================================================================|
     |    0     17317      C   ...go@blazingdb.com/.miniconda3/bin/python   387MiB |
     |    0     26848      C   ...on@blazingdb.com/.miniconda3/bin/python   395MiB |
     |    0     26907      C   ...on@blazingdb.com/.miniconda3/bin/python   387MiB |
     +-----------------------------------------------------------------------------+

     ***CPU***
     Architecture:        x86_64
     CPU op-mode(s):      32-bit, 64-bit
     Byte Order:          Little Endian
     CPU(s):              4
     On-line CPU(s) list: 0-3
     Thread(s) per core:  2
     Core(s) per socket:  2
     Socket(s):           1
     NUMA node(s):        1
     Vendor ID:           GenuineIntel
     CPU family:          6
     Model:               85
     Model name:          Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
     Stepping:            7
     CPU MHz:             3195.714
     BogoMIPS:            4999.99
     Hypervisor vendor:   KVM
     Virtualization type: full
     L1d cache:           32K
     L1i cache:           32K
     L2 cache:            1024K
     L3 cache:            36608K
     NUMA node0 CPU(s):   0-3
     Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke avx512_vnni

     ***CMake***

     ***g++***
     /usr/bin/g++
     g++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
     Copyright (C) 2017 Free Software Foundation, Inc.
     This is free software; see the source for copying conditions.  There is NO
     warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

     ***nvcc***

     ***Python***
     /home/winston@blazingdb.com/.miniconda3/bin/python
     Python 3.7.4

     ***Environment Variables***
     PATH                            : /home/winston@blazingdb.com/.miniconda3/bin:/home/winston@blazingdb.com/.miniconda3/bin:/home/winston@blazingdb.com/.miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
     LD_LIBRARY_PATH                 :
     NUMBAPRO_NVVM                   :
     NUMBAPRO_LIBDEVICE              :
     CONDA_PREFIX                    : /home/winston@blazingdb.com/.miniconda3
     PYTHON_PATH                     :

     ***conda packages***
     /home/winston@blazingdb.com/.miniconda3/bin/conda
     # packages in environment at /home/winston@blazingdb.com/.miniconda3:
     #
     # Name                    Version                   Build  Channel
     _libgcc_mutex             0.1                        main
     arrow-cpp                 0.15.0           py37h090bef1_2    conda-forge
     asn1crypto                1.2.0                    py37_0
     backcall                  0.1.0                    pypi_0    pypi
     blazingsql                0.12.0          cuda10.0_py37_6    blazingsql/label/cuda10.0
     bokeh                     1.4.0                    py37_0    conda-forge
     boost-cpp                 1.70.0               h8e57a91_2    conda-forge
     brotli                    1.0.7             he1b5a44_1000    conda-forge
     bsql-rapids-thirdparty    0.12.0                        0    blazingsql
     bsql-toolchain            0.12.0                        0    blazingsql
     bsql-toolchain-aws-cpp    0.12.0                        0    blazingsql
     bsql-toolchain-gcp-cpp    0.12.0                        0    blazingsql
     bzip2                     1.0.8                h516909a_2    conda-forge
     c-ares                    1.15.0            h516909a_1001    conda-forge
     ca-certificates           2019.11.28           hecc5488_0    conda-forge
     certifi                   2019.11.28               py37_0    conda-forge
     cffi                      1.13.0           py37h2e261b9_0
     chardet                   3.0.4                 py37_1003
     click                     7.0                        py_0    conda-forge
     cloudpickle               1.3.0                      py_0    conda-forge
     colorcet                  2.0.2                    pypi_0    pypi
     conda                     4.8.2                    py37_0    conda-forge
     conda-package-handling    1.6.0            py37h7b6447c_0
     cppzmq                    4.4.1                hc9558a2_0    conda-forge
     cryptography              2.8              py37h1ba5d50_0
     cudatoolkit               10.0.130                      0    nvidia
     cudf                      0.12.0                   py37_0    rapidsai
     cudnn                     7.6.0                cuda10.0_0    nvidia
     cupy                      7.2.0            py37he57b8b9_0    conda-forge
     curl                      7.68.0               hf8cf82a_0    conda-forge
     cycler                    0.10.0                   pypi_0    pypi
     cyrus-sasl                2.1.27               he38ecfd_0    conda-forge
     cytoolz                   0.10.1           py37h516909a_0    conda-forge
     dask                      2.10.1                     py_0    conda-forge
     dask-core                 2.10.1                     py_0    conda-forge
     dask-cuda                 0.12.0                   py37_0    rapidsai
     dask-cudf                 0.12.0                   py37_0    rapidsai
     datashader                0.10.0                   pypi_0    pypi
     datashape                 0.5.2                    pypi_0    pypi
     decorator                 4.4.1                    pypi_0    pypi
     distributed               2.10.0                     py_0    conda-forge
     dlpack                    0.2                  he1b5a44_1    conda-forge
     double-conversion         3.1.5                he1b5a44_2    conda-forge
     fastavro                  0.22.9           py37h516909a_0    conda-forge
     fastrlock                 0.4             py37he1b5a44_1000    conda-forge
     freetype                  2.10.0               he983fc9_1    conda-forge
     fsspec                    0.6.2                      py_0    conda-forge
     future                    0.18.2                   py37_0    conda-forge
     gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
     gflags                    2.2.2             he1b5a44_1002    conda-forge
     glog                      0.4.0                he1b5a44_1    conda-forge
     grpc-cpp                  1.23.0               h18db393_0    conda-forge
     heapdict                  1.0.1                      py_0    conda-forge
     icu                       64.2                 he1b5a44_1    conda-forge
     idna                      2.8                      py37_0
     imageio                   2.6.1                    pypi_0    pypi
     ipykernel                 5.1.4                    pypi_0    pypi
     ipython                   7.12.0                   pypi_0    pypi
     ipython-genutils          0.2.0                    pypi_0    pypi
     jedi                      0.16.0                   pypi_0    pypi
     jinja2                    2.11.1                     py_0    conda-forge
     jpeg                      9c                h14c3975_1001    conda-forge
     jpype1                    0.7.1            py37h9de70de_0    conda-forge
     jupyter-client            5.3.4                    pypi_0    pypi
     jupyter-core              4.6.2                    pypi_0    pypi
     kiwisolver                1.1.0                    pypi_0    pypi
     krb5                      1.16.4               h173b8e3_0
     libblas                   3.8.0               14_openblas    conda-forge
     libcblas                  3.8.0               14_openblas    conda-forge
     libcudf                   0.12.0               cuda10.0_0    rapidsai
     libcurl                   7.68.0               hda55be3_0    conda-forge
     libedit                   3.1.20181209         hc058e9b_0
     libevent                  2.1.10               h72c5cf5_0    conda-forge
     libffi                    3.2.1                hd88cf55_4
     libgcc-ng                 9.1.0                hdf63c60_0
     libgcrypt                 1.8.4             hf484d3e_1000    conda-forge
     libgfortran-ng            7.3.0                hdf63c60_5    conda-forge
     libgpg-error              1.36                 he1b5a44_0    conda-forge
     libgsasl                  1.8.0             h19a2143_1004    conda-forge
     libhdfs3                  2.3               h311b756_1006    conda-forge
     libiconv                  1.15              h516909a_1005    conda-forge
     liblapack                 3.8.0               14_openblas    conda-forge
     libllvm8                  8.0.1                hc9558a2_0    conda-forge
     libntlm                   1.4               h516909a_1002    conda-forge
     libnvstrings              0.12.0               cuda10.0_0    rapidsai
     libopenblas               0.3.7                h5ec1e0e_6    conda-forge
     libpng                    1.6.37               hed695b0_0    conda-forge
     libprotobuf               3.8.0                h8b12597_0    conda-forge
     librmm                    0.12.0               cuda10.0_0    rapidsai
     libsodium                 1.0.17               h516909a_0    conda-forge
     libssh2                   1.8.2                h22169c7_2    conda-forge
     libstdcxx-ng              9.1.0                hdf63c60_0
     libtiff                   4.1.0                hfc65ed5_0    conda-forge
     libuuid                   2.32.1            h14c3975_1000    conda-forge
     libxml2                   2.9.10               hee79883_0    conda-forge
     llvmlite                  0.31.0           py37h8b12597_0    conda-forge
     locket                    0.2.0                      py_2    conda-forge
     lz4-c                     1.8.3             he1b5a44_1001    conda-forge
     markupsafe                1.1.1            py37h516909a_0    conda-forge
     matplotlib                3.1.3                    pypi_0    pypi
     msgpack-python            0.6.2            py37hc9558a2_0    conda-forge
     multipledispatch          0.6.0                    pypi_0    pypi
     nccl                      2.4.6.1              cuda10.0_0    nvidia
     ncurses                   6.1                  he6710b0_1
     netifaces                 0.10.9          py37h516909a_1000    conda-forge
     networkx                  2.4                      pypi_0    pypi
     numba                     0.48.0           py37hb3f55d8_0    conda-forge
     numpy                     1.18.1           py37h95a1406_0    conda-forge
     nvstrings                 0.12.0                   py37_0    rapidsai
     olefile                   0.46                       py_0    conda-forge
     openjdk                   8.0.192           h516909a_1004    conda-forge
     openssl                   1.1.1d               h516909a_0    conda-forge
     packaging                 20.1                       py_0    conda-forge
     pandas                    0.25.3           py37hb3f55d8_0    conda-forge
     param                     1.9.3                    pypi_0    pypi
     parquet-cpp               1.5.1                         2    conda-forge
     parso                     0.6.1                    pypi_0    pypi
     partd                     1.1.0                      py_0    conda-forge
     pexpect                   4.8.0                    pypi_0    pypi
     pickleshare               0.7.5                    pypi_0    pypi
     pillow                    5.3.0           py37h00a061d_1000    conda-forge
     pip                       19.3.1                   py37_0
     prompt-toolkit            3.0.3                    pypi_0    pypi
     psutil                    5.6.7            py37h516909a_0    conda-forge
     ptyprocess                0.6.0                    pypi_0    pypi
     pyarrow                   0.15.0           py37h8b68381_1    conda-forge
     pycosat                   0.6.3            py37h14c3975_0
     pycparser                 2.19                     py37_0
     pyct                      0.4.6                    pypi_0    pypi
     pygments                  2.5.2                    pypi_0    pypi
     pyhive                    0.6.1                    py37_0
     pynvml                    8.0.4                      py_0    conda-forge
     pyopenssl                 19.0.0                   py37_0
     pyparsing                 2.4.6                      py_0    conda-forge
     pysocks                   1.7.1                    py37_0
     python                    3.7.4                h265db76_1
     python-dateutil           2.8.1                      py_0    conda-forge
     pytz                      2019.3                     py_0    conda-forge
     pywavelets                1.1.1                    pypi_0    pypi
     pyyaml                    5.1.2            py37h516909a_0    conda-forge
     pyzmq                     18.1.1                   pypi_0    pypi
     re2                       2020.01.01           he1b5a44_0    conda-forge
     readline                  7.0                  h7b6447c_5
     requests                  2.22.0                   py37_0
     rmm                       0.12.0                   py37_0    rapidsai
     ruamel_yaml               0.15.46          py37h14c3975_0
     sasl                      0.2.1           py37he1b5a44_1001    conda-forge
     scikit-image              0.16.2                   pypi_0    pypi
     scipy                     1.4.1                    pypi_0    pypi
     setuptools                41.4.0                   py37_0
     six                       1.12.0                   py37_0
     snappy                    1.1.8                he1b5a44_1    conda-forge
     sortedcontainers          2.1.0                      py_0    conda-forge
     sqlalchemy                1.3.13           py37h516909a_0    conda-forge
     sqlite                    3.30.0               h7b6447c_0
     tblib                     1.6.0                      py_0    conda-forge
     thrift                    0.11.0          py37he1b5a44_1001    conda-forge
     thrift-cpp                0.12.0            hf3afdfd_1004    conda-forge
     thrift_sasl               0.3.0           py37h516909a_1001    conda-forge
     tk                        8.6.8                hbc83047_0
     toolz                     0.10.0                     py_0    conda-forge
     tornado                   6.0.3            py37h516909a_4    conda-forge
     tqdm                      4.36.1                     py_0
     traitlets                 4.3.3                    pypi_0    pypi
     uriparser                 0.9.3                he1b5a44_1    conda-forge
     urllib3                   1.24.2                   py37_0
     wcwidth                   0.1.8                    pypi_0    pypi
     wheel                     0.33.6                   py37_0
     xarray                    0.15.0                   pypi_0    pypi
     xz                        5.2.4                h14c3975_4
     yaml                      0.1.7                had09818_2
     zeromq                    4.3.2                he1b5a44_2    conda-forge
     zict                      1.0.0                      py_0    conda-forge
     zlib                      1.2.11               h7b6447c_3
     zstd                      1.4.3                h3b9ef0a_0    conda-forge

Description of expected behavior and the observed behavior

Expected: plotting of drop off coordinates. Observed: CUDARuntimeError: cudaErrorIllegalAddress: an illegal memory access was encountered when passing .Canvas().points() with cuDF or dask_cuDF DataFrame into transfer_functions.shade().

Complete, minimal, self-contained example code that reproduces the issue

import cudf
import datashader as ds
from colorcet import fire
from datashader import transfer_functions as tf

!wget 'https://gist.githubusercontent.com/gumdropsteve/37dc0a12c30fd52d81bdfcff915da04b/raw/9d5854dd9b9dd7c3f28865e308984bf9966376fe/sample_taxi.csv'

gdf = cudf.read_csv('sample_taxi.csv')

agg = ds.Canvas().points(gdf, 'dropoff_longitude', 'dropoff_latitude')

tf.set_background(tf.shade(agg, cmap=fire), "black")

cuDF installed with BlazingSQL:

conda install -c blazingsql/label/cuda10.0 -c blazingsql -c rapidsai -c nvidia -c conda-forge -c defaults blazingsql python=3.7 cudatoolkit=10.0

Stack traceback and/or browser JavaScript console output

Click here to see output of above Python script
--2020-02-17 22:05:56--  https://gist.githubusercontent.com/gumdropsteve/37dc0a12c30fd52d81bdfcff915da04b/raw/9d5854dd9b9dd7c3f28865e308984bf9966376fe/sample_taxi.csv
Resolving gist.githubusercontent.com (gist.githubusercontent.com)... 151.101.248.133
Connecting to gist.githubusercontent.com (gist.githubusercontent.com)|151.101.248.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3810 (3.7K) [text/plain]
Saving to: ‘sample_taxi.csv.3’

sample_taxi.csv.3   100%[===================>]   3.72K  --.-KB/s    in 0s      

2020-02-17 22:05:56 (117 MB/s) - ‘sample_taxi.csv.3’ saved [3810/3810]

---------------------------------------------------------------------------
CUDARuntimeError                          Traceback (most recent call last)
 in 
     10 agg = ds.Canvas().points(gdf, 'dropoff_longitude', 'dropoff_latitude')
     11 
---> 12 tf.set_background(tf.shade(agg, cmap=fire), "black")

~/.miniconda3/lib/python3.7/site-packages/datashader/transfer_functions/__init__.py in shade(agg, cmap, color_key, how, alpha, min_alpha, span, name)
    445 
    446     if agg.ndim == 2:
--> 447         return _interpolate(agg, cmap, how, alpha, span, min_alpha, name)
    448     elif agg.ndim == 3:
    449         return _colorize(agg, color_key, how, min_alpha, name)

~/.miniconda3/lib/python3.7/site-packages/datashader/transfer_functions/__init__.py in _interpolate(agg, cmap, how, alpha, span, min_alpha, name)
    266     with np.errstate(invalid="ignore", divide="ignore"):
    267         # Transform data (log, eq_hist, etc.)
--> 268         data = interpolater(data, mask)
    269 
    270         # Transform span

~/.miniconda3/lib/python3.7/site-packages/datashader/transfer_functions/__init__.py in eq_hist(data, mask, nbins)
    164     data2 = data if mask is None else data[~mask]
    165     if data2.dtype == bool or np.issubdtype(data2.dtype, np.integer):
--> 166         hist = np.bincount(data2.ravel())
    167         bin_centers = np.arange(len(hist))
    168         idx = int(np.nonzero(hist)[0][0])

<__array_function__ internals> in bincount(*args, **kwargs)

cupy/core/core.pyx in cupy.core.core.ndarray.__array_function__()

~/.miniconda3/lib/python3.7/site-packages/cupy/statistics/histogram.py in bincount(x, weights, minlength)
    133             raise ValueError('minlength must be non-negative')
    134 
--> 135     size = int(cupy.max(x)) + 1
    136     if minlength is not None:
    137         size = max(size, minlength)

cupy/core/core.pyx in cupy.core.core.ndarray.__int__()

cupy/core/core.pyx in cupy.core.core.ndarray.get()

cupy/core/core.pyx in cupy.core.core.ndarray.get()

cupy/cuda/memory.pyx in cupy.cuda.memory.MemoryPointer.copy_to_host()

cupy/cuda/runtime.pyx in cupy.cuda.runtime.memcpy()

cupy/cuda/runtime.pyx in cupy.cuda.runtime.check_status()

CUDARuntimeError: cudaErrorIllegalAddress: an illegal memory access was encountered

Screenshots or screencasts of the bug in action

image image

Additional info

Installed via pip after failing to install via conda

Click here to see Anaconda Error Report
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/cli/install.py", line 265, in install
        should_retry_solve=(_should_retry_unfrozen or repodata_fn != repodata_fns[-1]),
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
        should_retry_solve)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 158, in solve_for_diff
        force_remove, should_retry_solve)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 281, in solve_final_state
        ssc = self._run_sat(ssc)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 808, in _run_sat
        should_retry_solve=ssc.should_retry_solve
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 1316, in solve
        raise UnsatisfiableError({})
    conda.exceptions.UnsatisfiableError: 
    Did not find conflicting dependencies. If you would like to know which
    packages conflict ensure that you have enabled unsatisfiable hints.

    conda config --set unsatisfiable_hints True

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/cli/main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 82, in do_call
        return getattr(module, func_name)(args, parser)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/cli/main_install.py", line 20, in execute
        install(args, parser, 'install')
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/cli/install.py", line 299, in install
        should_retry_solve=(repodata_fn != repodata_fns[-1]),
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 117, in solve_for_transaction
        should_retry_solve)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 158, in solve_for_diff
        force_remove, should_retry_solve)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 281, in solve_final_state
        ssc = self._run_sat(ssc)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/core/solve.py", line 808, in _run_sat
        should_retry_solve=ssc.should_retry_solve
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/common/io.py", line 88, in decorated
        return f(*args, **kwds)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 1318, in solve
        self.find_conflicts(specs, specs_to_add, history_specs)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 347, in find_conflicts
        bad_deps = self.build_conflict_map(specs, specs_to_add, history_specs)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 507, in build_conflict_map
        root, search_node, dep_graph, num_occurances)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/resolve.py", line 369, in breadth_first_search_for_dep_graph
        last_spec = MatchSpec.union((path[-1], target_paths[-1][-1]))[0]
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/models/match_spec.py", line 481, in union
        return cls.merge(match_specs, union=True)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/models/match_spec.py", line 475, in merge
        reduce(lambda x, y: x._merge(y, union), group) if len(group) > 1 else group[0]
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/models/match_spec.py", line 475, in 
        reduce(lambda x, y: x._merge(y, union), group) if len(group) > 1 else group[0]
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/models/match_spec.py", line 502, in _merge
        final = this_component.union(that_component)
      File "/home/winston@blazingdb.com/.miniconda3/lib/python3.7/site-packages/conda/models/match_spec.py", line 764, in union
        return '|'.join(options)
    TypeError: sequence item 0: expected str instance, Channel found

`$ /home/winston@blazingdb.com/.miniconda3/bin/conda install datashader`

  environment variables:
                 CIO_TEST=
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/home/winston@blazingdb.com/.miniconda3/bin/conda
             CONDA_PREFIX=/home/winston@blazingdb.com/.miniconda3
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/home/winston@blazingdb.com/.miniconda3/bin/python
               CONDA_ROOT=/home/winston@blazingdb.com/.miniconda3
              CONDA_SHLVL=1
     JAVA_LD_LIBRARY_PATH=/home/winston@blazingdb.com/.miniconda3/jre/lib/amd64/server
                     PATH=/home/winston@blazingdb.com/.miniconda3/bin:/home/winston@blazingdb.co
                          m/.miniconda3/bin:/home/winston@blazingdb.com/.miniconda3/condabin:/us
                          r/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
       REQUESTS_CA_BUNDLE=
            SSL_CERT_FILE=

     active environment : base
    active env location : /home/winston@blazingdb.com/.miniconda3
            shell level : 1
       user config file : /home/winston@blazingdb.com/.condarc
 populated config files : 
          conda version : 4.8.2
    conda-build version : not installed
         python version : 3.7.4.final.0
       virtual packages : __cuda=10.2
                          __glibc=2.27
       base environment : /home/winston@blazingdb.com/.miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/winston@blazingdb.com/.miniconda3/pkgs
                          /home/winston@blazingdb.com/.conda/pkgs
       envs directories : /home/winston@blazingdb.com/.miniconda3/envs
                          /home/winston@blazingdb.com/.conda/envs
               platform : linux-64
             user-agent : conda/4.8.2 requests/2.22.0 CPython/3.7.4 Linux/4.15.0-1058-aws ubuntu/18.04.3 glibc/2.27
                UID:GID : 1027:1000
             netrc file : None
           offline mode : False
philippjfr commented 4 years ago

Tested with all version of cuDF from 0.10 - 0.15 (nightlies) without issue. Appears to be fixed, please reopen if the error reoccurs.