[BUG] Fresh install only gives version 0.20.0 instead of 0.21.0

[nydaniel]

A fresh install of Darts with conda conda install -y -c conda-forge -c pytorch u8darts-all results only in version 0.20.0 instead of 0.21.0

print(darts.__version__) -> '0.20.0'

also the new data set is not available

from darts.datasets import AustralianTourismDataset -> ImportError: cannot import name 'AustralianTourismDataset' from 'darts.datasets' (/opt/conda/lib/python3.8/site-packages/darts/datasets/__init__.py)

Python 3.8.12 Conda 4.13.0

[hrzn]

This is strange. Could you try creating again a fresh conda environment, running conda config --set channel_priority strict, and then again conda install -y -c conda-forge -c pytorch u8darts-all ? We have observed that sometimes conda gets stuck solving the environment for 0.21.0 (for some yet unknown reason), and setting the channel priority seems to solve this. We have tested the conda install of 0.21.0 for both Python 3.7 and 3.9. Please let us know if this still doesn't work for you.

[nydaniel]

I set the priority to false instead of strict but I guess that's the same. It's really a problem and the conda installation takes forever. Hope you can find the bug.

[hrzn]

@tomasvanpottelbergh Would you maybe have an idea regarding what could be wrong with conda here? I have also observed this issue (which in my case was solved running conda config --set channel_priority strict), but I'd like to understand what's going on.

[nydaniel]

Hi guys, we've really tried to get this to work but can't succeed. For our docker image we used the following command conda create -n darts -c conda-forge --strict-channel-priority -v python==3.9.10 numpy pandas scikit-learn matplotlib pyhive lightgbm pytorch u8darts-all This installs version 0.18.0 of darts. I hope you are able to settle this issue. Thanks.

[nydaniel]

Here are some error messages that maybe help you

conda install -c conda-forge -c pytorch --strict-channel-priority -v u8darts-all==0.21.0

`Output in format: Requested package -> Available versionsThe following specifications were found to be incompatible with your system:

• feature:/linux-64::__cuda==11.6=0
• feature:/linux-64::__glibc==2.27=0
• feature:|@/linux-64::__cuda==11.6=0
• u8darts-all==0.21.0 -> pytorch[version='>=1.8.0'] -> __cuda
• u8darts-all==0.21.0 -> pytorch[version='>=1.8.0'] -> __glibc[version='>=2.17|>=2.17,<3.0.a0']

Your installed version is: 2.27

Note that strict channel priority may have removed packages required for satisfiability.

(darts) root@darts:/workspace# python Python 3.9.10 | packaged by conda-forge | (main, Feb 1 2022, 21:24:11) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import darts darts.version '0.18.0' quit() `

[tomasvanpottelbergh]

Strange, but unfortunately I'm not able to pinpoint the cause exactly. As your logs show, it might be a dependency conflict between certain Python and PyTorch versions on glibc? On my machine conda install -c pytorch -c conda-forge u8darts-all==0.21.0 python=3.9 fails, but the same command works with Python 3.8 and 3.10.

Since it seems to depend on system libraries, could you try with another Python version? If you give me the Docker image and tag I can also try to reproduce the issue there.

[nydaniel]

Hi Tomas,

here is the docker file. I hope it helps to figure out the problem because we're really not able to install the latest version.

Thanks a lot.

`FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-devel RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub RUN apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64/7fa2af80.pub

install basic commands

RUN apt-get update && \ apt-get install -y wget && \ apt-get install -y vim && \ apt-get install -y git && \ apt-get install -y htop && \ apt-get install -y curl && \ apt-get install -y libprotobuf-dev && \ apt-get install -y protobuf-compiler && \ apt-get install -y cmake

install openssh-server

RUN echo 'tzdata tzdata/Areas select Europe' | debconf-set-selections && \ echo 'tzdata tzdata/Zone/Europe select Berlin' | debconf-set-selections && \ DEBIAN_FRONTEND="noninteractive" apt install -y tzdata && \ apt-get install -y openssh-server && \ rm -rf /var/lib/apt/lists/*

SSH

COPY /authorized_keys /root/.ssh/authorized_keys RUN sed -ri 's/^#?PermitRootLogin\s+./PermitRootLogin yes/' /etc/ssh/sshd_config RUN mkdir /var/run/sshd RUN apt-get clean && rm -rf /var/lib/apt/lists/ /tmp/ /var/tmp/

install DS libraries

RUN conda install -y -c conda-forge -c pytorch --strict-channel-priority -v u8darts-all==0.21.0 python==3.10`

[tomasvanpottelbergh]

I just tested on your Docker base image (pytorch/pytorch:1.11.0-cuda11.3-cudnn8-devel) and the command conda install -c conda-forge u8darts=0.21 seems to work, while u8darts-torch and u8darts-all take longer than I can wait for.

The problem is almost certainly in one of the dependencies, but the challenge is figuring out which one. This short test would point to pytorch-lightning, which is weird, because it doesn't seem to have very strict dependencies.

Any help in figuring out the blocking dependency is welcome! For now, would it help for you to use u8darts instead of u8darts-all and manually add the dependencies you need?

[nydaniel]

Thanks a lot for looking into this. We've now changed to using pip instead of conda. This is working without problems now. With conda we had problems with darts version and/or python version.

[hrzn]

Thanks for looking into it @tomasvanpottelbergh !

For the last release (0.21.0) we changed the core dependencies as follows: So I would suspect one of these (maybe Prophet?), although the issue seems to be with u8darts-all, which anyway had most of these dependencies too before. In addition, not shown in this diff, we also fixed statsforecast==0.6.0 before releasing, so that could also be a lead.

[tomasvanpottelbergh]

I tried relaxing the pytorch-lightning requirements (see https://github.com/conda-forge/pytorch-lightning-feedstock/pull/97), but that doesn't seem to help. Maybe the statsforecast pin is the culprit? I will investigate again after the next release, as I see the pin has been removed.

[hrzn]

OK, sounds good 👍 Many thanks @tomasvanpottelbergh :)

[egiacomazzi]

I also have problems with installing darts. conda install -c conda-forge -c pytorch u8darts-all tries to solve the environment now for longer than 24h. So I tried pip in the meantime and first installed pytorch: pip3 install torch torchvision torchaudio and then darts with pip install darts. That results in the following error:

  Using cached darts-0.21.0-py3-none-any.whl (424 kB)
Collecting pandas>=1.0.5
  Using cached pandas-1.4.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
Collecting joblib>=0.16.0
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting xarray>=0.17.0
  Using cached xarray-2022.6.0-py3-none-any.whl (927 kB)
Collecting statsmodels>=0.13.0
  Using cached statsmodels-0.13.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.9 MB)
Collecting matplotlib>=3.3.0
  Using cached matplotlib-3.5.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (11.3 MB)
Collecting holidays>=0.11.1
  Using cached holidays-0.15-py3-none-any.whl (181 kB)
Requirement already satisfied: requests>=2.22.0 in ./lib/python3.8/site-packages (from darts) (2.28.1)
Collecting tbats>=1.1.0
  Using cached tbats-1.1.0-py3-none-any.whl (43 kB)
Collecting scipy>=1.3.2
  Using cached scipy-1.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.4 MB)
Requirement already satisfied: torch>=1.8.0 in ./lib/python3.8/site-packages (from darts) (1.12.1)
Collecting lightgbm>=2.2.3
  Using cached lightgbm-3.3.2-py3-none-manylinux1_x86_64.whl (2.0 MB)
Collecting nfoursid>=1.0.0
  Using cached nfoursid-1.0.1-py3-none-any.whl (16 kB)
Collecting scikit-learn>=1.0.1
  Using cached scikit_learn-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (31.2 MB)
Collecting statsforecast==0.6.0
  Using cached statsforecast-0.6.0-py3-none-any.whl (44 kB)
Collecting tqdm>=4.60.0
  Using cached tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
Collecting catboost>=1.0.6
  Using cached catboost-1.0.6-cp38-none-manylinux1_x86_64.whl (76.6 MB)
Requirement already satisfied: numpy>=1.19.0 in ./lib/python3.8/site-packages (from darts) (1.22.3)
Collecting prophet>=1.1
  Using cached prophet-1.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.9 MB)
Collecting ipython>=5.0.0
  Using cached ipython-8.5.0-py3-none-any.whl (752 kB)
Collecting pytorch-lightning>=1.5.0
  Downloading pytorch_lightning-1.7.6-py3-none-any.whl (707 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 707.5/707.5 kB 9.0 MB/s eta 0:00:00
Collecting pmdarima>=1.8.0
  Using cached pmdarima-2.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (1.9 MB)
Collecting numba
  Using cached numba-0.56.2-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.5 MB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting graphviz
  Using cached graphviz-0.20.1-py3-none-any.whl (47 kB)
Collecting plotly
  Using cached plotly-5.10.0-py2.py3-none-any.whl (15.2 MB)
Collecting korean-lunar-calendar
  Using cached korean_lunar_calendar-0.2.1-py3-none-any.whl (8.0 kB)
Collecting convertdate>=2.3.0
  Using cached convertdate-2.4.0-py3-none-any.whl (47 kB)
Collecting hijri-converter
  Using cached hijri_converter-2.2.4-py3-none-any.whl (14 kB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting prompt-toolkit<3.1.0,>3.0.1
  Using cached prompt_toolkit-3.0.31-py3-none-any.whl (382 kB)
Collecting stack-data
  Using cached stack_data-0.5.0-py3-none-any.whl (24 kB)
Collecting traitlets>=5
  Using cached traitlets-5.4.0-py3-none-any.whl (107 kB)
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting decorator
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting pygments>=2.4.0
  Using cached Pygments-2.13.0-py3-none-any.whl (1.1 MB)
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting jedi>=0.16
  Using cached jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)
Collecting pexpect>4.3
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting matplotlib-inline
  Using cached matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
Requirement already satisfied: wheel in ./lib/python3.8/site-packages (from lightgbm>=2.2.3->darts) (0.37.1)
Collecting cycler>=0.10
  Using cached cycler-0.11.0-py3-none-any.whl (6.4 kB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.4.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.2 MB)
Collecting pyparsing>=2.2.1
  Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting packaging>=20.0
  Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting fonttools>=4.22.0
  Using cached fonttools-4.37.1-py3-none-any.whl (957 kB)
Requirement already satisfied: pillow>=6.2.0 in ./lib/python3.8/site-packages (from matplotlib>=3.3.0->darts) (9.2.0)
Collecting pytz>=2020.1
  Using cached pytz-2022.2.1-py2.py3-none-any.whl (500 kB)
Requirement already satisfied: urllib3 in ./lib/python3.8/site-packages (from pmdarima>=1.8.0->darts) (1.26.12)                                                                     
Requirement already satisfied: Cython!=0.29.18,!=0.29.31,>=0.29 in ./lib/python3.8/site-packages (from pmdarima>=1.8.0->darts) (0.29.28)                                            
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in ./lib/python3.8/site-packages (from pmdarima>=1.8.0->darts) (41.2.0)                                                  
Collecting LunarCalendar>=0.0.9
  Using cached LunarCalendar-0.0.9-py2.py3-none-any.whl (18 kB)
Collecting setuptools-git>=1.2
  Using cached setuptools_git-1.2-py2.py3-none-any.whl (10 kB)
Collecting cmdstanpy>=1.0.4
  Using cached cmdstanpy-1.0.7-py3-none-any.whl (80 kB)
Collecting setuptools!=50.0.0,>=38.6.0
  Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
Collecting PyYAML>=5.4
  Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)                                                  
Collecting pyDeprecate>=0.3.1
  Using cached pyDeprecate-0.3.2-py3-none-any.whl (10 kB)
Collecting tensorboard>=2.9.1
  Using cached tensorboard-2.10.0-py3-none-any.whl (5.9 MB)
Collecting fsspec[http]!=2021.06.0,>=2021.05.0
  Using cached fsspec-2022.8.2-py3-none-any.whl (140 kB)
Requirement already satisfied: typing-extensions>=4.0.0 in ./lib/python3.8/site-packages (from pytorch-lightning>=1.5.0->darts) (4.3.0)                                             
Collecting torchmetrics>=0.7.0
  Using cached torchmetrics-0.9.3-py3-none-any.whl (419 kB)
Requirement already satisfied: charset-normalizer<3,>=2 in ./lib/python3.8/site-packages (from requests>=2.22.0->darts) (2.1.1)                                                     
Requirement already satisfied: idna<4,>=2.5 in ./lib/python3.8/site-packages (from requests>=2.22.0->darts) (3.4)                                                                   
Requirement already satisfied: certifi>=2017.4.17 in ./lib/python3.8/site-packages (from requests>=2.22.0->darts) (2022.9.14)                                                       
Collecting threadpoolctl>=2.0.0
  Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting patsy>=0.5.2
  Using cached patsy-0.5.2-py2.py3-none-any.whl (233 kB)
Collecting ujson
  Using cached ujson-5.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (45 kB)                                                                                         
Collecting pymeeus<=1,>=0.3.13
  Using cached PyMeeus-0.5.11.tar.gz (5.4 MB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [1 lines of output]
      ERROR: Can not execute `setup.py` since setuptools is not available in the build environment.                                                                                 
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I work in an environment with Python 3.8.3 and pip list tells me that I have setuptools 41.2.0 installed in it.

I would highly appreciate some help :)

[tomasvanpottelbergh]

Sorry, but I have no idea what could be wrong, apart from something in the PyMeeus package as pip suggests. Hopefully the next release of Darts with looser requirements will make the conda installation work.

[hrzn]

@nydaniel @egiacomazzi Could you try again? 0.22.0 has been released on conda forge today, and I've just made a successful test installing with conda. Would be interested to hear if it also works for you.

[nydaniel]

Looks like it's still not installing. Sorry :-(

[madtoinou]

Last comments of #1217 might be related to this one, the problem seems to persist for release 0.23.1...