[documentation/installation] Larch dependencies fail to install with pip -> uniform conda installs over scripts and workflows

[maurov]

I create this issue to uniform the way dependencies are installed with conda, as they fail with pip. This is a recurrent problem found by the users (e.g. here).

The installation scripts and the github workflows files work fine, but have the dependencies hard-coded in the conda install command. I propose to use the requirements.txt file or an environment.yml file.

Furthermore, we should update the documentation in order to explain how to install Larch for those users who have already Anaconda/miniconda distribution installed on their system.

@newville if you agree with that, I can propose a pull request.

[newville]

@maurov Yes! Any attempt to clean up or clarify would be great!

[maurov]

@newville ideally, we would like to have pip install xraylarch working, but it is a matter of fact that some required dependencies fails with pip, then conda is required. It seems to me that the installation scripts work nicely under Windows, Linux and Mac, while the binary installer for Linux fails (did not investigate yet).

Since we have to go via conda anyway, what about having xraylarch on conda-forge directly?

Looking at the installation scripts and the github workflows, an environment.yml file could be:

name: xraylarch
channels:
    - defaults
    - conda-forge
dependencies:
    - python == 3.8 #easier to stick to a given python version
    - pip
    - numpy => 1.20
    - scipy => 1.5
    - matplotlib => 3.0
    - scikit-learn
    - scikit-image
    - pandas
    - pyparsing
    - pytest
    - pytest-cov
    - coverage
    - h5py >= 2.10
    - pillow >= 8.3.2
    - imageio
    - sqlalchemy
    - psutil
    - pyyaml
    - psycopg2-binary
    - numdifftools
    - emcee
    - cython
    - tomopy
    - wxpython
    - pymatgen
    - pycifrw
    #- pyqt >= 5.9
    #- conda-forge::pyqtwebengine
    #- qtconsole
    - pip:
        - -r requirements.txt
              #where requirements.txt is:
              #lmfit
              #peakutils
              #pyepics
              #pyshortcuts
              #termcolor
              #sphinx
              #dill
              #xraydb
              #wxmplot
              #wxutils
              #fabio
              #silx>=1.0.0
              #imageio
        - xraylarch

[newville]

@maurov sorry for the delay. I think that a conda-forge package is a decent idea, but I also think it might be some work. I'm not opposed. I've supported a lot of conda package in the past, but PyPI seems more versatile.

I sort of think that moving to pyproject.toml and thinking about system-dependent wheels might be the right "next step". But binary packages for wxPython and pymatgen on PyPI is a completely separate matter....

[maurov]

@newville OK for not going with a conda package on conda forge. I am not familiar with pyproject.toml but I am afraid it will not solve the problem with dependencies that are not pip installable.

I gave few more trials under Windows and Linux/Ubuntu (I do not have access to Mac) with pip and I confirm that wxpython and pymatgen give errors and should be installed via conda. Thus, I suggest moving pymatgen to requirements-extra.txt. This should avoid pip install xraylarch failing, even if some functionalities will not be available.

I propose to update the documentation for those users how have already conda installed and do not want to (re)install conda, as is done if they use the installation scripts:

1. activate your conda environment (base by default) and update it:

   conda activate
   conda update -y conda python pip

2. (optional) create a dedicated environment for Larch and activate it:

   conda create -y --name xraylarch python==3.8
   conda activate xraylarch

3. install main dependencies:

   conda install -y "numpy=>1.20" "scipy=>1.5" "matplotlib=>3.0" scikit-learn pandas
   conda install -y -c conda-forge wxpython
   conda install -y -c conda-forge tomopy
   conda install -y -c conda-forge pymatgen

4. install Larch (latest release): pip install xraylarch

5. (optional) install Larch (development version): pip install git+https://github.com/xraypy/xraylarch.git

6. if anything of above fails, report it here