Conda-Forge packages

[shyamd]

enumLib is important enough for a number of different algorithms these days. Any chance of getting a conda-forge package that is compiled for Mac, Linux, and Windows? This would make it much easier for novices to get access to it. Maybe @jan-janssen wouldn't mind helping?

Note: I don't think the matsci conda channel is being maintained anymore.

[glwhart]

I'd love to do this. It's not quite in my wheelhouse. I'd love some coaching. And I have no idea if I could actually get it working for windows...I'm so unfamiliar with it...

I'm planning a complete re-write of enumlib without using fortran so we can avoid compilation issues. Native python is too slow but Julia is REALLY fast and can be called directly from python. I've got a small testcase of the symmetry stuff working and it is very impressive. Fast and very easy to maintain...

I'm planning a number of algorithmic improvements, and some have been discussed by others in the literature. A massive speed-up is possible in some cases.

[jan-janssen]

I tried to create conda packages for Linux and Mac Os for both symlib and enumlib. symlib works but currently I struggle with enumlib as sorting.f90 requires num_types.mod which is part of symlib. https://github.com/conda-forge/staged-recipes/pull/15613

[shyamd]

Are you compiling the symlib mdoule in the enumlib install. I don't think installing it as a library works. I'll see if I can't find the original recipe for whats on matsci.

[shyamd]

Here is the build script @shyuep used for the matsci channel. I think the easiest way to do this might not be to try and package symlib separately, but just incorporate it into enumlib. If something else needs it, I imagine the build system is such that it needs a local source compile as well. The other issue might be if conda is not doing a recursive pull so that symlib is pulled down when building enumlib.

#!/bin/bash

BINARY=enum
BINARY_HOME=$PREFIX/bin

git clone --recursive https://github.com/msg-byu/enumlib.git

export F90=gfortran

# Make symlib
cd $SRC_DIR/enumlib/symlib/src
make

# Make polya
cd $SRC_DIR/enumlib/polya/fortran
make

# Make enum
cd $SRC_DIR/enumlib/src

make
make enum.x
make makestr.x

cp enum.x $BINARY_HOME
cp makestr.x $BINARY_HOME

# Cleanup.
rm -rf $SRC_DIR/enumlib

[jan-janssen]

Done - the package for Linux and Mac Os should be available for download in the next few minutes. If any of you want to join me in maintaining the conda package, I would be happy to add you to the conda recipe.

[shyamd]

Sounds good. I'll spend a little bit of time over the next few weeks figuring out how to do build the windows version and PR that!

[glwhart]

Do I need to do anything in the repo? Should one of you update the README and do a PR for that?

[CompRhys]

@jan-janssen I tried the linux conda-forge package and after doing so it doesn't find makestr.x or equivalents in my path after installing. This util is needed for EnumerateStructureTransformation in pymatgen. is makestr.x in your recipe? and if not could you add it?

[jan-janssen]

I just added it to the conda package - it should be available within the next hour https://github.com/conda-forge/enumlib-feedstock/pull/1

[glwhart]

Coming back from being dean and teaching again, I've been too busy to keep up with all my github repos. At the end of April, I hope to start addressing all the outstanding issues and PRs

[shyuep]

Just a note here - yes the matsci channel was always a temporary thing. Ideally, if everything is in conda-forge, there is no reason for us to maintain a separate channel.

That said, maintenance of matsci was not the main problem here. The main problem is maintaining a specific package on conda-forge. Unlike Python distributions (e.g., pymatgen) where a new release version automatically triggers a conda-forge rebuild, this does not happen for non-Python packages like enumlib as far as I know (let me know if I am wrong on this). So this requires the enumlib developers to manually trigger a conda-forge build each time there is a release. I would say this is a serious hassle. In fact, if before the age of automatic rebuilds in conda-forge, updating the pymatgen package in conda-forge was sporadic also. Manual processes are bad unless the maintainer has infinite time and patience and discipline (and I know I don't have it).

[jan-janssen]

To my knowledge new tagged releases here on Github trigger a new pull request for the conda forge package.

[shyuep]

Ok cool. If that is the case, then it just requires the maintainers to regularly release Github versions. Note that the last version was from 2019. I am semi-sure that there has been a number of changes and bug fixes since 2019.

[glwhart]

I'm trying to tag releases after fixing bugs. Hopefully that will help with these issues. Closing this until someone tells me there are problems keeping the conda-forge package up to date. Thanks for your help/discussion.

[jan-janssen]

The new version fails to build for conda-forge with the following error message:

$BUILD_PREFIX/bin/x86_64-conda-linux-gnu-gfortran -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem $PREFIX/include -fdebug-prefix-map=$SRC_DIR=/usr/local/src/conda/enumlib-2.0.5 -fdebug-prefix-map=$PREFIX=/usr/local/src/conda-prefix -ffree-line-length-none -I$SRC_DIR/symlib/src -c ../aux_src/makeStr.f90
../aux_src/makeStr.f90:9:19:

    9 | use io_utils, only: co_ca
      |                   1
Error: Symbol 'co_ca' referenced at (1) not found in module 'io_utils'
make: *** [Makefile:76: makeStr.o] Error 1

[jan-janssen]

The new version 2.0.6 worked fine - so I released the update for conda-forge.