search

bastibe / python-soundfile

SoundFile is an audio library based on libsndfile, CFFI, and NumPy
BSD 3-Clause "New" or "Revised" License
711 stars 108 forks source link

Use conda-forge CI builds #219

Open stefan-balke opened 6 years ago

stefan-balke commented 6 years ago

Hi,

there is a conda-forge recipe for this package: https://github.com/conda-forge/libsndfile-feedstock

The only missing part is the windows build at the moment, but there exists some existing solutions, see this issue:

https://github.com/conda-forge/libsndfile-feedstock/issues/4

Once this is set-up, the new GitHub releases could be easily integrated...

What do you think?

bastibe commented 6 years ago

It would be great to be able to remove the dependency on our libsndfile-binaries repo. However, as far as I can tell, the conda-forge repo creates a dynamically-linked binary, which requires additional libraries to be installed.

The goal of providing our own binaries is to not rely on separately-installed binaries, which is why we link everything statically. I'm sure this could be integrated into conda-forge somehow, though.

mgeier commented 6 years ago

I wouldn't remove our pre-packaged binaries, because not everyone uses conda.

Regarding conda packages, those are already available for conda:

So I think the libsndfile DLL from conda-forge can indeed just dynamically link those, which it is already supposed to do, see: https://github.com/conda-forge/libsndfile-feedstock/blob/6c3ce6d399a74b379e84727ec9eadef640fdf23c/recipe/meta.yaml#L18-L27.

Note that there is a question that talks about removing those dependencies (https://github.com/conda-forge/libsndfile-feedstock/issues/2), which would be bad.

bastibe commented 6 years ago

Were you proposing to use the conda-forge binaries in our wheels or as a dependency for our code?

mgeier commented 6 years ago

Were you proposing to use the conda-forge binaries in our wheels or as a dependency for our code?

Neither. Sorry that I was unclear about that. I was mainly addressing @stefan-balke regarding his suggestions.

I'm proposing to keep our wheels unchanged and I'm not proposing to add any dependencies to our code. Except probably adding another check with _find_library() if that's necessary. I've just created a PR for the sounddevice module with such a check: https://github.com/spatialaudio/python-sounddevice/pull/122.

Then, somebody (@stefan-balke for example?) could create a binary conda-forge package with a libsndfile DLL which should be available at https://anaconda.org/conda-forge/libsndfile afterwards. And an upgrade to version 1.0.28 would be good, too. And after that, https://anaconda.org/conda-forge/pysoundfile should finally work and it would be possible to install the soundfile module on Windows with

conda install -c conda-forge pysoundfile

Also, somebody should probably rename the conda-forge package, probably to python-soundfile or something.

stefan-balke commented 6 years ago

Hey,

thanks for the insights. I was just wondering if it was handy to have such an automatic CI but your points are valid and I haven't looked that deep into it! I will have an eye on https://anaconda.org/conda-forge/libsndfile and might find the time for fiddling around with the Windows build. That could be a way at least for conda users.

mgeier commented 6 years ago

@stefan-balke Sure it would also be possible to use some automatic CI process to create the DLLs that are part of the official wheels.

But you should be aware that libsndfile releases happen quite rarely. In the last 3 years, there have been two updates: https://github.com/bastibe/libsndfile-binaries/commits/master.

OTOH, an automatic generation of the wheel packages would be more interesting (because it will most likely happen more often). There were some discussions here: #207. If you want to help, please chime in!

If it turns out that it makes sense to re-create the DLLs with each (automated) wheel build, that's of course also fine. But then the dependencies (OGG, FLAC) should be included statically, as @bastibe mentioned.

HaHeho commented 5 years ago

I would like to install the latest version rather from conda instead of PyPI. If I do conda install pysoundfile -c conda-forge I'll successfully get to install version conda-forge/noarch::pysoundfile-0.10.1-py_0. That got me suspicious since version 0.10.2 is available on conda-forge, right?

When trying to enforce the version I get the following:

$ conda install pysoundfile=0.10.2 -c conda-forge
Collecting package metadata: done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - pysoundfile=0.10.2 -> libgcc-ng[version='>=7.3.0']

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

Regarding other packages I've found discussion around similar error, which were a mistake in the build and ended up in getting rid of the unneeded GCC dependency eventually. What is happening in my case (mind using OSX)? Is this the error happening in a different place since right now libgcc-ng is not available for me from conda even though it maybe should?

mgeier commented 5 years ago

No compiler should be needed to build the soundfile module.

I think this dependency should be removed from the conda package:

https://github.com/conda-forge/pysoundfile-feedstock/blob/418115e543e0a14e3c698dbf6f8e3ba088275851/recipe/meta.yaml#L22

It was added here without comment: https://github.com/conda-forge/pysoundfile-feedstock/pull/3

@HaHeho Would you mind creating an issue/PR over there?

HaHeho commented 5 years ago

@mgeier I have no knowledge about how conda-forge works so far. I would not know how to reference or even where to create the issue.

So, to whoever will be able to do it, please reference here so I can look into it and try to understand! :)

mgeier commented 5 years ago

I don't really know how conda-forge works, either. I just had a hunch that removing the line mentioned above could probably help. I've created a PR:

https://github.com/conda-forge/pysoundfile-feedstock/pull/5

According to the test logs, the package libgcc-ng still seems to be installed. So probably this doesn't fix the problem.

Did you try to install libsndfile on its own?

HaHeho commented 5 years ago

Did you try to install libsndfile on its own?

Did not try separately before. But doing that seems to work without problems and installs libsndfile conda-forge/osx-64::libsndfile-1.0.28-h0a44026_1000

That is the same version as I've got before. So it does not change anything when trying to install pysoundfile=0.10.2. Still same error there. Was that expected to change?

mgeier commented 5 years ago

It looks like a new package is available: https://anaconda.org/conda-forge/pysoundfile/files

Could you please try it?

HaHeho commented 5 years ago

YES, works perfectly now!