A new release for soundfile

[bastibe]

This is a continuation of the discussion in #310, where I'm working on packaging a new version of soundfile.

It appears that the packaging problem is more complicated than anticipated. Our existing build system will probably need to be replaced, which will take some time.

If anyone would like to help me package soundfile, I'd be grateful!

The problem is, we need to package four wheels for

1. Windows 32 Bit
2. Windows 64 Bit
3. macOS Intel
4. macOS Arm

each containing their own libsndfile{.dll,.dylib}, and making sure that the right version gets installed on the right platform.

We have a build script in place for building packages for 1-3, but I don't yet know how to build wheels specifically for macOS Arm, and it seems that we should convert our build system to use PEP 517 (pyproject.toml/build et al).

I only had an hour or so today to work on this, so there wasn't much progress. I'll keep you updated.

[chuma9615]

Thanks for taking time to solve this, If u need any helping hands let me know

[HexDecimal]

pyproject.toml is super nice but I don't think it will resolve your current issues with wheels. What it can do is make it so that cffi is installed before the setup.py script is run which would crash otherwise.

I'm not sure about the specific CLI flags. I think it's adding -arch arm64 to the C/link flags will work it if the tools support it. Or -arch arm64 -arch x86_64 for a Universal 2 library.

The platform tags would be something like macosx_11_0_arm64. If the library supports both arm64 and x86_64 then the platform tag would be macosx_10_9_universal2 instead (pulling examples from my own projects, I don't know how to figure out the best version numbers.)

For my own projects I have to rely on GitHub Actions to build MacOS packages/libraries for me as downloadable artifacts. I can write a workflow for you if you're unable to compile the arm64 library yourself.

[bastibe]

Thank you so much for your help!

If you could point me to an example of how to have Github compile the macOS libraries, that would be fantastic! That might actually allow us to compile the wheels separately on each OS, instead of the current franken-system that packages precompiled libraries with the generic python code.

[HexDecimal]

This would be an example of a workflow. This script would be stored in a path like .github/workflows/build-libs.yml in the repository, and will take affect as soon as it's pushed to the repo.

name: Compile libraries

on:
  push:
  pull_request:

jobs:
  build-libs:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: ["macos-11"]
      fail-fast: true

    steps:
      - uses: actions/checkout@v2
      - name: Compile library
        run: ./mac_build.sh
      - uses: actions/upload-artifact@v2
        with:
          name: libs
          path: "*.dylib"
          retention-days: 7
          if-no-files-found: error 

I imagine this running the mac_build.sh script from bastibe/libsndfile-binaries, so this workflow would be added there for this example. You can also see the docs for actions/upload-artifact.

GitHub has good documentation on making workflows. Build artifacts uploaded this way can be downloaded from another job, such as a job that builds the wheel.

[bastibe]

This worked amazingly well, thank you a whole lot!

At the moment, Github runners do not yet provide M1 machines, but that's not super important at the moment, since @chuma9615 and @jurihock have already gracefully provided pre-build libraries in libsndfile-binaries#7 and libsndfile-binaries#8, respectively.

I'll investigate cross-compilation once I've managed to build wheels.

[bastibe]

I have drafted a new release at releases/0.11.0b1.

There is a known issue that will require a recompile of the Windows binaries with Visual Studio 14+, and will probably never work for all versions of Python. Apparently, file descriptor open does only work if libsndfile is compiled against the same Microsoft C Runtime as Python. The present libsndfile-provided binaries are apparently incompatible with my Python.

If anyone would like to contribute up-to-date Python libraries compiled with Visual Studio 14+, I'd be grateful for a pull request in bastibe/libsndfile-libraries. Or better yet, integrate it into the CI scripts to build the binaries automatically.

Also, you would help me tremendously if anyone could test the provided wheels on macOS and M1!

[SpotlightKid]

Will you provide source distribution archives (tarballs) for the latest and upcoming releases or should Linux distro packagers use the source archives automatically created by Github?

[bastibe]

I will upload a source distribution.

The reason I didn't attach it to the draft release linked above is that the current build script still bundles all the libraries with the source dist, making it unnecessarily big. This will need fixing before the final release is cut.

[HexDecimal]

There is a known issue that will require a recompile of the Windows binaries with Visual Studio 14+, and will probably never work for all versions of Python. Apparently, file descriptor open does only work if libsndfile is compiled against the same Microsoft C Runtime as Python. The present libsndfile-provided binaries are apparently incompatible with my Python.

Looks like if you compile to v140 then that binary will support all versions of Python after 3.5. The instructions I've found for telling MinGW to use specific runtimes seems convoluted so maybe it would be easier to use Vcpkg to get these builds.

[HaHeho]

Or better yet, integrate it into the CI scripts to build the binaries automatically.

I'm inexperienced with all the required compilation steps and CI. But I remembered that Mathias uses a script which builds a small version of libsndfile on Mac during some Github actions, build-small-libsndfile.sh. Forgive me if this doesn't help at all. :)

[bastibe]

@SpotlightKid I just uploaded a fixed sdist to the latest tag.

Right now, all that's missing for a release is

• someone should test the M1 wheels.
• I just noticed that the macos x64 wheel does not install. Any help as to why would be greatly appreciated.
• I've opened a pull request for the new release in #326 and noticed that the automated tests no longer run. Does anyone know why?

@HaHeho thank you for your response. Thanks to @HexDecimal, we now have automated builds for Windows and macOS! That's wild!

[HexDecimal]

... and noticed that the automated tests no longer run. Does anyone know why?

If you check the commits on GitHub then TravisCI's last status check was Nov 26, 2019. It stopped working sometime after that, probably when they dropped support for the travis-ci.org site. You'd have to migrate to travis-ci.com, but I don't think that site actually gives free time for FOSS projects. You might get a one time allotment to work with while you move to something else. I remember dropping TravisCI for GitHub Actions over this.

More info: https://travis-ci.community/t/org-com-migration-unexpectedly-comes-with-a-plan-change-for-oss-what-exactly-is-the-new-deal/10567

[SpotlightKid]

@SpotlightKid I just uploaded a fixed sdist to the latest tag.

Thanks for the heads up. The URLs for the attached files still have 0.11.0b1 in the path, which is slightly inconvenient. But I'm assuming this will be correct when a proper release is made.

[chuma9615]

@bastibe Tested the wheel on a M1 machine and it's working

Note that I tested soundfile-0.11.0b2-py2.py3-none-any.whl

[bastibe]

@chuma9615 that's the source-only wheel. Could you try the M1-specific soundfile-0.11.0b2-py2.py3-none-macosx-10.x-arm64.whl?

[HexDecimal]

Could you try the M1-specific soundfile-0.11.0b2-py2.py3-none-macosx-10.x-arm64.whl?

Note that macosx-10.x-arm64 isn't a valid platform tag since the dash - is used to separate tags from each other and this should all be one platform tag. The x and . in 10.x also isn't valid. Pip and PyPI might panic when seeing the tags in this state.

The libs were built with MACOSX_DEPLOYMENT_TARGET=10.9 so maybe you meant this tag to be macosx_10_9_arm64 and so the wheel would be called soundfile-0.11.0b2-py2.py3-none-macosx_10_9_arm64.whl

How wheel tags work is clear, but MacOS specific info is harder to find: https://www.python.org/dev/peps/pep-0425/ https://github.com/MacPython/wiki/wiki/Spinning-wheels https://lepture.com/en/2014/python-on-a-hard-wheel https://snarky.ca/the-challenges-in-designing-a-library-for-pep-425/

[chuma9615]

@bastibe When using the macOS ARM wheel it crashes

ERROR: soundfile-0.11.0b2-py2.py3-none-macosx-10.x-arm64.whl is not a supported wheel on this platform.

[msuch]

@bastibe there was a tiny typo bug in the code when importing sndfile on Mac. I fixed this in #328.

[azuwis]

I was trying to package python-soundfile 1.11.0 (https://github.com/bastibe/python-soundfile/tree/bastibe/0.11.0) for Nix on macOS arm64, and encountered the not a supported wheel problem:

Full log

``` Sourcing python-remove-tests-dir-hook Sourcing python-catch-conflicts-hook.sh Sourcing python-remove-bin-bytecode-hook.sh Sourcing setuptools-build-hook Using setuptoolsBuildPhase Using setuptoolsShellHook Sourcing pip-install-hook Using pipInstallPhase Sourcing python-imports-check-hook.sh Using pythonImportsCheckPhase Sourcing python-namespaces-hook Sourcing setuptools-check-hook Using setuptoolsCheckPhase @nix { "action": "setPhase", "phase": "unpackPhase" } unpacking sources unpacking source archive /nix/store/vrckfw0c4wg41ak9nik5gymaigx6i11j-source source root is source setting SOURCE_DATE_EPOCH to timestamp 315619200 of file source/tests/test_soundfile.py @nix { "action": "setPhase", "phase": "patchPhase" } patching sources @nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" } updateAutotoolsGnuConfigScriptsPhase @nix { "action": "setPhase", "phase": "configurePhase" } configuring no configure script, doing nothing @nix { "action": "setPhase", "phase": "buildPhase" } building Executing setuptoolsBuildPhase running bdist_wheel running build running build_py file _soundfile.py (for module _soundfile) not found creating build creating build/lib copying soundfile.py -> build/lib package init file '_soundfile_data/__init__.py' not found (or not a regular file) file _soundfile.py (for module _soundfile) not found generating cffi module 'build/lib/_soundfile.py' installing to build/bdist.macosx-10.6-arm64/wheel running install running install_lib creating build/bdist.macosx-10.6-arm64 creating build/bdist.macosx-10.6-arm64/wheel copying build/lib/soundfile.py -> build/bdist.macosx-10.6-arm64/wheel copying build/lib/_soundfile.py -> build/bdist.macosx-10.6-arm64/wheel running install_egg_info running egg_info creating soundfile.egg-info writing soundfile.egg-info/PKG-INFO writing dependency_links to soundfile.egg-info/dependency_links.txt writing requirements to soundfile.egg-info/requires.txt writing top-level names to soundfile.egg-info/top_level.txt writing manifest file 'soundfile.egg-info/SOURCES.txt' reading manifest file 'soundfile.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' adding license file 'LICENSE' writing manifest file 'soundfile.egg-info/SOURCES.txt' Copying soundfile.egg-info to build/bdist.macosx-10.6-arm64/wheel/soundfile-0.11.0-py3.9.egg-info running install_scripts adding license file "LICENSE" (matched pattern "LICEN[CS]E*") creating build/bdist.macosx-10.6-arm64/wheel/soundfile-0.11.0.dist-info/WHEEL creating 'dist/soundfile-0.11.0-py2.py3-none-macosx-10.x-arm64.whl' and adding 'build/bdist.macosx-10.6-arm64/wheel' to it adding '_soundfile.py' adding 'soundfile.py' adding 'soundfile-0.11.0.dist-info/LICENSE' adding 'soundfile-0.11.0.dist-info/METADATA' adding 'soundfile-0.11.0.dist-info/WHEEL' adding 'soundfile-0.11.0.dist-info/top_level.txt' adding 'soundfile-0.11.0.dist-info/RECORD' removing build/bdist.macosx-10.6-arm64/wheel Finished executing setuptoolsBuildPhase @nix { "action": "setPhase", "phase": "installPhase" } installing Executing pipInstallPhase /private/tmp/nix-build-python3.9-soundfile-0.11.0.drv-0/source/dist /private/tmp/nix-build-python3.9-soundfile-0.11.0.drv-0/source ERROR: soundfile-0.11.0-py2.py3-none-macosx-10.x-arm64.whl is not a supported wheel on this platform. ```

After replace macosx-10.x-arm64 in setup.py with any or macosx_11_0_arm64, it built fine, but pytest failed with many MemoryError:

__________________ ERROR at setup of test_blocks_rplus[obj] ___________________

file_stereo_rplus = <_io.FileIO name='tests/delme.please' mode='rb+' closefd=True>

    @pytest.fixture
    def sf_stereo_rplus(file_stereo_rplus):
>       with sf.SoundFile(file_stereo_rplus, 'r+') as f:

tests/test_soundfile.py:124: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
soundfile.py:646: in __init__
    self._file = self._open(file, mode_int, closefd)
soundfile.py:1197: in _open
    file_ptr = _snd.sf_open_virtual(self._init_virtual_io(file),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = SoundFile(<_io.FileIO name='tests/delme.please' mode='rb+' closefd=True>, mode='r+', samplerate=0, channels=0, format='FILE', subtype='FILE', endian='FILE')
file = <_io.FileIO name='tests/delme.please' mode='rb+' closefd=True>

    def _init_virtual_io(self, file):
        """Initialize callback functions for sf_open_virtual()."""
        @_ffi.callback("sf_vio_get_filelen")
>       def vio_get_filelen(user_data):
E       MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

soundfile.py:1217: MemoryError

According to https://cffi.readthedocs.io/en/latest/using.html#callbacks, ffi.callback() is old style callback, and has many drawbacks, should be replaced by new style callback def_extern().

[bastibe]

Thank you all so much for your help with this release!

And especially thank you @HexDecimal, for implementing the CI builds and tests!

I've uploaded a new set of pre-release wheels: 0.11.0b3.

I hope that these wheels now work on macOS M1. Again, I'd be grateful if anyone with access to an M1 Mac could test the macOS ARM wheel.

[azuwis]

Thank you all so much for your help with this release!

And especially thank you @HexDecimal, for implementing the CI builds and tests!

I've uploaded a new set of pre-release wheels: 0.11.0b3.

I hope that these wheels now work on macOS M1. Again, I'd be grateful if anyone with access to an M1 Mac could test the macOS ARM wheel.

$ pip install https://github.com/bastibe/python-soundfile/releases/download/0.11.0b3/soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl
ERROR: soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl is not a supported wheel on this platform.

MacBook Pro (13-inch, M1, 2020), macOS 12.2.1

[bastibe]

According to https://cffi.readthedocs.io/en/latest/using.html#callbacks, ffi.callback() is old style callback, and has many drawbacks, should be replaced by new style callback def_extern().

I've looked into this, and actually drafted an implementation of the callbacks locally. However, there is a problem with this approach: Currently, we are using the ABI mode of CFFI. The "new style" callbacks you mentioned require that soundfile uses the API mode of CFFI, which means compiling a bespoke version of soundfile for every operating system and version of Python we support.

While I am certainly open to doing this, especially now that we have CI runners to automate the task, I think it is out-of-scope for this pull request. Please raise the issue in a new issue, or preferably, try to draft a pull request for it.

[bastibe]

$ pip install https://github.com/bastibe/python-soundfile/releases/download/0.11.0b3/soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl ERROR: soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl is not a supported wheel on this platform.

MacBook Pro (13-inch, M1, 2020), macOS 12.2.1

I've seen a similar error on my wife's old mac. But it seemed to be related to her old version of pip. Perhaps that was wrong. Does your error go away if you install a current version of pip?

Regardless, the issue still needs fixing. I set the OS tag to macosx_10_9_XXX since it seemed to work on my wife's mac, and seemed preferable to macosx_10_9_XXX.macosx_10_10_XXX.macosx_11_0_XXX.future_macosx_that_we_dont_even_know_yet_XXX. But if that's what it takes then I'll do that instead. How annoying.

Could you confirm the correct OS tag to use? Perferably with some measure of backwards and forwards compatibility.

Anyway, my time to work on this for this week is up, so I'll have to defer the issue for next wednesday.

[azuwis]

I've seen a similar error on my wife's old mac. But it seemed to be related to her old version of pip. Perhaps that was wrong. Does your error go away if you install a current version of pip?

No.

bash-3.2$ pip install -U pip
Requirement already satisfied: pip in ./lib/python3.8/site-packages (22.0.3)
bash-3.2$ curl -sLO https://github.com/bastibe/python-soundfile/releases/download/0.11.0b3/soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl
bash-3.2$ pip install soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl
ERROR: soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl is not a supported wheel on this platform.

Could you confirm the correct OS tag to use? Perferably with some measure of backwards and forwards compatibility.

macosx_11_0_arm64 works for arm64.

bash-3.2$ mv soundfile-0.11.0b3-py2.py3-none-macosx_10_9_arm64.whl soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl
bash-3.2$ pip install soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl
Processing ./soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl
Requirement already satisfied: cffi>=1.0 in ./lib/python3.8/site-packages (from soundfile==0.11.0b3) (1.15.0)
Requirement already satisfied: pycparser in ./lib/python3.8/site-packages (from cffi>=1.0->soundfile==0.11.0b3) (2.21)
Installing collected packages: soundfile
Successfully installed soundfile-0.11.0b3

[bastibe]

I changed the platform tags to

• Intel: macosx_10_9_x86_64.macosx_11_0_x86_64
• M1: macosx_11_0_arm64

Please check them out again in the 0.11.b3 beta release. If I remember correctly, there was no M1 before macOS 11.0, so we don't need to include the 10.9 tag, right?

[azuwis]

Please check them out again in the 0.11.b3 beta release. If I remember correctly, there was no M1 before macOS 11.0, so we don't need to include the 10.9 tag, right?

It works after upgrading pip:

bash-3.2$ curl -sLO https://github.com/bastibe/python-soundfile/releases/download/0.11.0b3/soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl
bash-3.2$ pip3 install soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl 
ERROR: soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl is not a supported wheel on this platform.
WARNING: You are using pip version 20.2.3; however, version 22.0.3 is available.
You should consider upgrading via the '/private/tmp/venv/bin/python3 -m pip install --upgrade pip' command.
bash-3.2$ pip3 install -U pip
Collecting pip
  Using cached pip-22.0.3-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
Successfully installed pip-22.0.3
bash-3.2$ pip3 install soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl 
Processing ./soundfile-0.11.0b3-py2.py3-none-macosx_11_0_arm64.whl
Collecting cffi>=1.0
  Using cached cffi-1.15.0.tar.gz (484 kB)
  Preparing metadata (setup.py) ... done
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Using legacy 'setup.py install' for cffi, since package 'wheel' is not installed.
Installing collected packages: pycparser, cffi, soundfile
  Running setup.py install for cffi ... done
Successfully installed cffi-1.15.0 pycparser-2.21 soundfile-0.11.0b3

[bastibe]

Thank you for the confirmation!

Does anyone know why we need an up-to-date pip for this?

[HexDecimal]

Does anyone know why we need an up-to-date pip for this?

How tags are handed can change from version to version. My guess is that older versions of Pip are looking for exact MacOS versions rather than compatible versions or something similar. If that's the case then adding redundant versions would solve the issue for older versions of pip.

The links I've posted previously should help: https://github.com/bastibe/python-soundfile/issues/325#issuecomment-1044392935, mostly with which MacOS versions Python might be expecting, which could be 10.6, 10.7, or 10.9.

I think adding tags for 10.9 in addition to 11.0 might help. For example the platform tag macosx_10_9_arm64.macosx_11_0_arm64 instead of macosx_11_0_arm64.

[bastibe]

I will try that. Thank you.

[bastibe]

libsndfile 1.1.0 just released, so now it's time to rebuild all the libraries with MP3 support, and then cut the release.

[faroit]

libsndfile 1.1.0 just released, so now it's time to rebuild all the libraries with MP3 support, and then cut the release.

Can't wait to benchmark mp3 performance!

[bastibe]

I probably won't have time to look into this in-depth until next week. If anyone wants to take a stab at providing rebuilt libsndfile binaries, or updating our automated CI builds, please open a pull request in libsndfile-binaries.

[bastibe]

Our automated build system for Windows currently relies on vcpkg, which does not yet support libsndfile 1.1.0.

So far, I am failing at getting libsndfile 1.1.0 to compile (using vcpkg) on my computer. But I'll keep trying. Why must compiling on Windows be so painful?

[HexDecimal]

There's a PR for libsndfile 1.1.0 on Vcpkg: https://github.com/microsoft/vcpkg/pull/23800 You checkout that PR to build libsndfile 1.1.0 on your computer, or tell the automated builds to use that revision. I didn't use a submodule to handle Vcpkg in my original PR which would've made this easier to do.

[bastibe]

Thank you!

I had searched the repo for open pull requests earlier, but only for "sndfile", not "libsndfile" 🙄. My bad.

At any rate, I think we can wait for the vcpkg to update, at which point our CI builds should be able to build the new version automatically. Do you happen to know how quickly these vcpkg pull requests are accepted?

[bastibe]

I've worked on the macOS build script: https://github.com/bastibe/libsndfile-binaries/blob/bastibe/1.1.0-build/mac_build.sh

~It now seems to compile mpg123 and liblame correctly, but somehow none of our custom-built libraries are picked up by the libsndfile configure script any longer. Any help on this would be greatly appreciated! Fighting compilers is not my strong suite.~

Apparently this is only an issue on my local machine, the CI seems to run the macOS build script without issue.

We don't have a good solution for M1 builds yet, though. May I once again ask for volunteers to compile a current version of libsndfile on an M1 Mac?

[bastibe]

Here's some progress: The VCPKG pull request for libsndfile 1.1.0 has been merged, and I fixed the build script for the CI runner.

However, I currently still have two major issues over there:

• VCPKG on the CI runners does not seem to be up-to-date, and still fetches libsndfile 1.0.31, and vcpkg update does not work.
• I don't yet know how to build M1 binaries with Github's CI.

Both of these are currently blocking issues for a new release of soundfile.

If you'd like to help, please head on over to https://github.com/bastibe/libsndfile-binaries/pull/11.

[faroit]

@bastibe m1 mac is not currently supported, so I guess one would have to build locally :-/ https://github.com/actions/virtual-environments/issues/2187

[bastibe]

It should be possible, in theory, to cross-compile M1 binaries from an Intel Mac. But given that the one Mac I have access to is somewhat unreliable and not mine to tinker with, that's not something I can implement 🤷‍♀️. If someone else wants to take a stab at it, I'd be very interested!

[hiccup7]

Alternatives to VCPKG that update frequently: https://winlibs.com/ Use UCRT for Python 3.5+ https://www.msys2.org/ Easy updates

I don't have experience with compiling for Windows, but I am trying to learn. I saw these projects recommended. Hope they are helpful.

[hiccup7]

https://github.com/libsndfile/libsndfile/ says "Autotools is the primary and recommended build toolchain". WinLibs and MSYS2 both support GNU Autotools.

[hiccup7]

@cgohlke You have a link to https://github.com/bastibe/PySoundFile at https://www.lfd.uci.edu/~gohlke/pythonlibs Please update your link to the renamed project: https://github.com/bastibe/python-soundfile

As you know, there are various problems when Python and the libsndfile DLL are linked to different versions of the Microsoft C runtime DLL. See https://github.com/libsndfile/libsndfile/ This team is trying to correct the problem for Python v3.5+. However, we Mac, Linux and inexperienced Windows users are having difficulty navigating the subtleties of Windows building for Python.

We would greatly appreciate you advice and help on how to design the CI scripts for Windows building at https://github.com/bastibe/libsndfile-binaries A problem with VCPKG is that the version in the CI runner is not up-to-date or slow to update for the latest libsndfile release. Is there a way to use build tools such that one libsndfile DLL will work with all Python versions starting with v3.5?

[bastibe]

Good news: the CI runners were updated, and are building libsndfile binaries with MP3 support now!

Which means we now have up-to-date binaries for Windows 32/64, and macOS Intel over at https://github.com/bastibe/libsndfile-binaries/pull/11. We're still lacking macOS ARM binaries, however, and I don't have an appropriate computer to compile them on. If anyone with an M1 Mac would like to help, please open a pull request at https://github.com/bastibe/libsndfile-binaries/

[SpotlightKid]

We're still lacking macOS ARM binaries, however, and I don't have an appropriate computer to compile them on.

Maybe this can help?

https://github.com/quickemu-project/quickemu

[bastibe]

I now have a full set of binaries available! Thank you @faroit for providing hand-built ARM binaries! And thank you very much to everyone who helped make this happen!

I'll soon cut new wheels to beta-test, and then we can hopefully release the new version!

[bastibe]

A new beta release is available here: https://github.com/bastibe/python-soundfile/releases/tag/0.11.0b4

If you can, please make sure the wheels work on your platform, and can indeed open MP3 files!

[BobCN2017]

If you can, please make sure the wheels work on your platform, and can indeed open MP3 files!

could you build a new version for linux platform? thank you very much.

[adrian-stepien]

A new beta release is available here: https://github.com/bastibe/python-soundfile/releases/tag/0.11.0b4

@bastibe I tried this beta on MacOS with M1 and arm64 version of Python 3.9.12. Generated an mp3: sox -n sine.mp3 synth 10 sine 1000

Upon loading, I got an error Format not recognised

---------------------------------------------------------------------------
LibsndfileError                           Traceback (most recent call last)
Input In [7], in <cell line: 1>()
----> 1 sf.read("sine.mp3")

File ~/miniforge/lib/python3.9/site-packages/soundfile.py:282, in read(file, frames, start, stop, dtype, always_2d, fill_value, out, samplerate, channels, format, subtype, endian, closefd)
    196 def read(file, frames=-1, start=0, stop=None, dtype='float64', always_2d=False,
    197          fill_value=None, out=None, samplerate=None, channels=None,
    198          format=None, subtype=None, endian=None, closefd=True):
    199     """Provide audio data from a sound file as NumPy array.
    200
    201     By default, the whole file is read from the beginning, but the
   (...)
    280
    281     """
--> 282     with SoundFile(file, 'r', samplerate, channels,
    283                    subtype, endian, format, closefd) as f:
    284         frames = f._prepare_read(start, stop, frames)
    285         data = f.read(frames, dtype, always_2d, fill_value, out)

File ~/miniforge/lib/python3.9/site-packages/soundfile.py:655, in SoundFile.__init__(self, file, mode, samplerate, channels, subtype, endian, format, closefd)
    652 self._mode = mode
    653 self._info = _create_info_struct(file, mode, samplerate, channels,
    654                                  format, subtype, endian)
--> 655 self._file = self._open(file, mode_int, closefd)
    656 if set(mode).issuperset('r+') and self.seekable():
    657     # Move write position to 0 (like in Python file objects)
    658     self.seek(0)

File ~/miniforge/lib/python3.9/site-packages/soundfile.py:1213, in SoundFile._open(self, file, mode_int, closefd)
   1210 if file_ptr == _ffi.NULL:
   1211     # get the actual error code
   1212     err = _snd.sf_error(file_ptr)
-> 1213     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
   1214 if mode_int == _snd.SFM_WRITE:
   1215     # Due to a bug in libsndfile version <= 1.0.25, frames != 0
   1216     # when opening a named pipe in SFM_WRITE mode.
   1217     # See http://github.com/erikd/libsndfile/issues/77.
   1218     self._info.frames = 0

LibsndfileError: Error opening 'sine.mp3': Format not recognised.

I can't see MPEG/MP3 in the output of sf.available_formats()

{'AIFF': 'AIFF (Apple/SGI)',
 'AU': 'AU (Sun/NeXT)',
 'AVR': 'AVR (Audio Visual Research)',
 'CAF': 'CAF (Apple Core Audio File)',
 'FLAC': 'FLAC (Free Lossless Audio Codec)',
 'HTK': 'HTK (HMM Tool Kit)',
 'SVX': 'IFF (Amiga IFF/SVX8/SV16)',
 'MAT4': 'MAT4 (GNU Octave 2.0 / Matlab 4.2)',
 'MAT5': 'MAT5 (GNU Octave 2.1 / Matlab 5.0)',
 'MPC2K': 'MPC (Akai MPC 2k)',
 'OGG': 'OGG (OGG Container format)',
 'PAF': 'PAF (Ensoniq PARIS)',
 'PVF': 'PVF (Portable Voice Format)',
 'RAW': 'RAW (header-less)',
 'RF64': 'RF64 (RIFF 64)',
 'SD2': 'SD2 (Sound Designer II)',
 'SDS': 'SDS (Midi Sample Dump Standard)',
 'IRCAM': 'SF (Berkeley/IRCAM/CARL)',
 'VOC': 'VOC (Creative Labs)',
 'W64': 'W64 (SoundFoundry WAVE 64)',
 'WAV': 'WAV (Microsoft)',
 'NIST': 'WAV (NIST Sphere)',
 'WAVEX': 'WAVEX (Microsoft)',
 'WVE': 'WVE (Psion Series 3)',
 'XI': 'XI (FastTracker 2)'}

[bastibe]

Bugs so far:

• soundfile.available_formats (and therefore, libsndfile's sf_command(NULL, SF_FORMAT_INFO, ...) does not seem to report MP3 as an available format.
• soundfile.write does not know about .mp3 file extensions, but will happily save an mp3 if you supply format='MPEG', subtype='MPEG_LAYER_III', or use the extension .mpeg.

[bastibe]

could you build a new version for linux platform? thank you very much.

The none-any wheel should work on Linux. But you'll have to provide your own libsndfile with MP3 support.

I tried this beta on MacOS with M1 and arm64 version of Python 3.9.12. Generated an mp3: sox -n sine.mp3 synth 10 sine 1000

Upon loading, I got an error Format not recognised

Does it work if you rename the file to $filename.mpeg, or supply the format/subtype manually (format='MPEG', subtype='MPEG_LAYER_III')?