Managed to solve this by bumping sentencepiece version to latest. PR to follow
Processing /mnt/ssd-2/spar/jon/elk
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Installing backend dependencies: started
Installing backend dependencies: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Collecting datasets>=2.9.0 (from eleuther-elk==0.1.1)
...
Building wheels for collected packages: sentencepiece, eleuther-elk
Building wheel for sentencepiece (setup.py): started
Building wheel for sentencepiece (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [86 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-cpython-311
creating build/lib.linux-x86_64-cpython-311/sentencepiece
copying src/sentencepiece/__init__.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
copying src/sentencepiece/_version.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.linux-x86_64-cpython-311/sentencepiece
running build_ext
/bin/sh: 1: pkg-config: not found
Cloning into 'sentencepiece'...
Note: checking out '58f256cf6f01bb86e6fa634a5cc560de5bd1667d'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
CMake Error: The source directory "/tmp/pip-install-7fpyf3pb/sentencepiece_26ada9fb1424472687069dc2fb205af6/bundled" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
Unknown argument --parallel
Unknown argument 96
Usage: cmake --build <dir> [options] [-- [native-options]]
Options:
<dir> = Project binary directory to be built.
--target <tgt> = Build <tgt> instead of default targets.
May only be specified once.
--config <cfg> = For multi-configuration tools, choose <cfg>.
--clean-first = Build target 'clean' first, then build.
(To clean only, use --target 'clean'.)
--use-stderr = Ignored. Behavior is default in CMake >= 3.0.
-- = Pass remaining options to the native tool.
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-install-7fpyf3pb/sentencepiece_26ada9fb1424472687069dc2fb205af6/setup.py", line 136, in <module>
setup(
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 325, in run
self.run_command("build")
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 132, in run
self.run_command(cmd_name)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/dist.py", line 1208, in run_command
super().run_command(command)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 84, in run
_build_ext.run(self)
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
self.build_extensions()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 468, in build_extensions
self._build_extensions_serial()
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 494, in _build_extensions_serial
self.build_extension(ext)
File "/tmp/pip-install-7fpyf3pb/sentencepiece_26ada9fb1424472687069dc2fb205af6/setup.py", line 89, in build_extension
subprocess.check_call(['./build_bundled.sh', __version__])
File "/mnt/ssd-2/spar/jon/miniconda3/envs/elk/lib/python3.11/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./build_bundled.sh', '0.1.97']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sentencepiece
Running setup.py clean for sentencepiece
Building wheel for eleuther-elk (pyproject.toml): started
Building wheel for eleuther-elk (pyproject.toml): finished with status 'done'
Created wheel for eleuther-elk: filename=eleuther_elk-0.1.1-py3-none-any.whl size=72237 sha256=529707e80b6aff6fb4d5edf9e5e393704ecf1f017da04035f26aec3f05f077d8
Stored in directory: /tmp/pip-ephem-wheel-cache-t2y1slmz/wheels/aa/9f/64/acb739d837c6fad64f72b029e4a2eef18da30cc38375df955a
Successfully built eleuther-elk
Failed to build sentencepiece
ERROR: Could not build wheels for sentencepiece, which is required to install pyproject.toml-based projects
Managed to solve this by bumping sentencepiece version to latest. PR to follow