I am trying to install -e, but I keep getting this error:
Obtaining file:///Users/jihyelee/bigcode-evaluation-harness
Preparing metadata (setup.py) ... done
Collecting transformers>=4.25.1 (from lm-eval==0.0.0)
Obtaining dependency information for transformers>=4.25.1 from https://files.pythonhosted.org/packages/5b/0b/e45d26ccd28568013523e04f325432ea88a442b4e3020b757cf4361f0120/transformers-4.30.2-py3-none-any.whl.metadata
Using cached transformers-4.30.2-py3-none-any.whl.metadata (113 kB)
Collecting accelerate>=0.13.2 (from lm-eval==0.0.0)
Obtaining dependency information for accelerate>=0.13.2 from https://files.pythonhosted.org/packages/10/d3/5382aa337d3e67214003a17b06bfc07cf0334356b4e8aaf3b12b0d38c83f/accelerate-0.20.3-py3-none-any.whl.metadata
Using cached accelerate-0.20.3-py3-none-any.whl.metadata (17 kB)
Collecting datasets>=2.6.1 (from lm-eval==0.0.0)
Obtaining dependency information for datasets>=2.6.1 from https://files.pythonhosted.org/packages/d3/95/ef83542e7a8e2bfc4432ee2cd8a6b52eb30fb1e605871e8871e94ce65fb1/datasets-2.13.2-py3-none-any.whl.metadata
Using cached datasets-2.13.2-py3-none-any.whl.metadata (20 kB)
Collecting evaluate>=0.3.0 (from lm-eval==0.0.0)
Using cached evaluate-0.4.0-py3-none-any.whl (81 kB)
Collecting pyext==0.7 (from lm-eval==0.0.0)
Using cached pyext-0.7.tar.gz (7.8 kB)
Preparing metadata (setup.py) ... done
Collecting mosestokenizer==1.0.0 (from lm-eval==0.0.0)
Using cached mosestokenizer-1.0.0-py3-none-any.whl (51 kB)
Collecting huggingface_hub>=0.11.1 (from lm-eval==0.0.0)
Obtaining dependency information for huggingface_hub>=0.11.1 from https://files.pythonhosted.org/packages/7f/c4/adcbe9a696c135578cabcbdd7331332daad4d49b7c43688bc2d36b3a47d2/huggingface_hub-0.16.4-py3-none-any.whl.metadata
Using cached huggingface_hub-0.16.4-py3-none-any.whl.metadata (12 kB)
Collecting docopt (from mosestokenizer==1.0.0->lm-eval==0.0.0)
Using cached docopt-0.6.2.tar.gz (25 kB)
Preparing metadata (setup.py) ... done
Collecting openfile (from mosestokenizer==1.0.0->lm-eval==0.0.0)
Using cached openfile-0.0.7-py3-none-any.whl (2.4 kB)
Collecting toolwrapper (from mosestokenizer==1.0.0->lm-eval==0.0.0)
Using cached toolwrapper-2.1.0.tar.gz (3.2 kB)
Preparing metadata (setup.py) ... done
Collecting numpy>=1.17 (from accelerate>=0.13.2->lm-eval==0.0.0)
Using cached numpy-1.21.6.zip (10.3 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper
return func(self, options, args)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 378, in run
reqs, check_supported_wheels=not options.target_dir
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve
collected.requirements, max_rounds=limit_how_complex_resolution_can_be
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 230, in _get_updated_criteria
self._add_to_criteria(criteria, requirement, parent=candidate)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
if not criterion.candidates:
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in bool
return bool(self._sequence)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool
return any(self)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link
version=version,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 299, in init
version=version,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in init
self.dist = self._prepare()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 658, in _prepare_linked_requirement
self.check_build_deps,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 70, in _get_prepared_distribution
finder, build_isolation, check_build_deps
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
self._install_build_reqs(finder)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
return super().get_requires_for_build_wheel(config_settings=cs)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 167, in get_requires_for_build_wheel
'config_settings': config_settings
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
obj = import_module(mod_path)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/init.py", line 18, in
from setuptools.dist import Distribution
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 38, in
from setuptools import windows_support
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in
import ctypes
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/ctypes/init.py", line 7, in
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
Here are the packages that I've installed with versions
Package Version
I am trying to install -e, but I keep getting this error:
Obtaining file:///Users/jihyelee/bigcode-evaluation-harness Preparing metadata (setup.py) ... done Collecting transformers>=4.25.1 (from lm-eval==0.0.0) Obtaining dependency information for transformers>=4.25.1 from https://files.pythonhosted.org/packages/5b/0b/e45d26ccd28568013523e04f325432ea88a442b4e3020b757cf4361f0120/transformers-4.30.2-py3-none-any.whl.metadata Using cached transformers-4.30.2-py3-none-any.whl.metadata (113 kB) Collecting accelerate>=0.13.2 (from lm-eval==0.0.0) Obtaining dependency information for accelerate>=0.13.2 from https://files.pythonhosted.org/packages/10/d3/5382aa337d3e67214003a17b06bfc07cf0334356b4e8aaf3b12b0d38c83f/accelerate-0.20.3-py3-none-any.whl.metadata Using cached accelerate-0.20.3-py3-none-any.whl.metadata (17 kB) Collecting datasets>=2.6.1 (from lm-eval==0.0.0) Obtaining dependency information for datasets>=2.6.1 from https://files.pythonhosted.org/packages/d3/95/ef83542e7a8e2bfc4432ee2cd8a6b52eb30fb1e605871e8871e94ce65fb1/datasets-2.13.2-py3-none-any.whl.metadata Using cached datasets-2.13.2-py3-none-any.whl.metadata (20 kB) Collecting evaluate>=0.3.0 (from lm-eval==0.0.0) Using cached evaluate-0.4.0-py3-none-any.whl (81 kB) Collecting pyext==0.7 (from lm-eval==0.0.0) Using cached pyext-0.7.tar.gz (7.8 kB) Preparing metadata (setup.py) ... done Collecting mosestokenizer==1.0.0 (from lm-eval==0.0.0) Using cached mosestokenizer-1.0.0-py3-none-any.whl (51 kB) Collecting huggingface_hub>=0.11.1 (from lm-eval==0.0.0) Obtaining dependency information for huggingface_hub>=0.11.1 from https://files.pythonhosted.org/packages/7f/c4/adcbe9a696c135578cabcbdd7331332daad4d49b7c43688bc2d36b3a47d2/huggingface_hub-0.16.4-py3-none-any.whl.metadata Using cached huggingface_hub-0.16.4-py3-none-any.whl.metadata (12 kB) Collecting docopt (from mosestokenizer==1.0.0->lm-eval==0.0.0) Using cached docopt-0.6.2.tar.gz (25 kB) Preparing metadata (setup.py) ... done Collecting openfile (from mosestokenizer==1.0.0->lm-eval==0.0.0) Using cached openfile-0.0.7-py3-none-any.whl (2.4 kB) Collecting toolwrapper (from mosestokenizer==1.0.0->lm-eval==0.0.0) Using cached toolwrapper-2.1.0.tar.gz (3.2 kB) Preparing metadata (setup.py) ... done Collecting numpy>=1.17 (from accelerate>=0.13.2->lm-eval==0.0.0) Using cached numpy-1.21.6.zip (10.3 MB) Installing build dependencies ... done Getting requirements to build wheel ... done ERROR: Exception: Traceback (most recent call last): File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper status = run_func(*args) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 248, in wrapper return func(self, options, args) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 378, in run reqs, check_supported_wheels=not options.target_dir File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 93, in resolve collected.requirements, max_rounds=limit_how_complex_resolution_can_be File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve failure_causes = self._attempt_to_pin_criterion(name) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion criteria = self._get_updated_criteria(candidate) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 230, in _get_updated_criteria self._add_to_criteria(criteria, requirement, parent=candidate) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria if not criterion.candidates: File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in bool return bool(self._sequence) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool return any(self) File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in
return (c for c in iterator if id(c) not in self._incompatible_ids)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
candidate = func()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link
version=version,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 299, in init
version=version,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in init
self.dist = self._prepare()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 538, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 658, in _prepare_linked_requirement
self.check_build_deps,
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 70, in _get_prepared_distribution
finder, build_isolation, check_build_deps
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 48, in prepare_distribution_metadata
self._install_build_reqs(finder)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 118, in _install_build_reqs
build_reqs = self._get_build_requires_wheel()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 95, in _get_build_requires_wheel
return backend.get_requires_for_build_wheel()
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 697, in get_requires_for_build_wheel
return super().get_requires_for_build_wheel(config_settings=cs)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 167, in get_requires_for_build_wheel
'config_settings': config_settings
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
obj = import_module(mod_path)
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 953, in _find_and_load_unlocked
File "", line 219, in _call_with_frames_removed
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/init.py", line 18, in
from setuptools.dist import Distribution
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 38, in
from setuptools import windows_support
File "/private/var/folders/2z/9mppbxqj7vgfrxlt416vn4zc0000gn/T/pip-build-env-c73jqf/overlay/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in
import ctypes
File "/Users/jihyelee/.pyenv/versions/3.7.10/lib/python3.7/ctypes/init.py", line 7, in
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
Here are the packages that I've installed with versions Package Version
pip 23.2.1 setuptools 68.0.0 torch 1.12.1 typing_extensions 4.7.1 wheel 0.41.2
Python 3.7.10