Open yifanmai opened 10 months ago
@yifanmai I was able to work around the issue with PyExt by creating a fork matthewdouglas/PyExt@13447e5.
However, there's more work needed, as helm-run
tells me. This is fixed in huggingface/datasets#5238 so this dep would need to be bumped as well (>=2.7.0).
Traceback (most recent call last):
File "/home/matt/helm/.venv/bin/helm-run", line 5, in <module>
from helm.benchmark.run import main
File "/home/matt/helm/.venv/lib/python3.11/site-packages/helm/benchmark/run.py", line 19, in <module>
from helm.benchmark import vlm_run_specs # noqa
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/matt/helm/.venv/lib/python3.11/site-packages/helm/benchmark/vlm_run_specs.py", line 6, in <module>
from .run_specs import run_spec_function, get_exact_match_metric_specs
File "/home/matt/helm/.venv/lib/python3.11/site-packages/helm/benchmark/run_specs.py", line 32, in <module>
from .scenarios.lex_glue_scenario import (
File "/home/matt/helm/.venv/lib/python3.11/site-packages/helm/benchmark/scenarios/lex_glue_scenario.py", line 5, in <module>
import datasets
File "/home/matt/helm/.venv/lib/python3.11/site-packages/datasets/__init__.py", line 47, in <module>
from .builder import ArrowBasedBuilder, BeamBasedBuilder, BuilderConfig, DatasetBuilder, GeneratorBasedBuilder
File "/home/matt/helm/.venv/lib/python3.11/site-packages/datasets/builder.py", line 91, in <module>
@dataclass
^^^^^^^^^
File "/usr/lib/python3.11/dataclasses.py", line 1230, in dataclass
return wrap(cls)
^^^^^^^^^
File "/usr/lib/python3.11/dataclasses.py", line 1220, in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/dataclasses.py", line 958, in _process_class
cls_fields.append(_get_field(cls, name, type, kw_only))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/dataclasses.py", line 815, in _get_field
raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'datasets.utils.version.Version'> for field version is not allowed: use default_factory```
scikit-images
also needs to be upgraded to support Python 3.11. See #2858 for details.
Currently HELM doesn't work on Python 3.11 because it depends on pyext, which breaks in Python 3.11 due to
getargspec
being removed frominspect
in Python 3.11.