search

mozilla / firefox-translations-training

Training pipelines for Firefox Translations neural machine translation models
https://mozilla.github.io/firefox-translations-training/
Mozilla Public License 2.0
135 stars 28 forks source link

tests fail with numpy 2.0 #689

Closed bhearsum closed 2 days ago

bhearsum commented 1 week ago 
[task 2024-06-19T13:43:12.117Z] ==================================== ERRORS ====================================
[task 2024-06-19T13:43:12.117Z] _________________ ERROR collecting tests/test_tracking_cli.py __________________
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/runner.py:341: in from_call
[task 2024-06-19T13:43:12.117Z]     result: Optional[TResult] = func()
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/runner.py:372: in <lambda>
[task 2024-06-19T13:43:12.117Z]     call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/python.py:531: in collect
[task 2024-06-19T13:43:12.117Z]     self._inject_setup_module_fixture()
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/python.py:545: in _inject_setup_module_fixture
[task 2024-06-19T13:43:12.117Z]     self.obj, ("setUpModule", "setup_module")
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/python.py:310: in obj
[task 2024-06-19T13:43:12.117Z]     self._obj = obj = self._getobj()
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/python.py:528: in _getobj
[task 2024-06-19T13:43:12.117Z]     return self._importtestmodule()
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/python.py:617: in _importtestmodule
[task 2024-06-19T13:43:12.117Z]     mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/pathlib.py:567: in import_path
[task 2024-06-19T13:43:12.117Z]     importlib.import_module(module_name)
[task 2024-06-19T13:43:12.117Z] /usr/lib/python3.10/importlib/__init__.py:126: in import_module
[task 2024-06-19T13:43:12.117Z]     return _bootstrap._gcd_import(name[level:], package, level)
[task 2024-06-19T13:43:12.117Z] <frozen importlib._bootstrap>:1050: in _gcd_import
[task 2024-06-19T13:43:12.117Z]     ???
[task 2024-06-19T13:43:12.117Z] <frozen importlib._bootstrap>:1027: in _find_and_load
[task 2024-06-19T13:43:12.117Z]     ???
[task 2024-06-19T13:43:12.117Z] <frozen importlib._bootstrap>:1006: in _find_and_load_unlocked
[task 2024-06-19T13:43:12.117Z]     ???
[task 2024-06-19T13:43:12.117Z] <frozen importlib._bootstrap>:688: in _load_unlocked
[task 2024-06-19T13:43:12.117Z]     ???
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:186: in exec_module
[task 2024-06-19T13:43:12.117Z]     exec(co, module.__dict__)
[task 2024-06-19T13:43:12.117Z] tests/test_tracking_cli.py:10: in <module>
[task 2024-06-19T13:43:12.117Z]     from translations_parser.cli import experiments as experiments_publish
[task 2024-06-19T13:43:12.117Z] tracking/translations_parser/cli/experiments.py:16: in <module>
[task 2024-06-19T13:43:12.117Z]     from translations_parser.parser import TrainingParser
[task 2024-06-19T13:43:12.117Z] tracking/translations_parser/parser.py:13: in <module>
[task 2024-06-19T13:43:12.117Z]     from translations_parser.publishers import Publisher
[task 2024-06-19T13:43:12.117Z] tracking/translations_parser/publishers.py:9: in <module>
[task 2024-06-19T13:43:12.117Z]     import wandb
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/wandb/__init__.py:27: in <module>
[task 2024-06-19T13:43:12.117Z]     from wandb import sdk as wandb_sdk
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/wandb/sdk/__init__.py:4: in <module>
[task 2024-06-19T13:43:12.117Z]     from .artifacts.artifact import Artifact  # noqa: F401
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/wandb/sdk/artifacts/artifact.py:35: in <module>
[task 2024-06-19T13:43:12.117Z]     from wandb import data_types, env, util
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/wandb/data_types.py:33: in <module>
[task 2024-06-19T13:43:12.117Z]     from .sdk.data_types import _dtypes
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/wandb/sdk/data_types/_dtypes.py:396: in <module>
[task 2024-06-19T13:43:12.117Z]     NumberType.types.append(np.float_)
[task 2024-06-19T13:43:12.117Z] ../../.cache/pypoetry/virtualenvs/firefox-translations-training-hlUNdH3O-py3.10/lib/python3.10/site-packages/numpy/__init__.py:397: in __getattr__
[task 2024-06-19T13:43:12.117Z]     raise AttributeError(
[task 2024-06-19T13:43:12.117Z] E   AttributeError: `np.float_` was removed in the NumPy 2.0 release. Use `np.float64` instead.. Did you mean: 'float16'?
[task 2024-06-19T13:43:12.117Z] =========================== short test summary info ============================
[task 2024-06-19T13:43:12.117Z] ERROR tests/test_tracking_cli.py - AttributeError: `np.float_` was removed in...
[task 2024-06-19T13:43:12.117Z] !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
[task 2024-06-19T13:43:12.117Z] =============================== 1 error in 1.91s ===============================
eu9ene commented 1 week ago

@vrigal @La0 we should pin the transient dependencies for the publishing package. We use two strategies: pip-compile and poetry lock. Probably a requirements.in file + pip-compiled requrements.txt can work here. Here's an example: https://github.com/mozilla/firefox-translations-training/tree/main/pipeline/clean/requirements