search

recommenders-team / recommenders

Best Practices on Recommendation Systems
https://recommenders-team.github.io/recommenders/intro.html
MIT License
18.47k stars 3.04k forks source link

Support of Python 3.12 #2098

Open miguelgfierro opened 2 months ago

miguelgfierro commented 2 months ago

Description

Related Issues

https://github.com/recommenders-team/recommenders/issues/2097

References

Checklist:

miguelgfierro commented 2 months ago

let's try, this can be as simple as changing a few lines of code, or we might need to fix many dependencies.

Error in Cornac dependency:

2024-05-15T05:14:26: Collecting cornac<2,>=1.15.2 (from recommenders[dev,gpu]@ git+https://github.com/recommenders-team/recommenders.git@809a1d39c3e4bdab72aa56ee80d9eb79413ac29d->-r /azureml-environment-setup/condaenv.8fr4a5nk.requirements.txt (line 3))
2024-05-15T05:14:26:   Downloading cornac-1.18.0.tar.gz (5.8 MB)
2024-05-15T05:14:26:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.8/5.8 MB 76.2 MB/s eta 0:00:00
2024-05-15T05:14:26:   Preparing metadata (setup.py): started
2024-05-15T05:14:26:   Preparing metadata (setup.py): finished with status 'error'

2024-05-15T05:14:26: failed
2024-05-15T05:14:26: Pip subprocess error:
2024-05-15T05:14:26:   Running command git clone --filter=blob:none --quiet https://github.com/recommenders-team/recommenders.git /tmp/pip-install-va67dnkb/recommenders_830bb823f0204ab9ae042e7d5057b430
2024-05-15T05:14:26:   Running command git rev-parse -q --verify 'sha^809a1d39c3e4bdab72aa56ee80d9eb79413ac29d'
2024-05-15T05:14:26:   Running command git fetch -q https://github.com/recommenders-team/recommenders.git 809a1d39c3e4bdab72aa56ee80d9eb79413ac29d
2024-05-15T05:14:26:   Running command git checkout -q 809a1d39c3e4bdab72aa56ee80d9eb79413ac29d
2024-05-15T05:14:26:   error: subprocess-exited-with-error
2024-05-15T05:14:26:   
2024-05-15T05:14:26:   √ó python setup.py egg_info did not run successfully.
2024-05-15T05:14:26:   │ exit code: 1
2024-05-15T05:14:26:   ╰─> [2 lines of output]
2024-05-15T05:14:26:       We need some dependencies to build Cornac.
2024-05-15T05:14:26:       Run: pip3 install Cython numpy scipy
2024-05-15T05:14:26:       [end of output]
2024-05-15T05:14:26:   
2024-05-15T05:14:26:   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-05-15T05:14:26: error: metadata-generation-failed

2024-05-15T05:14:26: √ó Encountered error while generating package metadata.
2024-05-15T05:14:26: ╰─> See above for output.

2024-05-15T05:14:26: note: This is an issue with the package mentioned above, not pip.
2024-05-15T05:14:26: hint: See above for details.
2024-05-15T05:14:26: CondaEnvException: Pip failed
miguelgfierro commented 2 months ago

Other error, it seems it's coming from dependencies:

"error": ***
        "message": "Activity Failed:\n***\n    \"error\": ***\n        \"code\": \"UserError\",\n        \"message\": \"Image build failed. For more details, check log file azureml-logs/20_image_build_log.txt.\",\n        \"messageFormat\": \"Image build failed. For more details, check log file ***ArtifactPath***.\",\n        \"messageParameters\": ***\n            \"ArtifactPath\": \"azureml-logs/20_image_build_log.txt\"\n        ***,\n        \"details\": [],\n        \"innerError\": ***\n            \"code\": \"BadArgument\",\n            \"innerError\": ***\n                \"code\": \"ImageBuildFailure\"\n            ***\n        ***\n    ***,\n    \"correlation\": ***\n        \"operation\": \"5f6a3d4a1da7af81649b6ef9e4ebee05\",\n        \"request\": \"b3c45b90ff1ad97d\"\n    ***,\n    \"environment\": \"eastus\",\n    \"location\": \"eastus\",\n    \"time\": \"2024-05-15T06:37:34.302265Z\",\n    \"componentName\": \"RunHistory\"\n***"
    ***
***
2024-05-15T06:37:11:   Running command git fetch -q https://github.com/recommenders-team/recommenders.git 496b22fca0faf861ff98739c8deec46b031e8d44
2024-05-15T06:37:11:   Running command git checkout -q 496b22fca0faf861ff98739c8deec46b031e8d44
2024-05-15T06:37:11:   error: subprocess-exited-with-error
2024-05-15T06:37:11:   
2024-05-15T06:37:11:   × Getting requirements to build wheel did not run successfully.
2024-05-15T06:37:11:   │ exit code: 1
2024-05-15T06:37:11:   ╰─> [33 lines of output]
2024-05-15T06:37:11:       Traceback (most recent call last):
2024-05-15T06:37:11:         File "/azureml-envs/azureml_06f0f6a5b197cd9c88[281](https://github.com/recommenders-team/recommenders/actions/runs/9090676506/job/24984293755#step:3:288)4406eaed054/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
2024-05-15T06:37:11:           main()
2024-05-15T06:37:11:         File "/azureml-envs/azureml_06f0f6a5b197cd9c882814406eaed054/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
2024-05-15T06:37:11:           json_out['return_val'] = hook(**hook_input['kwargs'])
2024-05-15T06:37:11:                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T06:37:11:         File "/azureml-envs/azureml_06f0f6a5b197cd9c882814406eaed054/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
2024-05-15T06:37:11:           backend = _build_backend()
2024-05-15T06:37:11:                     ^^^^^^^^^^^^^^^^
2024-05-15T06:37:11:         File "/azureml-envs/azureml_06f0f6a5b197cd9c882814406eaed054/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
2024-05-15T06:37:11:           obj = import_module(mod_path)
2024-05-15T06:37:11:                 ^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T06:37:11:         File "/azureml-envs/azureml_06f0f6a5b197cd9c882814406eaed054/lib/python3.12/importlib/__init__.py", line 90, in import_module
2024-05-15T06:37:11:           return _bootstrap._gcd_import(name[level:], package, level)
2024-05-15T06:37:11:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1304, in _find_and_load_unlocked
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap_external>", line 994, in exec_module
2024-05-15T06:37:11:         File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
2024-05-15T06:37:11:         File "/tmp/pip-build-env-boxwps2r/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
2024-05-15T06:37:11:           import setuptools.version
2024-05-15T06:37:11:         File "/tmp/pip-build-env-boxwps2r/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
2024-05-15T06:37:11:           import pkg_resources
2024-05-15T06:37:11:         File "/tmp/pip-build-env-boxwps2r/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
2024-05-15T06:37:11:           register_finder(pkgutil.ImpImporter, find_on_path)
2024-05-15T06:37:11:                           ^^^^^^^^^^^^^^^^^^^
2024-05-15T06:37:11:       AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
2024-05-15T06:37:11:       [end of output]
2024-05-15T06:37:11:   
2024-05-15T06:37:11:   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-05-15T06:37:11: error: subprocess-exited-with-error

2024-05-15T06:37:11: × Getting requirements to build wheel did not run successfully.
2024-05-15T06:37:11: │ exit code: 1
2024-05-15T06:37:11: ╰─> See above for output.

source: https://github.com/recommenders-team/recommenders/actions/runs/9090676506/job/24984293755

daviddavo commented 2 months ago

pkgutil.ImpImporter and ImpLoader have been deprecated since python 3.3, and they removed it in 3.12

As the problem is with setuptools, perhaps upgrading it or forcing a more recent version should suffice. I think cornac fails just because it is the first thing to be installed with that method.

According to that issue, changing the dependency to setuptools>=67 should work, but this might create a cascade of having to upgrade numpy, among other things.

Note: the distutils package was removed, but now is provided by setuptools. setuptools is no longer automatically installed by venv, but that's no problem because it's already specified in the pyproject

daviddavo commented 2 months ago

Now the requirements are fine, but it errors after installing build dependencies and I can't see why

2024-05-15T06:37:11:   Installing build dependencies: started
2024-05-15T06:37:11:   Installing build dependencies: finished with status 'done'
2024-05-15T06:37:11:   Getting requirements to build wheel: started
2024-05-15T06:37:11:   Getting requirements to build wheel: finished with status 'error'

2024-05-15T06:37:11: failed
2024-05-15T06:37:11: 
2024-05-15T06:37:11: CondaEnvException: Pip failed

Source: https://github.com/recommenders-team/recommenders/actions/runs/9090676506/job/24984293755

miguelgfierro commented 2 months ago

It seems the error is the same:

ErrorResponse 
***
    "error": ***
        "message": "Activity Failed:\n***\n    \"error\": ***\n        \"code\": \"UserError\",\n        \"message\": \"Image build failed. For more details, check log file azureml-logs/20_image_build_log.txt.\",\n        \"messageFormat\": \"Image build failed. For more details, check log file ***ArtifactPath***.\",\n        \"messageParameters\": ***\n            \"ArtifactPath\": \"azureml-logs/20_image_build_log.txt\"\n        ***,\n        \"details\": [],\n        \"innerError\": ***\n            \"code\": \"BadArgument\",\n            \"innerError\": ***\n                \"code\": \"ImageBuildFailure\"\n            ***\n        ***\n    ***,\n    \"correlation\": ***\n        \"operation\": \"0eb40f8a3a5a1b8bdc7eeb65cbe1a34b\",\n        \"request\": \"c0470de765320b12\"\n    ***,\n    \"environment\": \"eastus\",\n    \"location\": \"eastus\",\n    \"time\": \"2024-05-15T08:53:48.984678Z\",\n    \"componentName\": \"RunHistory\"\n***"
    ***
***
2024-05-15T08:53:17:   error: subprocess-exited-with-error
2024-05-15T08:53:17:   
2024-05-15T08:53:17:   × Getting requirements to build wheel did not run successfully.
2024-05-15T08:53:17:   │ exit code: 1
2024-05-15T08:53:17:   ╰─> [33 lines of output]
2024-05-15T08:53:17:       Traceback (most recent call last):
2024-05-15T08:53:17:         File "/azureml-envs/azureml_5e1c99080a501af5b78a0863f3002a98/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
2024-05-15T08:53:17:           main()
2024-05-15T08:53:17:         File "/azureml-envs/azureml_5e1c99080a501af5b78a0863f3002a98/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
2024-05-15T08:53:17:           json_out['return_val'] = hook(**hook_input['kwargs'])
2024-05-15T08:53:17:                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T08:53:17:         File "/azureml-envs/azureml_5e1c99080a501af5b78a0863f3002a98/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 112, in get_requires_for_build_wheel
2024-05-15T08:53:17:           backend = _build_backend()
2024-05-15T08:53:17:                     ^^^^^^^^^^^^^^^^
2024-05-15T08:53:17:         File "/azureml-envs/azureml_5e1c99080a501af5b78a0863f3002a98/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
2024-05-15T08:53:17:           obj = import_module(mod_path)
2024-05-15T08:53:17:                 ^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T08:53:17:         File "/azureml-envs/azureml_5e1c99080a501af5b78a0863f[300](https://github.com/recommenders-team/recommenders/actions/runs/9092635078/job/24989789632#step:3:307)2a98/lib/python3.12/importlib/__init__.py", line 90, in import_module
2024-05-15T08:53:17:           return _bootstrap._gcd_import(name[level:], package, level)
2024-05-15T08:53:17:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1[304](https://github.com/recommenders-team/recommenders/actions/runs/9092635078/job/24989789632#step:3:311), in _find_and_load_unlocked
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap_external>", line 994, in exec_module
2024-05-15T08:53:17:         File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
2024-05-15T08:53:17:         File "/tmp/pip-build-env-yq5nkn2j/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 16, in <module>
2024-05-15T08:53:17:           import setuptools.version
2024-05-15T08:53:17:         File "/tmp/pip-build-env-yq5nkn2j/overlay/lib/python3.12/site-packages/setuptools/version.py", line 1, in <module>
2024-05-15T08:53:17:           import pkg_resources
2024-05-15T08:53:17:         File "/tmp/pip-build-env-yq5nkn2j/overlay/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
2024-05-15T08:53:17:           register_finder(pkgutil.ImpImporter, find_on_path)
2024-05-15T08:53:17:                           ^^^^^^^^^^^^^^^^^^^
2024-05-15T08:53:17:       AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
2024-05-15T08:53:17:       [end of output]
2024-05-15T08:53:17:   
2024-05-15T08:53:17:   note: This error originates from a subprocess, and is likely not a problem with pip.
2024-05-15T08:53:17: error: subprocess-exited-with-error

This seems like a long problem to debug. In my experience, it is better to debug it first in local. @daviddavo would you have time to take a look?

daviddavo commented 2 months ago

Yeah, I can try and give it some time next week

daviddavo commented 2 months ago

I can't install lightfm on Python 3.12, this affects recommenders too

daviddavo commented 1 month ago

Forked the repository and made a PR

The package can be installed from my repository:

pip install git+https://github.com/daviddavo/lightfm

If the pull requests wasn't approved, I can transfer ownership of this repository to recommenders-team, and then that github repository could be used in the requirements.

daviddavo commented 1 month ago

By changing temporarily changing the requirements, it works on my machine. Let's see if it works on the test groups

daviddavo commented 1 month ago

The following command works on my machine:

python -m venv venv
source venv/bin/activate
pip install git+https://github.com/recommenders-team/recommenders@python312

But perhaps my system comes with some build dependencies that are not installed in the test groups...

Tomorrow I'll try and use conda to install the package on my machine and see how it goes

miguelgfierro commented 1 month ago

hey @daviddavo we were talking today in the weekly meeting about this issue. Do you have any news?

FYI @SimonYansenZhao @anargyri

daviddavo commented 2 weeks ago

Ready for review just to run tests

miguelgfierro commented 2 weeks ago

reruning the pr gate: https://github.com/recommenders-team/recommenders/actions/runs/9747599995

daviddavo commented 2 weeks ago

Trying again on my system it seems that the problem is related to tensorflow

Tensorflow added support for 3.12 exactly in release 2.16...

On of the breaking changes is the removal of tf.estimator, which is used in tf_utils.py and wide_deep_utils.py

I will follow this guide:

daviddavo commented 2 weeks ago

It is now failing because its installing numpy 1.23, which did not have support yet for Python 3.12.

It is installing numpy 1.23 because of pymanopt

2024-07-03T11:12:14: Collecting numpy>=1.16 (from pymanopt@ https://github.com/pymanopt/pymanopt/archive/fb36a272cdeecb21992cfd9271eb82baafeb316d.zip->-r /azureml-environment-setup/condaenv.dxlry2qs.requirements.txt (line 2))
2024-07-03T11:12:14:   Downloading numpy-1.23.5.tar.gz (10.7 MB)

Still, pymanopt doesn't support python 3.12 yet

daviddavo commented 1 week ago

The current error seems to be related to numpy installation

Extract of the log ``` 2024-07-03T14:36:10.1109567Z 2024-07-03T14:35:41: ╰─> [103 lines of output] 2024-07-03T14:36:10.1111199Z 2024-07-03T14:35:41: Ignoring numpy: markers 'python_version < "3.9"' don't match your environment 2024-07-03T14:36:10.1112762Z 2024-07-03T14:35:41: Collecting cython>=0.29 2024-07-03T14:36:10.1114611Z 2024-07-03T14:35:41: Using cached Cython-3.0.10-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.2 kB) 2024-07-03T14:36:10.1116376Z 2024-07-03T14:35:41: Collecting numpy==1.19.4 2024-07-03T14:36:10.1117669Z 2024-07-03T14:35:41: Downloading numpy-1.19.4.zip (7.3 MB) 2024-07-03T14:36:10.1119728Z 2024-07-03T14:35:41: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 58.6 MB/s eta 0:00:00 2024-07-03T14:36:10.1121426Z 2024-07-03T14:35:41: Installing build dependencies: started 2024-07-03T14:36:10.1123058Z 2024-07-03T14:35:41: Installing build dependencies: finished with status 'done' 2024-07-03T14:36:10.1124735Z 2024-07-03T14:35:41: Getting requirements to build wheel: started 2024-07-03T14:36:10.1126352Z 2024-07-03T14:35:41: Getting requirements to build wheel: finished with status 'done' 2024-07-03T14:36:10.1127780Z 2024-07-03T14:35:41: ERROR: Exception: 2024-07-03T14:36:10.1128933Z 2024-07-03T14:35:41: Traceback (most recent call last): 2024-07-03T14:36:10.1131370Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper 2024-07-03T14:36:10.1133566Z 2024-07-03T14:35:41: status = run_func(*args) 2024-07-03T14:36:10.1134794Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1136954Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper 2024-07-03T14:36:10.1139135Z 2024-07-03T14:35:41: return func(self, options, args) 2024-07-03T14:36:10.1140431Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1142908Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 377, in run 2024-07-03T14:36:10.1145006Z 2024-07-03T14:35:41: requirement_set = resolver.resolve( 2024-07-03T14:36:10.1146426Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1149160Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve 2024-07-03T14:36:10.1151810Z 2024-07-03T14:35:41: result = self._result = resolver.resolve( 2024-07-03T14:36:10.1153282Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1155698Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve 2024-07-03T14:36:10.1158187Z 2024-07-03T14:35:41: state = resolution.resolve(requirements, max_rounds=max_rounds) 2024-07-03T14:36:10.1159995Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1162423Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 397, in resolve 2024-07-03T14:36:10.1164759Z 2024-07-03T14:35:41: self._add_to_criteria(self.state.criteria, r, parent=None) 2024-07-03T14:36:10.1167362Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria 2024-07-03T14:36:10.1169184Z 2024-07-03T14:35:41: if not criterion.candidates: 2024-07-03T14:36:10.1170018Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1171324Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__ 2024-07-03T14:36:10.1172619Z 2024-07-03T14:35:41: return bool(self._sequence) 2024-07-03T14:36:10.1173458Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1174884Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__ 2024-07-03T14:36:10.1176211Z 2024-07-03T14:35:41: return any(self) 2024-07-03T14:36:10.1176876Z 2024-07-03T14:35:41: ^^^^^^^^^ 2024-07-03T14:36:10.1178356Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in 2024-07-03T14:36:10.1179866Z 2024-07-03T14:35:41: return (c for c in iterator if id(c) not in self._incompatible_ids) 2024-07-03T14:36:10.1180927Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1182674Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built 2024-07-03T14:36:10.1184201Z 2024-07-03T14:35:41: candidate = func() 2024-07-03T14:36:10.1184854Z 2024-07-03T14:35:41: ^^^^^^ 2024-07-03T14:36:10.1186380Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link 2024-07-03T14:36:10.1187832Z 2024-07-03T14:35:41: self._link_candidate_cache[link] = LinkCandidate( 2024-07-03T14:36:10.1188833Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1190275Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in __init__ 2024-07-03T14:36:10.1191598Z 2024-07-03T14:35:41: super().__init__( 2024-07-03T14:36:10.1192958Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__ 2024-07-03T14:36:10.1194597Z 2024-07-03T14:35:41: self.dist = self._prepare() 2024-07-03T14:36:10.1195570Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1197042Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare 2024-07-03T14:36:10.1198447Z 2024-07-03T14:35:41: dist = self._prepare_distribution() 2024-07-03T14:36:10.1199297Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1200840Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution 2024-07-03T14:36:10.1202432Z 2024-07-03T14:35:41: return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) 2024-07-03T14:36:10.1203535Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1205069Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement 2024-07-03T14:36:10.1206606Z 2024-07-03T14:35:41: return self._prepare_linked_requirement(req, parallel_builds) 2024-07-03T14:36:10.1207538Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1209110Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement 2024-07-03T14:36:10.1210492Z 2024-07-03T14:35:41: dist = _get_prepared_distribution( 2024-07-03T14:36:10.1211361Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1212780Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution 2024-07-03T14:36:10.1214250Z 2024-07-03T14:35:41: abstract_dist.prepare_distribution_metadata( 2024-07-03T14:36:10.1215848Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 54, in prepare_distribution_metadata 2024-07-03T14:36:10.1217181Z 2024-07-03T14:35:41: self._install_build_reqs(finder) 2024-07-03T14:36:10.1218654Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 124, in _install_build_reqs 2024-07-03T14:36:10.1220003Z 2024-07-03T14:35:41: build_reqs = self._get_build_requires_wheel() 2024-07-03T14:36:10.1220982Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1222774Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 101, in _get_build_requires_wheel 2024-07-03T14:36:10.1224576Z 2024-07-03T14:35:41: return backend.get_requires_for_build_wheel() 2024-07-03T14:36:10.1225436Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1226988Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_internal/utils/misc.py", line 751, in get_requires_for_build_wheel 2024-07-03T14:36:10.1228422Z 2024-07-03T14:35:41: return super().get_requires_for_build_wheel(config_settings=cs) 2024-07-03T14:36:10.1229433Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1231109Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel 2024-07-03T14:36:10.1232888Z 2024-07-03T14:35:41: return self._call_hook('get_requires_for_build_wheel', *** 2024-07-03T14:36:10.1233824Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1235305Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook 2024-07-03T14:36:10.1236749Z 2024-07-03T14:35:41: raise BackendUnavailable(data.get('traceback', '')) 2024-07-03T14:36:10.1237885Z 2024-07-03T14:35:41: pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last): 2024-07-03T14:36:10.1239637Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend 2024-07-03T14:36:10.1241005Z 2024-07-03T14:35:41: obj = import_module(mod_path) 2024-07-03T14:36:10.1241880Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1243066Z 2024-07-03T14:35:41: File "/azureml-envs/azureml_9eab27162e8e2d6f98a42f938c8ebacc/lib/python3.12/importlib/__init__.py", line 90, in import_module 2024-07-03T14:36:10.1244416Z 2024-07-03T14:35:41: return _bootstrap._gcd_import(name[level:], package, level) 2024-07-03T14:36:10.1245335Z 2024-07-03T14:35:41: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-07-03T14:36:10.1246400Z 2024-07-03T14:35:41: File "", line 1381, in _gcd_import 2024-07-03T14:36:10.1247423Z 2024-07-03T14:35:41: File "", line 1354, in _find_and_load 2024-07-03T14:36:10.1248563Z 2024-07-03T14:35:41: File "", line 1304, in _find_and_load_unlocked 2024-07-03T14:36:10.1249692Z 2024-07-03T14:35:41: File "", line 488, in _call_with_frames_removed 2024-07-03T14:36:10.1250840Z 2024-07-03T14:35:41: File "", line 1381, in _gcd_import 2024-07-03T14:36:10.1251876Z 2024-07-03T14:35:41: File "", line 1354, in _find_and_load 2024-07-03T14:36:10.1253009Z 2024-07-03T14:35:41: File "", line 1325, in _find_and_load_unlocked 2024-07-03T14:36:10.1254082Z 2024-07-03T14:35:41: File "", line 929, in _load_unlocked 2024-07-03T14:36:10.1255234Z 2024-07-03T14:35:41: File "", line 994, in exec_module 2024-07-03T14:36:10.1256409Z 2024-07-03T14:35:41: File "", line 488, in _call_with_frames_removed 2024-07-03T14:36:10.1257766Z 2024-07-03T14:35:41: File "/tmp/pip-build-env-qo_olf7o/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 18, in 2024-07-03T14:36:10.1259047Z 2024-07-03T14:35:41: from setuptools.extern.six import PY3, string_types 2024-07-03T14:36:10.1260092Z 2024-07-03T14:35:41: ModuleNotFoundError: No module named 'setuptools.extern.six' 2024-07-03T14:36:10.1260928Z 2024-07-03T14:35:41: 2024-07-03T14:36:10.1261489Z 2024-07-03T14:35:41: [end of output] 2024-07-03T14:36:10.1262123Z 2024-07-03T14:35:41: 2024-07-03T14:36:10.1263291Z 2024-07-03T14:35:41: note: This error originates from a subprocess, and is likely not a problem with pip. 2024-07-03T14:36:10.1264404Z 2024-07-03T14:35:41: error: subprocess-exited-with-error ```

Why is it installing numpy 1.19 if it is not compatible with Python 3.12? It says Ignoring numpy: markers 'python_version < "3.9"' don't match your environment. Perhaps I'm missing something of how the build system works...

miguelgfierro commented 1 week ago

We have numpy>=1.19 https://github.com/recommenders-team/recommenders/blob/main/setup.py#L126

I don't understand why it is fixing it to 1.19.4 maybe it is because another dependency? What do you think @daviddavo @SimonYansenZhao

Numpy has recently move to 2 https://pypi.org/project/numpy/#history not sure if we will have problems from this front