search

Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
16.9k stars 4.13k forks source link

release-21-branch numpy can't install successful. #6047

Closed Feipian closed 6 months ago

Feipian commented 8 months ago

When you install mlagent-env using python virtual environment according to the install instruction document, you will encounter numpy can't install, This is because python 3.10.12 does not support the numpy version of setup.py in the ml-agents-envs folder, so setup.py needs to be changed

import os
import sys
from setuptools import setup, find_packages
from setuptools.command.install import install
import mlagents_envs

VERSION = mlagents_envs.__version__
EXPECTED_TAG = mlagents_envs.__release_tag__

here = os.path.abspath(os.path.dirname(__file__))

class VerifyVersionCommand(install):
"" "
Custom command to verify that the git tag is the expected one for the release.
Originally based on https://circleci.com/blog/continuously-deploying-python-packages-to-pypi-with-circleci/
This differs slightly because our tags and versions are different.
"" "

description = "verify that the git tag matches our version"

def run(self):
tag = os.getenv("GITHUB_REF", "NO GITHUB TAG!" ).replace("refs/tags/", "")

if tag ! = EXPECTED_TAG:
info = "Git tag: {} does not match the expected tag of this app: {}".format(
tag, EXPECTED_TAG
)
sys.exit(info)

# Get the long description from the README file
with open(os.path.join(here, "README.md"), encoding="utf-8") as f:
long_description = f.read()

setup(
name="mlagents_envs",
version=VERSION,
description="Unity Machine Learning Agents Interface",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/Unity-Technologies/ml-agents",
author="Unity Technologies",
author_email="ML-Agents@unity3d.com",
classifiers=[
"Intended Audience :: Developers",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
"License :: OSI Approved :: Apache Software License",
Programming Language :: Python :: 3.10",
].
packages=find_packages(
exclude=["*.tests", "*.tests.*", "tests.*", "tests", "colabs", "*.ipynb"]
),
zip_safe=False,
install_requires=[
"cloudpickle",
"grpcio> = 1.11.0, & lt; = 1.48.2 ",
"Pillow> = 2 ",
"protobuf> = 3.6, & lt; 3.20 ",
"pyyaml> = 3.1.0 ",
"gym> = 0.21.0 ",
"Pettingzoo = = 1.15.0",
"numpy==1.22.4", # change the numpy version, because default version will display 'can't successful install numpy'
"filelock> = 3.4.0 ",
].
python_requires="> = 3.10.1, & lt; = 3.10.12 ",
# TODO: Remove this once mypy stops having spurious setuptools issues.
cmdclass={"verify": VerifyVersionCommand},  # type: ignore
)

Can run

AbhiG7 commented 8 months ago

Hey, I've been having the same issue as you so I tried changing the numpy version in the file like you showed and it allowed me to install the packages but now I get another error and I was wondering if you were getting it too. I get an error that says: Warning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xf (function operator ()). Have you gotten this before and do you know how to fix this?

Feipian commented 8 months ago

Maybe you come cross this problem and try:https://stackoverflow.com/questions/71986643/userwarning-failed-to-initialize-numpy-module-compiled-against-api-version-0xf

Feipian commented 8 months ago

Hey, I've been having the same issue as you so I tried changing the numpy version in the file like you showed and it allowed me to install the packages but now I get another error and I was wondering if you were getting it too. I get an error that says: Warning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xf (function operator ()). Have you gotten this before and do you know how to fix this?

General flow:

Feipian commented 8 months ago

Hey, I've been having the same issue as you so I tried changing the numpy version in the file like you showed and it allowed me to install the packages but now I get another error and I was wondering if you were getting it too. I get an error that says: Warning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xf (function operator ()). Have you gotten this before and do you know how to fix this?

image

I change numpy version to 1.22.4 because it support python3.10.12

AbhiG7 commented 8 months ago

I followed those steps but met with many issues with the numpy version. I ended up finding the right solution for me, and it was to change the numpy verison in the setup.py file in the ml-agents-envs folder to "numpy>=1.21.2,<1.24.0" rather than the other suggested ways. It's shown in this screenshot: image which is taken from this branch of the ml-agents repo: https://github.com/Unity-Technologies/ml-agents/blob/develop/ml-agents-envs/setup.py.

Thanks for the help, though!

ICOnce commented 8 months ago

I am having a problem that causes this error when running mlagents-learn --help after having changed the numpy version and successfully gotten it to install image

ICOnce commented 8 months ago

I am having a problem that causes this error when running mlagents-learn --help after having changed the numpy version and successfully gotten it to install image

I seem to have solved, that issue, but now it gives a numpy error upon typing mlagents-learn

ashwinsnambiar commented 7 months ago

I had faced a similar issue. It seemed to be caused by an incorrect numpy version. I removed the existing numpy and mlagents packages from the virtual env and reinstalled everything by using the setup.py from a branch release. Please check the following link for the same. https://github.com/Unity-Technologies/ml-agents/issues/6008#issuecomment-1814741015

IvanVeridian commented 7 months ago

I had this issue as well and fixed it by following @ashwinsnambiar recommendation of following @onurkurum https://github.com/Unity-Technologies/ml-agents/issues/6008#issuecomment-1815607844

"Workaround: Changed "numpy==1.21.2", to "numpy==1.23.3",

in the file path-to-ml-agents\ml-agents-envs\setup.py"

ICOnce commented 7 months ago

I ended up fixing all the issues, here is a document about how I fixed everything MLAgents setup guide.pdf

Feipian commented 7 months ago

I ended up fixing all the issues, here is a document about how I fixed everything MLAgents setup guide.pdf

Very detailed teaching, thank you!!

ICOnce commented 7 months ago

I ended up fixing all the issues, here is a document about how I fixed everything MLAgents setup guide.pdf

Very detailed teaching, thank you!!

NP!

wanghang2000221 commented 6 months ago

I do the same but face this problem

TypeError: CCompiler_spawn() got an unexpected keyword argument 'env' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

ICOnce commented 6 months ago

I do the same but face this problem

TypeError: CCompiler_spawn() got an unexpected keyword argument 'env' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

You might need to also change the Python version in the other setup file

wanghang2000221 commented 6 months ago

Very thanks for reply! But when I change the setup.py both in ml-agents and ml-agents-envs, I will face another bugERROR: Cannot install mlagents and mlagents==1.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:     mlagents 1.0.0 depends on numpy>=1.23.3     mlagents-envs 1.0.0 depends on numpy==1.21.2

To fix this you could try to:

  1. loosen the range of package versions you've specified
  2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2024年3月7日(星期四) 下午3:22 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [Unity-Technologies/ml-agents] release-21-branch numpy can't install successful. (Issue #6047)

I do the same but face this problem

TypeError: CCompiler_spawn() got an unexpected keyword argument 'env' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

You might need to also change the Python version in the other setup file

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

punkouter24 commented 6 months ago

anaconda navigator env/ create / name is env3103 / slected python 3.10.13 open terminal goto downloads cd .\ml-agents\ git clone --branch release_21 https://github.com/Unity-Technologies/ml-agents.git conda install pytorch=1.13.1 -c pytorch pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117

mlagents env setup.py- install_requires=[ "cloudpickle", "grpcio>=1.11.0,<=1.48.2", "Pillow>=4.2.1", "protobuf>=3.6,<3.20", "pyyaml>=3.1.0", "gym>=0.21.0", "pettingzoo==1.15.0", "numpy==1.21.3", "filelock>=3.4.0", ], python_requires=">=3.10.1,<=3.10.13",

mlagents setup.py- install_requires=[

Test-only dependencies should go in test_requirements.txt, not here.

    "grpcio>=1.11.0,<=1.48.2",
    "h5py>=2.9.0",
    f"mlagents_envs=={VERSION}",
    "numpy>=1.23.3",
    "Pillow>=4.2.1",
    "protobuf>=3.6,<3.20",
    "pyyaml>=3.1.0",
    "torch>=1.13.1",
    "tensorboard>=2.14",
    # adding six explicit dependency since tensorboard needs it but doesn't declare it as a dep
    "six>=1.16",
    # cattrs 1.1.0 dropped support for python 3.6, but 1.0.0 doesn't work for python 3.9
    # Since there's no version that supports both, we have to draw the line somewhere.
    "cattrs>=1.1.0,<1.7; python_version>='3.8'",
    "attrs>=19.3.0",
    "huggingface_hub>=0.14",
    'pypiwin32==223;platform_system=="Windows"',
    "onnx==1.12.0",
],
python_requires=">=3.10.1,<=3.10.13",

python -m pip install ./ml-agents-envs

ERROR: Cannot install mlagents, mlagents-envs==1.0.0 and mlagents==1.0.0 because these package versions have conflicting dependencies. The conflict is caused by: mlagents 1.0.0 depends on protobuf<3.21 and >=3.6 mlagents-envs 1.0.0 depends on protobuf<3.20 and >=3.6 onnx 1.15.0 depends on protobuf>=3.20.2 To fix this you could try to: 1. loosen the range of package versions you've specified 2. remove package versions to allow pip attempt to solve the dependency conflict ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
punkouter24 commented 6 months ago

it broke there for me

AlexGama11 commented 6 months ago

Hi Guys, having an issue with this as well. Been trying to install it for class. We're required to use Python 3.10.11, and I changed the setup.py to use numpy 1.23.1 (version teacher said we gotta use), however, no matter what I do, when I run python -m pip install mlagents==1.0.0 in the console, it always installs numpy 1.21.2, I'll post the setup files and command below:

Console:

(mlagents) D:\UnityHiren\MLAgents\ml-agents>python -m pip install mlagents==1.0.0 Collecting mlagents==1.0.0 Downloading mlagents-1.0.0-py3-none-any.whl (171 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 171.3/171.3 kB 3.4 MB/s eta 0:00:00 Requirement already satisfied: tensorboard>=2.14 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (2.16.2) Requirement already satisfied: onnx==1.12.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (1.12.0) Requirement already satisfied: numpy<2.0,>=1.13.3 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (1.23.1) Requirement already satisfied: attrs>=19.3.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (23.2.0) Requirement already satisfied: protobuf<3.20,>=3.6 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (3.19.6) Requirement already satisfied: torch>=1.13.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (2.2.1) Requirement already satisfied: pyyaml>=3.1.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (6.0.1) Requirement already satisfied: Pillow>=4.2.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (10.2.0) Requirement already satisfied: six>=1.16 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (1.16.0) Requirement already satisfied: grpcio<=1.48.2,>=1.11.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (1.48.2) Collecting mlagents-envs==1.0.0 Downloading mlagents_envs-1.0.0-py3-none-any.whl (89 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 89.5/89.5 kB ? eta 0:00:00 Requirement already satisfied: huggingface-hub>=0.14 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (0.21.4) Requirement already satisfied: h5py>=2.9.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (3.10.0) Requirement already satisfied: cattrs<1.7,>=1.1.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (1.5.0) Requirement already satisfied: pypiwin32==223 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents==1.0.0) (223) Requirement already satisfied: gym>=0.21.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents-envs==1.0.0->mlagents==1.0.0) (0.26.2) Collecting numpy<2.0,>=1.13.3 Downloading numpy-1.21.2.zip (10.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.3/10.3 MB 8.6 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: filelock>=3.4.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents-envs==1.0.0->mlagents==1.0.0) (3.13.1) Requirement already satisfied: pettingzoo==1.15.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents-envs==1.0.0->mlagents==1.0.0) (1.15.0) Requirement already satisfied: cloudpickle in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from mlagents-envs==1.0.0->mlagents==1.0.0) (3.0.0) Requirement already satisfied: typing-extensions>=3.6.2.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from onnx==1.12.0->mlagents==1.0.0) (4.10.0) Requirement already satisfied: pywin32>=223 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from pypiwin32==223->mlagents==1.0.0) (306) Requirement already satisfied: packaging>=20.9 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from huggingface-hub>=0.14->mlagents==1.0.0) (24.0) Requirement already satisfied: tqdm>=4.42.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from huggingface-hub>=0.14->mlagents==1.0.0) (4.66.2) Requirement already satisfied: requests in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from huggingface-hub>=0.14->mlagents==1.0.0) (2.31.0) Requirement already satisfied: fsspec>=2023.5.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from huggingface-hub>=0.14->mlagents==1.0.0) (2024.2.0) Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tensorboard>=2.14->mlagents==1.0.0) (0.7.2) Requirement already satisfied: absl-py>=0.4 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tensorboard>=2.14->mlagents==1.0.0) (2.1.0) Requirement already satisfied: markdown>=2.6.8 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tensorboard>=2.14->mlagents==1.0.0) (3.5.2) Requirement already satisfied: setuptools>=41.0.0 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tensorboard>=2.14->mlagents==1.0.0) (65.5.0) Requirement already satisfied: werkzeug>=1.0.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tensorboard>=2.14->mlagents==1.0.0) (3.0.1) Requirement already satisfied: sympy in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from torch>=1.13.1->mlagents==1.0.0) (1.12) Requirement already satisfied: networkx in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from torch>=1.13.1->mlagents==1.0.0) (3.2.1) Requirement already satisfied: jinja2 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from torch>=1.13.1->mlagents==1.0.0) (3.1.3) Requirement already satisfied: gym-notices>=0.0.4 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from gym>=0.21.0->mlagents-envs==1.0.0->mlagents==1.0.0) (0.0.8) Requirement already satisfied: colorama in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from tqdm>=4.42.1->huggingface-hub>=0.14->mlagents==1.0.0) (0.4.6) Requirement already satisfied: MarkupSafe>=2.1.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from werkzeug>=1.0.1->tensorboard>=2.14->mlagents==1.0.0) (2.1.5) Requirement already satisfied: certifi>=2017.4.17 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from requests->huggingface-hub>=0.14->mlagents==1.0.0) (2024.2.2) Requirement already satisfied: idna<4,>=2.5 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from requests->huggingface-hub>=0.14->mlagents==1.0.0) (3.6) Requirement already satisfied: charset-normalizer<4,>=2 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from requests->huggingface-hub>=0.14->mlagents==1.0.0) (3.3.2) Requirement already satisfied: urllib3<3,>=1.21.1 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from requests->huggingface-hub>=0.14->mlagents==1.0.0) (2.2.1) Requirement already satisfied: mpmath>=0.19 in d:\unityhiren\mlagents\ml-agents\mlagents\lib\site-packages (from sympy->torch>=1.13.1->mlagents==1.0.0) (1.3.0) Building wheels for collected packages: numpy Building wheel for numpy (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for numpy (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [271 lines of output] setup.py:63: RuntimeWarning: NumPy 1.21.2 may not yet support Python 3.10. warnings.warn( Running from numpy source directory. Processing numpy/random_bounded_integers.pxd.in Processing numpy/random\bit_generator.pyx Processing numpy/random\mtrand.pyx Processing numpy/random_bounded_integers.pyx.in Processing numpy/random_common.pyx Processing numpy/random_generator.pyx Processing numpy/random_mt19937.pyx Processing numpy/random_pcg64.pyx Processing numpy/random_philox.pyx Processing numpy/random_sfc64.pyx Cythonizing sources blas_opt_info: blas_mkl_info: No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils customize MSVCCompiler libraries mkl_rt not found in ['D:\UnityHiren\MLAgents\ml-agents\mlagents\lib', 'C:\'] NOT AVAILABLE

  blis_info:
    libraries blis not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  openblas_info:
    libraries openblas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
  get_default_fcompiler: matching types: '['gnu', 'intelv', 'absoft', 'compaqv', 'intelev', 'gnu95', 'g95', 'intelvem', 'intelem', 'flang']'
  customize GnuFCompiler
  Could not locate executable g77
  Could not locate executable f77
  customize IntelVisualFCompiler
  Could not locate executable ifort
  Could not locate executable ifl
  customize AbsoftFCompiler
  Could not locate executable f90
  customize CompaqVisualFCompiler
  Could not locate executable DF
  customize IntelItaniumVisualFCompiler
  Could not locate executable efl
  customize Gnu95FCompiler
  Could not locate executable gfortran
  Could not locate executable f95
  customize G95FCompiler
  Could not locate executable g95
  customize IntelEM64VisualFCompiler
  customize IntelEM64TFCompiler
  Could not locate executable efort
  Could not locate executable efc
  customize PGroupFlangCompiler
  Could not locate executable flang
  don't know how to compile Fortran code on platform 'nt'
    NOT AVAILABLE

  accelerate_info:
    NOT AVAILABLE

  atlas_3_10_blas_threads_info:
  Setting PTATLAS=ATLAS
    libraries tatlas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  atlas_3_10_blas_info:
    libraries satlas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  atlas_blas_threads_info:
  Setting PTATLAS=ATLAS
    libraries ptf77blas,ptcblas,atlas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  atlas_blas_info:
    libraries f77blas,cblas,atlas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\system_info.py:2026: UserWarning:
      Optimized (vendor) Blas libraries are not found.
      Falls back to netlib Blas library which has worse performance.
      A better performance should be easily gained by switching
      Blas library.
    if self._calc_info(blas):
  blas_info:
    libraries blas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\system_info.py:2026: UserWarning:
      Blas (http://www.netlib.org/blas/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [blas]) or by setting
      the BLAS environment variable.
    if self._calc_info(blas):
  blas_src_info:
    NOT AVAILABLE

  D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\system_info.py:2026: UserWarning:
      Blas (http://www.netlib.org/blas/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [blas_src]) or by setting
      the BLAS_SRC environment variable.
    if self._calc_info(blas):
    NOT AVAILABLE

  non-existing path in 'numpy\\distutils': 'site.cfg'
  lapack_opt_info:
  lapack_mkl_info:
    libraries mkl_rt not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  openblas_lapack_info:
    libraries openblas not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  openblas_clapack_info:
    libraries openblas,lapack not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  flame_info:
    libraries flame not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  atlas_3_10_threads_info:
  Setting PTATLAS=ATLAS
    libraries lapack_atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries tatlas,tatlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries lapack_atlas not found in C:\
    libraries tatlas,tatlas not found in C:\
  <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
    NOT AVAILABLE

  atlas_3_10_info:
    libraries lapack_atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries satlas,satlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries lapack_atlas not found in C:\
    libraries satlas,satlas not found in C:\
  <class 'numpy.distutils.system_info.atlas_3_10_info'>
    NOT AVAILABLE

  atlas_threads_info:
  Setting PTATLAS=ATLAS
    libraries lapack_atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries ptf77blas,ptcblas,atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries lapack_atlas not found in C:\
    libraries ptf77blas,ptcblas,atlas not found in C:\
  <class 'numpy.distutils.system_info.atlas_threads_info'>
    NOT AVAILABLE

  atlas_info:
    libraries lapack_atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries f77blas,cblas,atlas not found in D:\UnityHiren\MLAgents\ml-agents\mlagents\lib
    libraries lapack_atlas not found in C:\
    libraries f77blas,cblas,atlas not found in C:\
  <class 'numpy.distutils.system_info.atlas_info'>
    NOT AVAILABLE

  lapack_info:
    libraries lapack not found in ['D:\\UnityHiren\\MLAgents\\ml-agents\\mlagents\\lib', 'C:\\']
    NOT AVAILABLE

  D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\system_info.py:1858: UserWarning:
      Lapack (http://www.netlib.org/lapack/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [lapack]) or by setting
      the LAPACK environment variable.
    return getattr(self, '_calc_info_{}'.format(name))()
  lapack_src_info:
    NOT AVAILABLE

  D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\system_info.py:1858: UserWarning:
      Lapack (http://www.netlib.org/lapack/) sources not found.
      Directories to search for the sources can be specified in the
      numpy/distutils/site.cfg file (section [lapack_src]) or by setting
      the LAPACK_SRC environment variable.
    return getattr(self, '_calc_info_{}'.format(name))()
    NOT AVAILABLE

  numpy_linalg_lapack_lite:
    FOUND:
      language = c
      define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]

  Warning: attempted relative import with no known parent package
  D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\dist.py:275: UserWarning: Unknown distribution option: 'define_macros'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running config_cc
  unifing config_cc, config, build_clib, build_ext, build commands --compiler options
  running config_fc
  unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
  running build_src
  build_src
  building py_modules sources
  creating build
  creating build\src.win-amd64-3.10
  creating build\src.win-amd64-3.10\numpy
  creating build\src.win-amd64-3.10\numpy\distutils
  building library "npymath" sources
  Traceback (most recent call last):
    File "D:\UnityHiren\MLAgents\ml-agents\mlagents\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
    File "D:\UnityHiren\MLAgents\ml-agents\mlagents\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "D:\UnityHiren\MLAgents\ml-agents\mlagents\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\build_meta.py", line 211, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\build_meta.py", line 197, in _build_with_temp_dir
      self.run_setup()
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\build_meta.py", line 248, in run_setup
      super(_BuildMetaLegacyBackend,
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\build_meta.py", line 142, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 448, in <module>
      setup_package()
    File "setup.py", line 440, in setup_package
      setup(**metadata)
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\core.py", line 169, in setup
      return old_setup(**new_attr)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\__init__.py", line 165, in setup
      return distutils.core.setup(**attrs)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\core.py", line 148, in setup
      dist.run_commands()
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 967, in run_commands
      self.run_command(cmd)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
      cmd_obj.run()
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\wheel\bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
      cmd_obj.run()
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\build.py", line 61, in run
      old_build.run(self)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\command\build.py", line 135, in run
      self.run_command(cmd_name)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\dist.py", line 986, in run_command
      cmd_obj.run()
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\build_src.py", line 144, in run
      self.build_sources()
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\build_src.py", line 155, in build_sources
      self.build_library_sources(*libname_info)
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\build_src.py", line 288, in build_library_sources
      sources = self.generate_sources(sources, (lib_name, build_info))
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\build_src.py", line 378, in generate_sources
      source = func(extension, build_dir)
    File "numpy\core\setup.py", line 661, in get_mathlib_info
      st = config_cmd.try_link('int main(void) { return 0;}')
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 243, in try_link
      self._link(body, headers, include_dirs,
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\config.py", line 163, in _link
      return self._wrap_method(old_config._link, lang,
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\config.py", line 98, in _wrap_method
      ret = mth(*((self,)+args))
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 137, in _link
      (src, obj) = self._compile(body, headers, include_dirs, lang)
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\config.py", line 106, in _compile
      src, obj = self._wrap_method(old_config._compile, lang,
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\command\config.py", line 98, in _wrap_method
      ret = mth(*((self,)+args))
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\command\config.py", line 132, in _compile
      self.compiler.compile([src], include_dirs=include_dirs)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 401, in compile
      self.spawn(args)
    File "D:\Temp\TEMP\Temp\pip-build-env-j0x5tnf_\overlay\Lib\site-packages\setuptools\_distutils\_msvccompiler.py", line 505, in spawn
      return super().spawn(cmd, env=env)
    File "D:\Temp\TEMP\Temp\pip-install-sm1w3kd6\numpy_b431fd2b9bcc42cd9217589f1c5d56db\numpy\distutils\ccompiler.py", line 88, in <lambda>
      m = lambda self, *args, **kw: func(self, *args, **kw)
  TypeError: CCompiler_spawn() got an unexpected keyword argument 'env'
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

ml-agents setup.py:

import os import sys

from setuptools import setup, find_packages from setuptools.command.install import install from mlagents.plugins import ML_AGENTS_STATS_WRITER, ML_AGENTS_TRAINER_TYPE import mlagents.trainers

VERSION = mlagents.trainers.version EXPECTED_TAG = mlagents.trainers.__release_tag__

here = os.path.abspath(os.path.dirname(file))

class VerifyVersionCommand(install): """ Custom command to verify that the git tag is the expected one for the release. Originally based on https://circleci.com/blog/continuously-deploying-python-packages-to-pypi-with-circleci/ This differs slightly because our tags and versions are different. """

description = "verify that the git tag matches our version"

def run(self):
    tag = os.getenv("GITHUB_REF", "NO GITHUB TAG!").replace("refs/tags/", "")

    if tag != EXPECTED_TAG:
        info = "Git tag: {} does not match the expected tag of this app: {}".format(
            tag, EXPECTED_TAG
        )
        sys.exit(info)

Get the long description from the README file

with open(os.path.join(here, "README.md"), encoding="utf-8") as f: long_description = f.read()

setup( name="mlagents", version=VERSION, description="Unity Machine Learning Agents", long_description=long_description, long_description_content_type="text/markdown", url="https://github.com/Unity-Technologies/ml-agents", author="Unity Technologies", author_email="ML-Agents@unity3d.com", classifiers=[ "Intended Audience :: Developers", "Topic :: Scientific/Engineering :: Artificial Intelligence", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.10", ],

find_namespace_packages will recurse through the directories and find all the packages

packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
zip_safe=False,
install_requires=[
    # Test-only dependencies should go in test_requirements.txt, not here.
    "grpcio>=1.11.0,<=1.48.2",
    "h5py>=2.9.0",
    "mlagents_envs=={VERSION}",
    "numpy==1.23.1",
    "Pillow>=4.2.1",
    "protobuf>=3.6,<3.20",
    "pyyaml>=3.1.0",
    "torch>=2.1.1",
    "tensorboard>=2.14",
    # adding six explicit dependency since tensorboard needs it but doesn't declare it as a dep
    "six>=1.16",
    # cattrs 1.1.0 dropped support for python 3.6, but 1.0.0 doesn't work for python 3.9
    # Since there's no version that supports both, we have to draw the line somewhere.
    "cattrs>=1.1.0,<1.7; python_version>='3.8'",
    "attrs>=19.3.0",
    "huggingface_hub>=0.14",
    'pypiwin32==223;platform_system=="Windows"',
    "onnx==1.12.0",
],
python_requires=">=3.10.1,<=3.10.12",
entry_points={
    "console_scripts": [
        "mlagents-learn=mlagents.trainers.learn:main",
        "mlagents-run-experiment=mlagents.trainers.run_experiment:main",
        "mlagents-push-to-hf=mlagents.utils.push_to_hf:main",
        "mlagents-load-from-hf=mlagents.utils.load_from_hf:main",
    ],
    # Plugins - each plugin type should have an entry here for the default behavior
    ML_AGENTS_STATS_WRITER: [
        "default=mlagents.plugins.stats_writer:get_default_stats_writers"
    ],
    ML_AGENTS_TRAINER_TYPE: [
        "default=mlagents.plugins.trainer_type:get_default_trainer_types"
    ],
},
# TODO: Remove this once mypy stops having spurious setuptools issues.
cmdclass={"verify": VerifyVersionCommand},  # type: ignore

)

ml-agents-envs setup.py:

import os import sys from setuptools import setup, find_packages from setuptools.command.install import install import mlagents_envs

VERSION = mlagents_envs.version EXPECTED_TAG = mlagents_envs.__release_tag__

here = os.path.abspath(os.path.dirname(file))

class VerifyVersionCommand(install): """ Custom command to verify that the git tag is the expected one for the release. Originally based on https://circleci.com/blog/continuously-deploying-python-packages-to-pypi-with-circleci/ This differs slightly because our tags and versions are different. """

description = "verify that the git tag matches our version"

def run(self):
    tag = os.getenv("GITHUB_REF", "NO GITHUB TAG!").replace("refs/tags/", "")

    if tag != EXPECTED_TAG:
        info = "Git tag: {} does not match the expected tag of this app: {}".format(
            tag, EXPECTED_TAG
        )
        sys.exit(info)

Get the long description from the README file

with open(os.path.join(here, "README.md"), encoding="utf-8") as f: long_description = f.read()

setup( name="mlagents_envs", version=VERSION, description="Unity Machine Learning Agents Interface", long_description=long_description, long_description_content_type="text/markdown", url="https://github.com/Unity-Technologies/ml-agents", author="Unity Technologies", author_email="ML-Agents@unity3d.com", classifiers=[ "Intended Audience :: Developers", "Topic :: Scientific/Engineering :: Artificial Intelligence", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python :: 3.10", ], packages=find_packages( exclude=[".tests", ".tests.", "tests.", "tests", "colabs", "*.ipynb"] ), zip_safe=False, install_requires=[ "cloudpickle", "grpcio>=1.11.0,<=1.48.2", "Pillow>=4.2.1", "protobuf>=3.6,<3.20", "pyyaml>=3.1.0", "gym>=0.21.0", "pettingzoo==1.15.0", "numpy==1.23.1", "filelock>=3.4.0", ], python_requires=">=3.10.1,<=3.10.12",

TODO: Remove this once mypy stops having spurious setuptools issues.

cmdclass={"verify": VerifyVersionCommand},  # type: ignore

)

If you guys have any tips for fixing this, please tell me, been at it for 3 weeks, but I can't get my head around what's happening, I've reinstalled countless times, and deleted PIP Cache to prevent this (since before it used the cached numpy 1.21 as well)

Thank you in advance for your help

ashwinsnambiar commented 6 months ago

Installing mlagents from pip seems to be broken because of the numpy dependancy issues. I had faced the same problem earlier. Once you have updated the setup.py files, instead of installing it using pip, you have to install it from the cloned mlagents repo. Please read the section "Install the mlagents Python package" from the documentation https://unity-technologies.github.io/ml-agents/Installation/.

If its still not clear, check the pdf. @ICOnce has documented it well step wise. https://github.com/Unity-Technologies/ml-agents/issues/6047#issuecomment-1943131006

Hi Guys, having an issue with this as well. Been trying to install it for class. We're required to use Python 3.10.11, and I changed the setup.py to use numpy 1.23.1 (version teacher said we gotta use), however, no matter what I do, when I run python -m pip install mlagents==1.0.0 in the console, it always installs numpy 1.21.2, I'll post the setup files and command below:

If you guys have any tips for fixing this, please tell me, been at it for 3 weeks, but I can't get my head around what's happening, I've reinstalled countless times, and deleted PIP Cache to prevent this (since before it used the cached numpy 1.21 as well)

Thank you in advance for your help