dottxt-ai / outlines

Structured Text Generation
https://dottxt-ai.github.io/outlines/
Apache License 2.0
9.69k stars 495 forks source link

Rust compiler installation issue #1259

Closed SurenSK closed 13 hours ago

SurenSK commented 2 weeks ago

Describe the issue as clearly as possible:

pip install outlines fails due to rust compiler dependency. Downgrading with pip install outlines==0.0.46 also results in assertion errors on

from outlines import models, generate

model = models.transformers("mistralai/Mistral-7B-Instruct-v0.2")

regex_str = r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)"
generator = generate.regex(model, regex_str)

result = generator("What is the IP address of localhost?\nIP: ")
print(result)
# 127.0.0.100

Steps/code to reproduce the bug:

pip install outlines

Expected result:

outlines is installed

Error message:

pip install outlines
Collecting outlines
  Using cached outlines-0.1.3-py3-none-any.whl.metadata (15 kB)
Requirement already satisfied: interegular in c:\users\user\ml\lib\site-packages (from outlines) (0.3.3)
Requirement already satisfied: jinja2 in c:\users\user\ml\lib\site-packages (from outlines) (3.1.3)      
Requirement already satisfied: lark in c:\users\user\ml\lib\site-packages (from outlines) (1.2.2)        
Requirement already satisfied: nest-asyncio in c:\users\user\ml\lib\site-packages (from outlines) (1.6.0)
Requirement already satisfied: numpy<2.0.0 in c:\users\user\ml\lib\site-packages (from outlines) (1.26.3)
Requirement already satisfied: cloudpickle in c:\users\user\ml\lib\site-packages (from outlines) (3.0.0) 
Requirement already satisfied: diskcache in c:\users\user\ml\lib\site-packages (from outlines) (5.6.3)   
Requirement already satisfied: pydantic>=2.0 in c:\users\user\ml\lib\site-packages (from outlines) (2.9.1)
Requirement already satisfied: referencing in c:\users\user\ml\lib\site-packages (from outlines) (0.35.1)
Requirement already satisfied: jsonschema in c:\users\user\ml\lib\site-packages (from outlines) (4.23.0) 
Requirement already satisfied: requests in c:\users\user\ml\lib\site-packages (from outlines) (2.32.3)   
Requirement already satisfied: tqdm in c:\users\user\ml\lib\site-packages (from outlines) (4.66.5)       
Requirement already satisfied: datasets in c:\users\user\ml\lib\site-packages (from outlines) (3.0.0)    
Requirement already satisfied: typing-extensions in c:\users\user\ml\lib\site-packages (from outlines) (4.12.2)
Requirement already satisfied: pycountry in c:\users\user\ml\lib\site-packages (from outlines) (24.6.1)  
Collecting airportsdata (from outlines)
  Using cached airportsdata-20241001-py3-none-any.whl.metadata (8.9 kB)
Requirement already satisfied: torch in c:\users\user\ml\lib\site-packages (from outlines) (2.4.1+cu124)
Collecting outlines-core==0.1.14 (from outlines)
  Using cached outlines_core-0.1.14.tar.gz (59 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: annotated-types>=0.6.0 in c:\users\user\ml\lib\site-packages (from pydantic>=2.0->outlines) (0.7.0)
Requirement already satisfied: pydantic-core==2.23.3 in c:\users\user\ml\lib\site-packages (from pydantic>=2.0->outlines) (2.23.3)
Requirement already satisfied: filelock in c:\users\user\ml\lib\site-packages (from datasets->outlines) (3.13.1)
Requirement already satisfied: pyarrow>=15.0.0 in c:\users\user\ml\lib\site-packages (from datasets->outlines) (17.0.0)
Requirement already satisfied: dill<0.3.9,>=0.3.0 in c:\users\user\ml\lib\site-packages (from datasets->outlines) (0.3.8)
Requirement already satisfied: pandas in c:\users\user\ml\lib\site-packages (from datasets->outlines) (2.2.2)
Requirement already satisfied: xxhash in c:\users\user\ml\lib\site-packages (from datasets->outlines) (3.5.0)
Requirement already satisfied: multiprocess in c:\users\user\ml\lib\site-packages (from datasets->outlines) (0.70.16)
Requirement already satisfied: fsspec<=2024.6.1,>=2023.1.0 in c:\users\user\ml\lib\site-packages (from fsspec[http]<=2024.6.1,>=2023.1.0->datasets->outlines) (2024.2.0)
Requirement already satisfied: aiohttp in c:\users\user\ml\lib\site-packages (from datasets->outlines) (3.10.5)
Requirement already satisfied: huggingface-hub>=0.22.0 in c:\users\user\ml\lib\site-packages (from datasets->outlines) (0.24.7)
Requirement already satisfied: packaging in c:\users\user\ml\lib\site-packages (from datasets->outlines) (24.1)
Requirement already satisfied: pyyaml>=5.1 in c:\users\user\ml\lib\site-packages (from datasets->outlines) (6.0.2)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\user\ml\lib\site-packages (from requests->outlines) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\user\ml\lib\site-packages (from requests->outlines) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\user\ml\lib\site-packages (from requests->outlines) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\user\ml\lib\site-packages (from requests->outlines) (2024.8.30)
Requirement already satisfied: colorama in c:\users\user\ml\lib\site-packages (from tqdm->outlines) (0.4.6)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\user\ml\lib\site-packages (from jinja2->outlines) (2.1.5)
Requirement already satisfied: attrs>=22.2.0 in c:\users\user\ml\lib\site-packages (from jsonschema->outlines) (24.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in c:\users\user\ml\lib\site-packages (from jsonschema->outlines) (2023.12.1)
Requirement already satisfied: rpds-py>=0.7.1 in c:\users\user\ml\lib\site-packages (from jsonschema->outlines) (0.20.0)
Requirement already satisfied: sympy in c:\users\user\ml\lib\site-packages (from torch->outlines) (1.12)
Requirement already satisfied: networkx in c:\users\user\ml\lib\site-packages (from torch->outlines) (3.2.1)
Requirement already satisfied: setuptools in c:\users\user\ml\lib\site-packages (from torch->outlines) (75.1.0)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in c:\users\user\ml\lib\site-packages (from aiohttp->datasets->outlines) (2.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in c:\users\user\ml\lib\site-packages (from aiohttp->datasets->outlines) (1.3.1)
Requirement already satisfied: frozenlist>=1.1.1 in c:\users\user\ml\lib\site-packages (from aiohttp->datasets->outlines) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in c:\users\user\ml\lib\site-packages (from aiohttp->datasets->outlines) (6.1.0)
Requirement already satisfied: yarl<2.0,>=1.0 in c:\users\user\ml\lib\site-packages (from aiohttp->datasets->outlines) (1.11.1)
Requirement already satisfied: python-dateutil>=2.8.2 in c:\users\user\ml\lib\site-packages (from pandas->datasets->outlines) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\users\user\ml\lib\site-packages (from pandas->datasets->outlines) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in c:\users\user\ml\lib\site-packages (from pandas->datasets->outlines) (2024.1)
Requirement already satisfied: mpmath>=0.19 in c:\users\user\ml\lib\site-packages (from sympy->torch->outlines) (1.3.0)
Requirement already satisfied: six>=1.5 in c:\users\user\ml\lib\site-packages (from python-dateutil>=2.8.2->pandas->datasets->outlines) (1.16.0)
Using cached outlines-0.1.3-py3-none-any.whl (91 kB)
Using cached airportsdata-20241001-py3-none-any.whl (912 kB)
Building wheels for collected packages: outlines-core
  Building wheel for outlines-core (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for outlines-core (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [76 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build\lib.win-amd64-cpython-312\outlines_core
      copying python\outlines_core\_version.py -> build\lib.win-amd64-cpython-312\outlines_core
      copying python\outlines_core\__init__.py -> build\lib.win-amd64-cpython-312\outlines_core
      running egg_info
      writing python\outlines_core.egg-info\PKG-INFO
      writing dependency_links to python\outlines_core.egg-info\dependency_links.txt
      writing requirements to python\outlines_core.egg-info\requires.txt
      writing top-level names to python\outlines_core.egg-info\top_level.txt
      ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any       
      reading manifest file 'python\outlines_core.egg-info\SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file 'python\outlines_core.egg-info\SOURCES.txt'
      C:\Users\user\AppData\Local\Temp\pip-build-env-meep4lir\overlay\Lib\site-packages\setuptools\command\build_py.py:220: _Warning: Package 'outlines_core.fsm' is absent from the `packages` configuration.     
      !!

              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'outlines_core.fsm' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.

              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'outlines_core.fsm' is explicitly added
              to the `packages` configuration field.

              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).

              You can read more about "package discovery" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html

              If you don't want 'outlines_core.fsm' to be distributed and are
              already explicitly excluding 'outlines_core.fsm' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.

              You can read more about "package data files" on setuptools documentation page:

              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html

              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************

      !!
        check.warn(importable)
      copying python\outlines_core\py.typed -> build\lib.win-amd64-cpython-312\outlines_core
      creating build\lib.win-amd64-cpython-312\outlines_core\fsm
      copying python\outlines_core\fsm\__init__.py -> build\lib.win-amd64-cpython-312\outlines_core\fsm   
      copying python\outlines_core\fsm\guide.py -> build\lib.win-amd64-cpython-312\outlines_core\fsm      
      copying python\outlines_core\fsm\json_schema.py -> build\lib.win-amd64-cpython-312\outlines_core\fsm
      copying python\outlines_core\fsm\outlines_core_rs.pyi -> build\lib.win-amd64-cpython-312\outlines_core\fsm
      copying python\outlines_core\fsm\regex.py -> build\lib.win-amd64-cpython-312\outlines_core\fsm      
      running build_ext
      running build_rust
      error: can't find Rust compiler

      If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

      To update pip, run:

          pip install --upgrade pip

      and then retry package installation.

      If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for outlines-core
Failed to build outlines-core

[notice] A new release of pip is available: 24.2 -> 24.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (outlines-core)

Outlines/Python version information:

Version information

``` (command output here) ```

Context for the issue:

Cant use a library without installing it first

torymur commented 13 hours ago

Hi @SurenSK 👋 Thank you for creating the issue!

Wheels for python 3.12-3.13 of outlines-core were added starting from outlines 0.1.4, so we can close it now.