dottxt-ai / outlines

Structured Text Generation
https://dottxt-ai.github.io/outlines/
Apache License 2.0
9.57k stars 492 forks source link

ValidationError when trying to generate JSON using Ollama with OpenAI API #1221

Open DiyarD opened 1 month ago

DiyarD commented 1 month ago

Describe the issue as clearly as possible:

I'm trying to use Ollama with the Openai api to generate a Pydantic object. I think that's because Ollama doesn't support response_format which is used in /outlines/generate/json.py:

@json.register(OpenAI)
def json_openai(
    model, schema_object: Union[str, object], sampler: Sampler = multinomial()
):
    ...

    # create copied, patched model with normalized json schema set
    generator = model.new_with_replacements(
        response_format={
            "type": "json_schema",
            "json_schema": {
                "name": "default",
                "strict": True,
                "schema": pyjson.loads(schema),
            },
        }
    )

    generator.format_sequence = format_sequence

    return generator

Ollama does have a format: json mode but that doesn't support adding a json schema.

Is there any workaround for this?

Steps/code to reproduce the bug:

from typing import List, Literal
from pydantic import BaseModel
import outlines 

class Hobby(BaseModel):
    name: str
    description: str
    duration: int = Field(..., description="Years since having hobby")

class Character(BaseModel):
    name:str
    birth_year: int = Field(..., ge=1990, le=2025)
    hobbies: List[Hobby]
    alive: bool

lm = outlines.models.openai("llama3.2:latest", base_url="http://localhost:11434/v1", api_key="ollama")
generatorJson = outlines.generate.json(lm, Character)
print(generatorJson("Create a character with fields name, birth_year, hobbies (each hobby has a name, description, and duration), alive (bool)", max_tokens=20000))

Expected result:

A JSON object following the schema of the Character class.

Error message:


Traceback (most recent call last):
  File "c:\Users\TOTAL TECH\Desktop\mcts.py", line 30, in <module>
    pprint(generatorJson("Create a user with fields name, birth_year, hobbies (each hobby has a name, description, and duration), alive (bool)", format="json", temperature=0, max_tokens=20000))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Traceback (most recent call last):
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\site-packages\pydantic\main.py", line 1187, in parse_raw
    obj = parse.load_str_bytes(
          ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\site-packages\pydantic\deprecated\parse.py", line 49, in load_str_bytes
    return json_loads(b)  # type: ignore
           ^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\Users\TOTAL TECH\Desktop\mcts.py", line 27, in <module>
    pprint(generatorJson("Create a character with fields name, birth_year, hobbies (each hobby has a name, description, and duration), alive (bool)", max_tokens=20000))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\site-packages\outlines\models\openai.py", line 151, in __call__
    return self.format_sequence(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\site-packages\outlines\generate\json.py", line 83, in <lambda>
    format_sequence = lambda x: schema_object.parse_raw(x)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\TOTAL TECH\.conda\envs\outlines\Lib\site-packages\pydantic\main.py", line 1214, in parse_raw
    raise pydantic_core.ValidationError.from_exception_data(cls.__name__, [error])
pydantic_core._pydantic_core.ValidationError: 1 validation error for Character
__root__
  Expecting value: line 1 column 1 (char 0) [type=value_error.jsondecode, input_value='class Character:\n    de...alive)  # Output: False', input_type=str]

Outlines/Python version information:

Version information

- Outlines version 0.1.1 - Python 3.12.7 | packaged by Anaconda, Inc. | (main, Oct 4 2024, 13:17:27) [MSC v.1929 64 bit (AMD64)] - Pip freeze: absl-py==2.1.0 accelerate==1.0.1 aiobotocore @ file:///C:/b/abs_a0zxrsvpwx/croot/aiobotocore_1714464454692/work aiohappyeyeballs @ file:///C:/b/abs_b505trsapr/croot/aiohappyeyeballs_1725434036096/work aiohttp @ file:///C:/b/abs_13j6efxjb7/croot/aiohttp_1725529348885/work aioitertools @ file:///tmp/build/80754af9/aioitertools_1607109665762/work aiosignal @ file:///tmp/build/80754af9/aiosignal_1637843061372/work airportsdata==20241001 alabaster @ file:///C:/b/abs_45ba4vacaj/croot/alabaster_1718201502252/work altair @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/altair_1699497320503/work annotated-types==0.7.0 anyio @ file:///C:/b/abs_847uobe7ea/croot/anyio_1706220224037/work appdirs==1.4.4 argon2-cffi @ file:///opt/conda/conda-bld/argon2-cffi_1645000214183/work argon2-cffi-bindings @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/argon2-cffi-bindings_1699549801117/work arrow @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/arrow_1699549851004/work astroid @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/astroid_1699480418189/work astropy @ file:///C:/b/abs_5egnt0gyqe/croot/astropy_1726174619184/work astropy-iers-data @ file:///C:/b/abs_ecdfv0_ea2/croot/astropy-iers-data_1726000550777/work asttokens @ file:///opt/conda/conda-bld/asttokens_1646925590279/work astunparse==1.6.3 async-lru @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/async-lru_1701796859357/work atomicwrites==1.4.0 attrs @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/attrs_1699473735254/work Automat @ file:///tmp/build/80754af9/automat_1600298431173/work autopep8 @ file:///croot/autopep8_1708962882016/work Babel @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/babel_1699475785740/work bcrypt @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/bcrypt_1699483842258/work beautifulsoup4 @ file:///C:/b/abs_d5wytg_p0w/croot/beautifulsoup4-split_1718029833749/work binaryornot @ file:///tmp/build/80754af9/binaryornot_1617751525010/work black @ file:///C:/b/abs_3arhghl9_x/croot/black_1725573890945/work bleach @ file:///opt/conda/conda-bld/bleach_1641577558959/work blinker @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/blinker_1699474692786/work bokeh @ file:///C:/b/abs_b40zi4jp_s/croot/bokeh_1727914495891/work botocore @ file:///C:/b/abs_0dkeqep0ox/croot/botocore_1714460586193/work Bottleneck @ file:///C:/b/abs_f7un855idq/croot/bottleneck_1709069969633/work Brotli @ file:///C:/b/abs_3d36mno480/croot/brotli-split_1714483178642/work cachetools @ file:///C:/b/abs_792zbtc0ua/croot/cachetools_1713977157919/work captcha==0.1.1 certifi @ file:///C:/b/abs_1fw_exq1si/croot/certifi_1725551736618/work/certifi cffi @ file:///C:/b/abs_90yq4lmu83/croot/cffi_1726856448345/work chardet @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/chardet_1699498892802/work charset-normalizer @ file:///croot/charset-normalizer_1721748349566/work click @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/click_1699473800648/work cloudpickle @ file:///C:/b/abs_8abv3sva4p/croot/cloudpickle_1721657372632/work colorama @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/colorama_1699472650914/work colorcet @ file:///C:/b/abs_7ep0mfawez/croot/colorcet_1709758405232/work comm @ file:///C:/b/abs_67a8058udb/croot/comm_1709322909844/work constantly @ file:///C:/b/abs_cbuavw4443/croot/constantly_1703165617403/work contourpy @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/contourpy_1701817388761/work cookiecutter @ file:///C:/b/abs_66untpu1ub/croot/cookiecutter_1711059869779/work cryptography @ file:///C:/b/abs_35g500qir4/croot/cryptography_1724940575116/work cssselect @ file:///C:/b/abs_71gnjab7b0/croot/cssselect_1707339955530/work cycler @ file:///tmp/build/80754af9/cycler_1637851556182/work cytoolz @ file:///C:/b/abs_d43s8lnb60/croot/cytoolz_1701723636699/work dask @ file:///C:/b/abs_12bk4i_z1j/croot/dask-core_1725464350376/work dask-expr @ file:///C:/b/abs_d8e1rqubql/croot/dask-expr_1725523012670/work datasets==3.0.1 datashader @ file:///C:/b/abs_37m05texdf/croot/datashader_1720534925330/work debugpy @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/debugpy_1699554994633/work decorator==4.4.2 defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work diff-match-patch @ file:///Users/ktietz/demo/mc3/conda-bld/diff-match-patch_1630511840874/work dill @ file:///C:/b/abs_f79eg27d2q/croot/dill_1715094735295/work diskcache==5.6.3 distributed @ file:///C:/b/abs_52ju83vjqr/croot/distributed_1725523030746/work distro==1.9.0 docstring-to-markdown @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/docstring-to-markdown_1699484265167/work docutils @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/docutils_1699474820579/work et-xmlfile @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/et_xmlfile_1699500373144/work executing @ file:///opt/conda/conda-bld/executing_1646925071911/work fastjsonschema @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/python-fastjsonschema_1699475134300/work filelock @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/filelock_1701807523603/work flake8 @ file:///C:/b/abs_caud66drfv/croot/flake8_1708965316778/work Flask @ file:///C:/b/abs_f9w7doyu0h/croot/flask_1716545924884/work flatbuffers==24.3.25 fonttools @ file:///C:/b/abs_f47gnfqnx0/croot/fonttools_1713551644747/work frozenlist @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/frozenlist_1699557027827/work fsspec @ file:///C:/b/abs_02vh2bpxfo/croot/fsspec_1724855601370/work gast==0.6.0 gensim @ file:///C:/b/abs_efsh9stgvi/croot/gensim_1725376635600/work gitdb @ file:///tmp/build/80754af9/gitdb_1617117951232/work GitPython @ file:///C:/b/abs_2bkslnqz4i/croot/gitpython_1720455044865/work google-pasta==0.2.0 greenlet @ file:///C:/b/abs_a6c75ie0bc/croot/greenlet_1702060012174/work grpcio==1.64.1 h11 @ file:///C:/b/abs_1czwoyexjf/croot/h11_1706652332846/work h5py @ file:///C:/b/abs_c4ha_1xv14/croot/h5py_1715094776210/work HeapDict @ file:///Users/ktietz/demo/mc3/conda-bld/heapdict_1630598515714/work holoviews @ file:///C:/b/abs_5f962ukrbq/croot/holoviews_1720534381286/work httpcore @ file:///C:/b/abs_55n7g233bw/croot/httpcore_1706728507241/work httpx @ file:///C:/b/abs_43e135shby/croot/httpx_1723474830126/work huggingface-hub==0.26.1 hvplot @ file:///C:/b/abs_b05v45vms6/croot/hvplot_1727775587313/work hyperlink @ file:///tmp/build/80754af9/hyperlink_1610130746837/work idna @ file:///C:/b/abs_aad84bnnw5/croot/idna_1714398896795/work imagecodecs @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/imagecodecs_1699481498787/work imageio==2.35.1 imageio-ffmpeg==0.5.1 imagesize @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/imagesize_1699476341924/work imbalanced-learn @ file:///C:/b/abs_43zx11b5ex/croot/imbalanced-learn_1718132243956/work importlib-metadata @ file:///C:/b/abs_c1egths604/croot/importlib_metadata-suite_1704813568388/work incremental @ file:///croot/incremental_1708639938299/work inflection @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/inflection_1699500974258/work iniconfig @ file:///home/linux1/recipes/ci/iniconfig_1610983019677/work intake @ file:///C:/b/abs_ebww2no2x2/croot/intake_1726109581477/work interegular==0.3.3 intervaltree @ file:///Users/ktietz/demo/mc3/conda-bld/intervaltree_1630511889664/work ipykernel @ file:///C:/b/abs_c2u94kxcy6/croot/ipykernel_1705933907920/work ipython @ file:///C:/b/abs_53it5seiim/croot/ipython_1726064251844/work ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work ipywidgets @ file:///C:/b/abs_4fgmepzmbu/croot/ipywidgets_1710961556913/work isort @ file:///C:/b/abs_50iy840e4z/croot/isort_1718291849200/work itemadapter @ file:///tmp/build/80754af9/itemadapter_1626442940632/work itemloaders @ file:///C:/b/abs_5e3azgv25z/croot/itemloaders_1708639993442/work itsdangerous @ file:///C:/b/abs_c4vwgdr5yn/croot/itsdangerous_1716533399914/work jaraco.classes @ file:///tmp/build/80754af9/jaraco.classes_1620983179379/work jedi @ file:///C:/b/abs_1b8kmj7rrm/croot/jedi_1721058359741/work jellyfish @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jellyfish_1699558116601/work Jinja2 @ file:///C:/b/abs_92fccttino/croot/jinja2_1716993447201/work jiter==0.6.1 jmespath @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jmespath_1701807551985/work joblib @ file:///C:/b/abs_f4b98l6lgk/croot/joblib_1718217224240/work json5 @ file:///tmp/build/80754af9/json5_1624432770122/work jsonschema @ file:///C:/b/abs_394_t6__xq/croot/jsonschema_1728486718320/work jsonschema-specifications @ file:///C:/Users/dev-admin/py312/jsonschema-specifications_1706803038066/work jupyter @ file:///C:/b/abs_03nm7xrhxz/croot/jupyter_1709837239940/work jupyter-console @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jupyter_console_1707430148515/work jupyter-events @ file:///C:/b/abs_c2m9s5b5m5/croot/jupyter_events_1718738115254/work jupyter-lsp @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jupyter-lsp-meta_1701806528286/work jupyter_client @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jupyter_client_1701796382758/work jupyter_core @ file:///C:/b/abs_beftpbuevw/croot/jupyter_core_1718818307097/work jupyter_server @ file:///C:/b/abs_9a333nh6yu/croot/jupyter_server_1718827092223/work jupyter_server_terminals @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jupyter_server_terminals_1701798041635/work jupyterlab @ file:///C:/b/abs_88qjk9lo2r/croot/jupyterlab_1725895228333/work jupyterlab-pygments @ file:///tmp/build/80754af9/jupyterlab_pygments_1601490720602/work jupyterlab-widgets @ file:///tmp/build/80754af9/jupyterlab_widgets_1609884341231/work jupyterlab_server @ file:///C:/b/abs_fdi5r_tpjc/croot/jupyterlab_server_1725865372811/work keras==3.4.1 keyring @ file:///C:/b/abs_78uoj9sw00/croot/keyring_1709632550180/work kiwisolver @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/kiwisolver_1699476425777/work lark==1.2.2 lazy-object-proxy @ file:///C:/b/abs_19td_nzb6n/croot/lazy-object-proxy_1712908735070/work lazy_loader @ file:///C:/b/abs_3fs2i5w5p3/croot/lazy_loader_1718176758844/work lckr_jupyterlab_variableinspector @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/jupyterlab-variableinspector_1709167201477/work libclang==18.1.1 linkify-it-py @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/linkify-it-py_1699558957411/work llvmlite @ file:///C:/b/abs_80rinyr2fr/croot/llvmlite_1720455223586/work lmdb @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/python-lmdb_1699495888630/work locket @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/locket_1699476508014/work lxml==5.2.1 lz4 @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/lz4_1699559532268/work Markdown @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/markdown_1699503092856/work markdown-it-py @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/markdown-it-py_1699473886965/work MarkupSafe @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/markupsafe_1707425732791/work matplotlib==3.9.2 matplotlib-inline @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/matplotlib-inline_1699484796387/work mccabe @ file:///opt/conda/conda-bld/mccabe_1644221741721/work mdit-py-plugins @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/mdit-py-plugins_1699559788389/work mdurl @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/mdurl_1699473506455/work mistune @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/mistune_1699559923704/work mkl-service==2.4.0 mkl_fft @ file:///C:/b/abs_f55mv94vyg/croot/mkl_fft_1725370278455/work mkl_random @ file:///C:/b/abs_21ydbzdu8d/croot/mkl_random_1725370276095/work ml-dtypes==0.3.2 more-itertools @ file:///C:/b/abs_a2n4mhb8gn/croot/more-itertools_1727185463826/work moviepy==1.0.3 mpmath @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/mpmath_1699484863771/work msgpack @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/msgpack-python_1699473924872/work multidict @ file:///C:/b/abs_44ido987fv/croot/multidict_1701097803486/work multipledispatch @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/multipledispatch_1699473951974/work multiprocess==0.70.16 mypy @ file:///C:/b/abs_b8k0j63oc7/croot/mypy-split_1725574346532/work mypy-extensions @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/mypy_extensions_1699482634516/work namex==0.0.8 nbclient @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/nbclient_1701796752236/work nbconvert @ file:///C:/b/abs_ac6qnzi3no/croot/nbconvert_1728050663985/work nbformat @ file:///C:/b/abs_c2jkw46etm/croot/nbformat_1728050303821/work nest-asyncio @ file:///C:/b/abs_65d6lblmoi/croot/nest-asyncio_1708532721305/work networkx @ file:///C:/b/abs_36fmumtynt/croot/networkx_1720002497414/work nltk @ file:///C:/b/abs_3e10flry37/croot/nltk_1724427703710/work notebook @ file:///C:/b/abs_feeub5ouq6/croot/notebook_1727197380211/work notebook_shim @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/notebook-shim_1701806592322/work numba @ file:///C:/b/abs_51yyu3qucu/croot/numba_1720540987599/work numexpr @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/numexpr_1699503421264/work numpy @ file:///C:/b/abs_c1ywpu18ar/croot/numpy_and_numpy_base_1708638681471/work/dist/numpy-1.26.4-cp312-cp312-win_amd64.whl#sha256=becc06674317799ad0165a939a7613809d0bee9bd328a1e4308c57c39cacf08c numpydoc @ file:///C:/b/abs_bbspp5l8vu/croot/numpydoc_1718279185573/work openai==1.52.0 openpyxl @ file:///C:/b/abs_0e6ca21lac/croot/openpyxl_1721752965859/work opt-einsum==3.3.0 optree==0.12.0 outlines==0.1.1 outlines_core==0.1.14 overrides @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/overrides_1701806336503/work packaging @ file:///C:/b/abs_c3vlh0z4jw/croot/packaging_1720101866539/work pandas @ file:///C:/b/abs_9aotnvvz16/croot/pandas_1718308978393/work/dist/pandas-2.2.2-cp312-cp312-win_amd64.whl#sha256=93959056e02e9855025011adb18394296a58d49e72b9342733b7693a5267c790 pandocfilters @ file:///opt/conda/conda-bld/pandocfilters_1643405455980/work panel @ file:///C:/b/abs_fas2lxv_bs/croot/panel_1728066386547/work param @ file:///C:/b/abs_b4yato0oux/croot/param_1719347953072/work paramiko @ file:///opt/conda/conda-bld/paramiko_1640109032755/work parsel @ file:///C:/b/abs_ebc3tzm_c4/croot/parsel_1707503517596/work parso @ file:///opt/conda/conda-bld/parso_1641458642106/work partd @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/partd_1699482661603/work pathspec @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pathspec_1699472481068/work patsy @ file:///C:/b/abs_21hy_wyj13/croot/patsy_1718378184379/work pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work pillow @ file:///C:/b/abs_32o8er3uqp/croot/pillow_1721059447598/work platformdirs @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/platformdirs_1701797392447/work plotly @ file:///C:/b/abs_1014knmz1t/croot/plotly_1726245573566/work pluggy @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pluggy_1699472504117/work ply @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/ply_1699473999871/work proglog==0.1.10 prometheus-client @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/prometheus_client_1699495496669/work prompt-toolkit @ file:///C:/b/abs_68uwr58ed1/croot/prompt-toolkit_1704404394082/work Protego @ file:///tmp/build/80754af9/protego_1598657180827/work protobuf==4.25.3 psutil @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/psutil_1699482842340/work ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl pure-eval @ file:///opt/conda/conda-bld/pure_eval_1646925070566/work py-cpuinfo @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/py-cpuinfo_1699495567147/work pyarrow @ file:///C:/b/abs_cbc4sg_1e0/croot/pyarrow_1721664257308/work/python pyasn1 @ file:///Users/ktietz/demo/mc3/conda-bld/pyasn1_1629708007385/work pyasn1-modules==0.2.8 pycodestyle @ file:///C:/b/abs_e9pt4mahct/croot/pycodestyle_1701910643139/work pycountry==24.6.1 pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work pyct @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pyct_1699542988857/work pycurl @ file:///C:/b/abs_2bemyov07c/croot/pycurl_1725370252605/work pydantic==2.9.2 pydantic_core==2.23.4 pydeck @ file:///C:/b/abs_ad9p880wi1/croot/pydeck_1706194121328/work PyDispatcher @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pydispatcher_1699543033434/work pydocstyle @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pydocstyle_1699495611004/work pyerfa @ file:///C:/b/abs_477fivsfe8/croot/pyerfa_1717700768172/work pyflakes @ file:///C:/b/abs_b6t1aazq2e/croot/pyflakes_1708963020907/work Pygments @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pygments_1699474141968/work pylint @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pylint_1699495689806/work pylint-venv @ file:///C:/b/abs_3f6p_17zia/croot/pylint-venv_1709837680309/work pyls-spyder==0.4.0 PyNaCl @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pynacl_1699563077212/work pyodbc @ file:///C:/b/abs_cbgr1yn8qs/croot/pyodbc_1725560257385/work pyOpenSSL @ file:///C:/b/abs_61vvbgrloc/croot/pyopenssl_1723651600236/work pyparsing @ file:///C:/b/abs_520r19rysg/croot/pyparsing_1725041656021/work PyQt5==5.15.10 PyQt5-sip @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pyqt-split_1699478669290/work/pyqt_sip PyQtWebEngine==5.15.6 PySocks @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pysocks_1699473336188/work pytest @ file:///C:/b/abs_een_3z747j/croot/pytest_1717793253670/work python-dateutil @ file:///C:/b/abs_3au_koqnbs/croot/python-dateutil_1716495777160/work python-json-logger @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/python-json-logger_1699543626759/work python-lsp-black @ file:///C:/b/abs_5dhen_5vga/croot/python-lsp-black_1709232962589/work python-lsp-jsonrpc @ file:///croot/python-lsp-jsonrpc_1708962872556/work python-lsp-server @ file:///C:/b/abs_2cdyzvcq4z/croot/python-lsp-server_1708971796975/work python-slugify @ file:///tmp/build/80754af9/python-slugify_1620405669636/work pytoolconfig @ file:///C:/b/abs_f2j_xsvrpn/croot/pytoolconfig_1701728751207/work pytz @ file:///C:/b/abs_6ap4tsz1ox/croot/pytz_1713974360290/work pyviz_comms @ file:///C:/b/abs_674gfmqoxa/croot/pyviz_comms_1711136872853/work PyWavelets @ file:///C:/b/abs_95ujr8_xu1/croot/pywavelets_1725657962149/work pywin32==305.1 pywin32-ctypes @ file:///C:/b/abs_2cfx5l4nvi/croot/pywin32-ctypes_1709340246092/work pywinpty @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pywinpty_1699496010557/work/target/wheels/pywinpty-2.0.10-cp312-none-win_amd64.whl#sha256=227f3f94f568f63ab3418c02875c4b429a0a3638e1a2917a22bb8e4ba1762179 PyYAML @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/pyyaml_1699479991712/work pyzmq @ file:///C:/b/abs_89aq69t0up/croot/pyzmq_1705605705281/work QDarkStyle @ file:///croot/qdarkstyle_1709231003551/work qstylizer @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/qstylizer_1699565338341/work/dist/qstylizer-0.2.2-py2.py3-none-any.whl#sha256=be2f7d956d89710f13dee80f2d594e794c2710f12576fe60ecee84dd1ad72180 QtAwesome @ file:///C:/b/abs_e1w2vmh9q7/croot/qtawesome_1726169367822/work qtconsole @ file:///C:/b/abs_03f8rg9vl6/croot/qtconsole_1709231218069/work QtPy @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/qtpy_1701807198514/work queuelib @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/queuelib_1699543858829/work referencing @ file:///C:/Users/dev-admin/py312/referencing_1706802962559/work regex @ file:///C:/b/abs_5cm86yjgo3/croot/regex_1726670543261/work requests @ file:///C:/b/abs_9frifg92q2/croot/requests_1721410901096/work requests-file @ file:///Users/ktietz/demo/mc3/conda-bld/requests-file_1629455781986/work rfc3339-validator @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/rfc3339-validator_1699543924991/work rfc3986-validator @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/rfc3986-validator_1699543955651/work rich @ file:///C:/b/abs_21nw9z7xby/croot/rich_1720637504376/work rope @ file:///C:/b/abs_a4uy0nuc8z/croot/rope_1708963217026/work rpds-py @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/rpds-py_1699565648947/work Rtree @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/rtree_1699565751755/work s3fs @ file:///C:/b/abs_ddtjx92ddb/croot/s3fs_1724924140381/work safetensors==0.4.5 scikit-image @ file:///C:/b/abs_85dh_ywewv/croot/scikit-image_1726762076100/work scikit-learn @ file:///C:/b/abs_aajc9f7v6r/croot/scikit-learn_1722067652693/work/dist/scikit_learn-1.5.1-cp312-cp312-win_amd64.whl#sha256=96ed6defb86c154d2ff360afdbf0c37c33c4c729d4a15c9d8ada0b70fd817816 scipy==1.14.1 Scrapy @ file:///C:/b/abs_b407wrcuij/croot/scrapy_1708714755460/work seaborn @ file:///C:/b/abs_ca2mi1rgmn/croot/seaborn_1718303534355/work Send2Trash @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/send2trash_1701806400767/work service-identity @ file:///Users/ktietz/demo/mc3/conda-bld/service_identity_1629460757137/work setuptools==75.1.0 sip @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/sip_1699475590677/work six @ file:///tmp/build/80754af9/six_1644875935023/work smart-open @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/smart_open_1699483216058/work smmap @ file:///tmp/build/80754af9/smmap_1611694433573/work sniffio @ file:///C:/b/abs_3akdewudo_/croot/sniffio_1705431337396/work snowballstemmer @ file:///tmp/build/80754af9/snowballstemmer_1637937080595/work socks==0 sortedcontainers @ file:///tmp/build/80754af9/sortedcontainers_1623949099177/work soupsieve @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/soupsieve_1699496611169/work Sphinx @ file:///C:/b/abs_67j2y1kkd0/croot/sphinx_1718275396111/work sphinxcontrib-applehelp @ file:///home/ktietz/src/ci/sphinxcontrib-applehelp_1611920841464/work sphinxcontrib-devhelp @ file:///home/ktietz/src/ci/sphinxcontrib-devhelp_1611920923094/work sphinxcontrib-htmlhelp @ file:///tmp/build/80754af9/sphinxcontrib-htmlhelp_1623945626792/work sphinxcontrib-jsmath @ file:///home/ktietz/src/ci/sphinxcontrib-jsmath_1611920942228/work sphinxcontrib-qthelp @ file:///home/ktietz/src/ci/sphinxcontrib-qthelp_1611921055322/work sphinxcontrib-serializinghtml @ file:///C:/b/abs_91266tdnkr/croot/sphinxcontrib-serializinghtml_1718201501959/work spyder @ file:///C:/b/abs_2f04kjngiv/croot/spyder_1727197291220/work spyder-kernels @ file:///C:/b/abs_e5u6y4ldr2/croot/spyder-kernels_1707937767956/work SQLAlchemy @ file:///C:/b/abs_7duxw5rxx8/croot/sqlalchemy_1725885067126/work stack-data @ file:///opt/conda/conda-bld/stack_data_1646927590127/work statsmodels @ file:///C:/b/abs_54b33xdukx/croot/statsmodels_1718381209933/work streamlit @ file:///C:/b/abs_84p_54gim8/croot/streamlit_1724335176234/work sympy==1.13.1 tables @ file:///C:/b/abs_92un5wtnqq/croot/pytables_1725380800658/work tabulate @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/tabulate_1701812852133/work tblib @ file:///Users/ktietz/demo/mc3/conda-bld/tblib_1629402031467/work tenacity @ file:///C:/b/abs_d2_havtscu/croot/tenacity_1721222514290/work tensorboard-data-server==0.7.2 termcolor==2.4.0 terminado @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/terminado_1699545066607/work text-unidecode @ file:///Users/ktietz/demo/mc3/conda-bld/text-unidecode_1629401354553/work textdistance @ file:///tmp/build/80754af9/textdistance_1612461398012/work threadpoolctl @ file:///C:/b/abs_def0dwqlft/croot/threadpoolctl_1719407816649/work three-merge @ file:///tmp/build/80754af9/three-merge_1607553261110/work tifffile @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/tifffile_1699496794322/work tinycss2 @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/tinycss2_1699545185754/work tldextract @ file:///C:/b/abs_34gknp7o18/croot/tldextract_1723064394448/work tokenizers==0.20.1 toml @ file:///tmp/build/80754af9/toml_1616166611790/work tomli @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/tomli_1699472528897/work tomlkit @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/tomlkit_1699475633229/work toolz @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/toolz_1699474617784/work torch==2.5.0 tornado @ file:///C:/b/abs_7bua0304mj/croot/tornado_1718740122405/work tqdm @ file:///C:/b/abs_77wju137gk/croot/tqdm_1724853945787/work traitlets @ file:///C:/b/abs_bfsnoxl4pq/croot/traitlets_1718227069245/work transformers==4.45.2 Twisted @ file:///C:/b/abs_e7yqd811in/croot/twisted_1708702883769/work twisted-iocpsupport @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/twisted-iocpsupport_1699496891011/work typing_extensions @ file:///C:/b/abs_0as9mdbkfl/croot/typing_extensions_1715268906610/work tzdata @ file:///croot/python-tzdata_1690578112552/work uc-micro-py @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/uc-micro-py_1699545612984/work ujson @ file:///C:/b/abs_c93a278xy4/croot/ujson_1717598062082/work unicodedata2 @ file:///C:/b/abs_b6apldlg7y/croot/unicodedata2_1713212998255/work Unidecode @ file:///C:/b/abs_4cczv71djp/croot/unidecode_1724790062151/work urllib3 @ file:///C:/b/abs_9a_f8h_bn2/croot/urllib3_1727769836930/work w3lib @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/w3lib_1709162573908/work watchdog @ file:///C:/b/abs_b3l_3s276z/croot/watchdog_1717166538403/work wcwidth @ file:///Users/ktietz/demo/mc3/conda-bld/wcwidth_1629357192024/work webencodings @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/webencodings_1699497069416/work websocket-client @ file:///C:/b/abs_5dmnxxoci9/croot/websocket-client_1715878351319/work Werkzeug @ file:///C:/b/abs_8bittcw9jr/croot/werkzeug_1716533366070/work whatthepatch @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/whatthepatch_1699497134590/work wheel==0.44.0 widgetsnbextension @ file:///C:/b/abs_538o2crz7f/croot/widgetsnbextension_1710960099872/work win-inet-pton @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/win_inet_pton_1699472992992/work wrapt @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/wrapt_1699480231633/work xarray @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/xarray_1699480274069/work xlwings @ file:///C:/b/abs_bc7ro1xk70/croot/xlwings_1725400109105/work xxhash==3.5.0 xyzservices @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/xyzservices_1699548906163/work yapf @ file:///C:/b/abs_8aidayep18/croot/yapf_1708964372649/work yarl @ file:///C:/b/abs_00rcuicx5k/croot/yarl_1726015884215/work zict @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/zict_1699548979005/work zipp @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/zipp_1707493775475/work zope.interface @ file:///C:/Users/dev-admin/perseverance-python-buildout/croot/zope.interface_1699497164709/work

Context for the issue:

No response

cpfiffer commented 1 month ago

Related to https://github.com/ollama/ollama/issues/6473.

Basically, Ollama does not support structured generation, only JSON mode. JSON mode does not make any guarantees that you'll get the schema you asked for, or even that it will be valid. For the moment, I would recommend LM Studio as an alternative that supports Outlines. Here's a guide to using it.

lennartpollvogt commented 1 week ago

You can also use the instructor library or ollama-instructor (https://github.com/lennartpollvogt/ollama-instructor).

rlouf commented 1 week ago

Beware that Instructor doesn't do structured generation, it provides a way to parse the output of the LLM into a Pydantic object.