[datasets] investigate feasibility of using fastapi

[axiomofjoy]

We should be able to use fastapi without pinning a version of Pydantic by using standard library dataclasses: https://fastapi.tiangolo.com/advanced/dataclasses/. It sounds like using pydantic models might occasionally be necessary for deeply nested objects. The library provides a few options here, including a pydantic.dataclasses module that emulates the built-in modules and has undergone what sound like relatively minor changes between versions 1 and 2: https://docs.pydantic.dev/latest/migration/#changes-to-dataclasses. There is a pydantic.v1 module in v2 that provides access to the v1 API. These sound like they could provide us with escape hatches in cases where built-in dataclasses are insufficient.

[axiomofjoy]

It's possible to install a slim version of fastapi and add only the dependencies that you need. https://fastapi.tiangolo.com/#fastapi-slim

[axiomofjoy]

pydantic versions

llama-index unpinned dspy ^2 langchain ">=1,<3" openai >=1.9.0, <3

[axiomofjoy]

Some differences between pydantic v1 and v2 for adding extra JSON schema information https://fastapi.tiangolo.com/tutorial/schema-extra-example/#extra-json-schema-data-in-pydantic-models