Package request: DSPy (language model framework)

[ethanniser]

Project description

DSPy is a framework for algorithmically optimizing LM prompts and weights

Its a python package

would really appreciate someone putting a package together, or helping me figure it out

thanks!

Metadata

• homepage URL: https://dspy-docs.vercel.app/
• source URL: https://github.com/stanfordnlp/dspy
• license: mit
• platforms: python

---

Add a :+1: reaction to issues you find important.

[ethanniser]

        dspy = with pkgs.python312.pkgs;
          buildPythonPackage rec {
            pname = "dspy-ai";
            version = "2.4.10";
            format = "setuptools";

            src = pkgs.fetchFromGitHub {
              owner = "stanfordnlp";
              repo = "dspy";
              rev = "refs/tags/v${version}";
              hash = "sha256-jfPg1W9Qp0DlAbMXaFqZ6Ri2zMOW6EKUHwi7Azn/yl0=";
            };

            build-system = [setuptools];

            meta = with lib; {
              description = "The framework for programming—not prompting—foundation models";
              downloadPage = "https://github.com/stanfordnlp/dspy";
              homepage = "https://dspy-docs.vercel.app/";
              license = with licenses; [mit];
            };
          };

as far as I got nix loads it but dspy isnt available in python

[eclairevoyant]

nix-init in a local nixpkgs clone might give you a better starting point, you'll likely still have to tweak though

[ethanniser]

{
  lib,
  python3,
  fetchPypi,
}:
python3.pkgs.buildPythonApplication rec {
  pname = "dspy";
  version = "2.4.10";
  pyproject = true;

  src = fetchPypi {
    pname = "dspy-ai";
    inherit version;
    hash = "sha256-lAGfHQ8M2iKvHZ1xcil/yAAqhZoPCqUb68gR9N0fDxQ=";
  };

  nativeBuildInputs = [
    python3.pkgs.setuptools
    python3.pkgs.wheel
  ];

  propagatedBuildInputs = with python3.pkgs; [
    backoff
    datasets
    jinja2
    joblib
    openai
    optuna
    pandas
    pydantic
    regex
    requests
    rich
    structlog
    tqdm
    ujson
  ];

  passthru.optional-dependencies = with python3.pkgs; {
    aws = [
      boto3
    ];
    chromadb = [
      chromadb
    ];
    docs = [
      autodoc_pydantic
      docutils
      furo
      m2r2
      myst-nb
      myst-parser
      sphinx
      sphinx-autobuild
      sphinx-automodapi
      sphinx-reredirects
      sphinx_rtd_theme
    ];
    epsilla = [
      pyepsilla
    ];
    fastembed = [
      fastembed
    ];
    marqo = [
      marqo
    ];
    milvus = [
      pymilvus
    ];
    pinecone = [
      pinecone-client
    ];
    postgres = [
      pgvector
      psycopg2
    ];
    qdrant = [
      fastembed
      qdrant-client
    ];
    weaviate = [
      weaviate-client
    ];
  };

  pythonImportsCheck = ["dspy_ai"];

  meta = with lib; {
    description = "The framework for programming—not prompting—foundation models";
    downloadPage = "https://github.com/stanfordnlp/dspy";
    homepage = "https://dspy-docs.vercel.app/";
    license = licenses.mit;
  };
}

got to this but this cant get it to load in python

[ethanniser]

update:

@NobbZ on discord helped me out but it seems theres some upstream problems

Where is poetry? Wait… What is that? Which of these is canonical? Please ask upstream to fix their pyproject.toml As far as I can tell, those lists do not even agree…

Why do they mention poetry in the README, if the pyproject.toml states that setuptools was used? Why do they specify dependencies in setup.py (and a variety of requirements.txts) and in the project.dependencies list and in the tool.poetry.dependencies list? Why do those lists even disagree?

ive created stanfordnlp/dspy#1234

{ lib
, buildPythonPackage
, fetchPypi
, setuptools
, wheel
, backoff
, datasets
, jinja2
, joblib
, openai
, optuna
, pandas
, pydantic
, regex
, requests
, rich
, structlog
, tqdm
, ujson
, boto3
, chromadb
, autodoc-pydantic
, docutils
, furo
, m2r2
, myst-nb
, myst-parser
, sphinx
, sphinx-autobuild
, sphinx-automodapi
, sphinx-reredirects
, sphinx-rtd-theme
, pyepsilla
, fastembed
, marqo
, pymilvus
, pinecone-client
, pgvector
, psycopg2
, qdrant-client
, weaviate-client
}:

buildPythonPackage rec {
  pname = "dspy";
  version = "2.4.10";
  pyproject = true;

  src = fetchPypi {
    pname = "dspy-ai";
    inherit version;
    hash = "sha256-lAGfHQ8M2iKvHZ1xcil/yAAqhZoPCqUb68gR9N0fDxQ=";
  };

  nativeBuildInputs = [
    setuptools
    wheel
  ];

  propagatedBuildInputs = [
    backoff
    datasets
    jinja2
    joblib
    openai
    optuna
    pandas
    pydantic
    regex
    requests
    rich
    structlog
    tqdm
    ujson
  ];

  passthru.optional-dependencies = {
    aws = [
      boto3
    ];
    chromadb = [
      chromadb
    ];
    docs = [
      autodoc_pydantic
      docutils
      furo
      m2r2
      myst-nb
      myst-parser
      sphinx
      sphinx-autobuild
      sphinx-automodapi
      sphinx-reredirects
      sphinx_rtd_theme
    ];
    epsilla = [
      pyepsilla
    ];
    fastembed = [
      fastembed
    ];
    marqo = [
      marqo
    ];
    milvus = [
      pymilvus
    ];
    pinecone = [
      pinecone-client
    ];
    postgres = [
      pgvector
      psycopg2
    ];
    qdrant = [
      fastembed
      qdrant-client
    ];
    weaviate = [
      weaviate-client
    ];
  };

  pythonImportsCheck = [ "dspy_ai" ];

  meta = with lib; {
    description = "The framework for programming—not prompting—foundation models";
    downloadPage = "https://github.com/stanfordnlp/dspy";
    homepage = "https://dspy-docs.vercel.app/";
    license = licenses.mit;
  };
}