Installing from source (pip install -e .) fails with multiple top-level packags found

[ichernev]

System Info

Linux, Python 3.11.5

Who can help?

No response

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

1. python -m venv .venv
2. . .venv/bin/activate
3. pip install -e .

Expected behavior

To install package in a local virtualenv.

What happens:

% pip install -e .  
Obtaining file:///home/iskren/src/deepinfra/libs/langchain
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      error: Multiple top-level packages discovered in a flat-layout: ['libs', 'langchain'].

      To avoid accidental inclusion of unwanted files or directories,
      setuptools will not proceed with this build.

      If you are trying to create a single distribution with multiple packages
      on purpose, you should not rely on automatic discovery.
      Instead, consider the following options:

      1. set up custom discovery (`find` directive with `include` or `exclude`)
      2. use a `src-layout`
      3. explicitly set `py_modules` or `packages` with a list of names

      To find more information, look for "package discovery" on setuptools docs.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[dosubot[bot]]

🤖

Hello,

The error message you're seeing is due to the presence of multiple top-level packages in the repository. This is not allowed by setuptools when using a flat layout, which is the case when you're running pip install -e ..

The error message suggests three possible solutions:

1. Set up custom discovery (find directive with include or exclude)
2. Use a src-layout
3. Explicitly set py_modules or packages with a list of names

Given the structure of the LangChain repository, the third option seems to be the most appropriate. You can specify the packages to be installed in the setup.py file or pyproject.toml file, depending on which one you're using.

If you're using a setup.py file, you can specify the packages like this:

from setuptools import setup, find_packages

setup(
    name="LangChain",
    version="0.0.296",
    packages=find_packages(),
)

If you're using a pyproject.toml file, you can specify the packages like this:

[tool.poetry]
name = "LangChain"
version = "0.0.296"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.dependencies]
python = "^3.8"

In both cases, the find_packages() function will automatically discover all packages and subpackages. If you want to exclude certain packages, you can pass them as arguments to the exclude parameter.

Please try this solution and let me know if it works for you. If you encounter any other issues, feel free to ask.

Best regards, Dosu

Sources

• libs/langchain/langchain/chains/init.py
• libs/langchain/langchain/init.py
• libs/langchain/pyproject.toml

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ichernev]

So you need to: cd libs/langchain and do pip install -e . from there...

[emhagman]

So you need to: cd libs/langchain and do pip install -e . from there...

For anyone looking for a solution for requirements.txt here, you can use: langchain @ git+https://github.com/langchain-ai/langchain@master#subdirectory=libs/langchain

The #subdirectory= feature tells it where to find the actually library.