Backend/base pyproject missing vectorstore dependencies

[jordanrfrazier]

Describe the bug

Overall question: What is the purpose of the dependency separation in backend/base/langflow - is it intended as an option for users to import minimal dependencies (w/o frontend) to run in their applications? Can you explain the usage and roadmap a bit further?

Bug: The backend pulls in dependencies that are not included (even in extras): https://github.com/langflow-ai/langflow/blob/f9f1d9ba14735ef0adfab779a5ec692ad3aebe0e/src/backend/base/pyproject.toml#L26-L27

For example, if using a vector store component, all vector stores need to be installed to run. https://github.com/langflow-ai/langflow/blob/05cd6e4fd71866b6328ee669b7823def671c1a42/src/backend/base/langflow/components/vectorstores/__init__.py#L13

Browser and Version

• Browser [e.g. chrome, safari]: N/A
• Version [e.g. 22]: (Langflow version 1.0.0a30)

To Reproduce Steps to reproduce the behavior:

1. create env with backend deps
2. import a vector store component in a python file

[dosubot[bot]]

The issue you're experiencing with the langflow project, where the backend pulls in unlisted dependencies, particularly for vector store components, seems to stem from the project's approach to dependency management. The project aims to modularize dependencies, especially around vector stores, to allow users the flexibility to choose which components to install. This approach is intended to minimize the installation of unnecessary dependencies for users who do not require support for all vector stores.

To address the problem of unlisted dependencies being required, a solution is to implement conditional imports based on the project's configuration or the presence of specific packages. This would enable the project to dynamically load only the necessary vector store components. Here's a simplified example of how conditional imports can be structured:

vector_store_components = []

try:
    from .AstraDB import AstraDBVectorStoreComponent
    vector_store_components.append('AstraDBVectorStoreComponent')
except ImportError:
    pass

try:
    from .Chroma import ChromaComponent
    vector_store_components.append('ChromaComponent')
except ImportError:
    pass

# Repeat for other vector store components...

__all__ = vector_store_components

In this setup, each vector store component is attempted to be imported within a try block. If the import is successful, the component is added to the __all__ list. If the import fails (caught by the except ImportError), the component is skipped. This ensures that only the dependencies actually installed in the user's environment are loaded, mitigating the issue of pulling in unlisted dependencies.

This approach aligns with the project's goal of modularizing dependencies to reduce footprint and avoid unnecessary installations, providing a more flexible and user-friendly setup.

#### Details

**Sources** - [src/backend/base/langflow/components/vectorstores/__init__.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/base/langflow/components/vectorstores/__init__.py) - [pyproject.toml](https://github.com/logspace-ai/langflow/blob/dev/pyproject.toml) - [pyproject.toml](https://github.com/langflow-ai/langflow/blob/dev/pyproject.toml) - [src/backend/base/pyproject.toml](https://github.com/logspace-ai/langflow/blob/dev/src/backend/base/pyproject.toml) - [src/backend/base/pyproject.toml](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/pyproject.toml) - [src/backend/base/langflow/components/vectorstores/base/__init__.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/base/langflow/components/vectorstores/base/__init__.py) - [src/backend/langflow/components/toolkits/VectorStoreToolkit.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/langflow/components/toolkits/VectorStoreToolkit.py) - [src/backend/langflow/components/vectorstores/Vectara.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/langflow/components/vectorstores/Vectara.py) - [src/backend/base/langflow/components/vectorstores/Vectara.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/base/langflow/components/vectorstores/Vectara.py) - [src/backend/langflow/__init__.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/langflow/__init__.py) - [src/backend/base/langflow/initial_setup/starter_projects/VectorStore-RAG-Flows.json](https://github.com/langflow-ai/langflow/blob/dev/src/backend/base/langflow/initial_setup/starter_projects/VectorStore-RAG-Flows.json) - [src/backend/base/langflow/components/vectorstores/base/model.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/base/langflow/components/vectorstores/base/model.py) - [src/backend/langflow/components/agents/VectorStoreAgent.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/langflow/components/agents/VectorStoreAgent.py)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[nicoloboschi]

@ogabrielluiz @anovazzi1 @lucaseduoli can you please provide more information here?

[ogabrielluiz]

Hey all

Sorry for the delayed response.

This is going to be improved over time but the idea is this: If you want to have langflow with minimal dependencies and install your own to keep it more lightweight, you do so by installing langflow-base. The langflow package installs many more dependencies

We plan on checking which components have their dependencies installed later on so that components that can't run, don't show up on the sidebar.

[jordanrfrazier]

Thanks Gabriel, that sounds good. I'll close for this knowing that those improvements are planned.