CUSTOM COMPONENTS disappeared

[fenss]

Describe the bug A clear and concise description of what the bug is.

Browser and Version

• Browser FireFox/Chrome
• Version 6.6.7a2

To Reproduce Steps to reproduce the behavior:

1. Writing codes, add 3 CUSTOM COMPONENTS, naming A/B/C.
2. Setting environment variable: export LANGFLOW_COMPONENTS_PATH='["{your_code_path}/langflow_custom_components/"]'.
3. Run langflow, then see the COMPONENTS in the 'Other' menu bar, and build some flows successfully, so the COMPONENTS' codes are fine.
4. Continue adding COMPONENTS, we call it D, after re-run langflow, we can only find 3 COMPONENTS in the 'Other' menu card, for example B/C/D.
5. Add COMPONENTS E, and still fine 3, like B/D/E, seems like it only support 3 CUSTOM COMPONENTS?

Screenshots

Here's the menu look like:

Here's my project codes:

Additional context Add any other context about the problem here.

[dosubot[bot]]

🤖

Hey @fenss, nice to see you around here again! How have things been since our last chat?

Thank you for providing a detailed description of the issue you're facing. Based on the information you've provided, it seems like you're experiencing an issue where only three custom components are being loaded at a time.

From the information available in the langflow repository, there doesn't appear to be a limit to the number of custom components that can be loaded. The CustomComponent class, which is used to handle custom components, doesn't seem to have any restrictions on the number of instances that can be created.

However, it's possible that there might be a bug or an unintended restriction somewhere in the code that's causing this issue. Without more information, it's difficult to say for sure what might be causing this.

To help troubleshoot this issue, could you please provide the following information:

1. Are there any error messages being displayed in the console when you try to load more than three custom components?
2. Could you provide the code for the custom components that you're trying to load? This could help identify if there's something specific in the code that's causing the issue.
3. Could you provide more details about the environment you're running langflow in? For example, what operating system are you using, and what version of Node.js and npm are you using?

This information could help identify what might be causing this issue. Once we have more information, we can look into this further and try to find a solution.

Please note that while I strive to provide accurate and helpful information, there may be a better answer that I wasn't able to find. If you have any further questions or need more help, please don't hesitate to ask.

Sources

• tests/test_custom_component.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[fenss]

🤖

Hey @fenss, nice to see you around here again! How have things been since our last chat?

Thank you for providing a detailed description of the issue you're facing. Based on the information you've provided, it seems like you're experiencing an issue where only three custom components are being loaded at a time.

From the information available in the langflow repository, there doesn't appear to be a limit to the number of custom components that can be loaded. The CustomComponent class, which is used to handle custom components, doesn't seem to have any restrictions on the number of instances that can be created.

However, it's possible that there might be a bug or an unintended restriction somewhere in the code that's causing this issue. Without more information, it's difficult to say for sure what might be causing this.

To help troubleshoot this issue, could you please provide the following information:

1. Are there any error messages being displayed in the console when you try to load more than three custom components?

2. Could you provide the code for the custom components that you're trying to load? This could help identify if there's something specific in the code that's causing the issue.

3. Could you provide more details about the environment you're running langflow in? For example, what operating system are you using, and what version of Node.js and npm are you using?

This information could help identify what might be causing this issue. Once we have more information, we can look into this further and try to find a solution.

Please note that while I strive to provide accurate and helpful information, there may be a better answer that I wasn't able to find. If you have any further questions or need more help, please don't hesitate to ask.

Sources

* [tests/test_custom_component.py](https://github.com/logspace-ai/langflow/blob/dev/tests/test_custom_component.py)

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Sure.

1. Not any error messages. As I said, the component runs fine, I've tested, it just disappeared when I add more, It's strange.
2. Yes but can just provide one of them, It's quite simple, I just override the langchain LLM class, to output the prompt itself for debug purpose, here's my codes:

# {custom_python_path}/langchain_custom_utils/llm_request_utils.py

from typing import Any, Dict, List, Mapping, Optional
from langchain.llms.base import LLM
from langchain_core.callbacks.manager import CallbackManagerForLLMRun

……

class ParrotLLM(LLM):
    tag: str
    @property
    def _llm_type(self) -> str:
        return 'ResponseAsParrot'
    @property
    def _identifying_params(self) -> Mapping[str, Any]:
        return {
            'tag': self.tag,
        }

    def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        return prompt

# {langflow_custom_components_path}/parrot_llm_component.py
from langchain.llms.base import LLM
from langflow import CustomComponent

from langchain_custom_utils.llm_request_utils import ParrotLLM

class ParrotLLMComponent(CustomComponent):
    display_name = 'Parrot LLM Component'
    description = 'Just repeat the prompt.'

    def build_config(self) -> dict:
        return {}

    def build(self, entry_point: str) -> LLM:
        llm = ParrotLLM(tag='parrot', cache=False)
        return llm

Before running langflow, I set export PYTHONPATH={custom_python_path}, so the component can import the utils.

It's just an example, my other components' similar like this, wrapping chains, prompts or llms.

3. Just use the Docker Image: FROM python:3.10.13-slim-bullseye

And pip install langflow==6.6.7a2, then run it…… I think it's not relevant to the NPM?

[dosubot[bot]]

🤖

Based on the information you've provided, it seems like your code for the custom components is correct. The issue you're experiencing might be related to how Langflow discovers and loads custom components.

Langflow uses the importlib module in Python to dynamically load modules at runtime. It uses the import_module function to import the module and the getattr function to get the class from the module. This is done in the import_class function, which is called by the import_custom_component function with the fully qualified name of the CustomComponent class.

Here's the relevant code from Langflow:

def import_module(module_path: str) -> Any:
    """Import module from module path"""
    if "from" not in module_path:
        # Import the module using the module path
        return importlib.import_module(module_path)
    # Split the module path into its components
    _, module_path, _, object_name = module_path.split()

    # Import the module using the module path
    module = importlib.import_module(module_path)

    return getattr(module, object_name)

def import_class(class_path: str) -> Any:
    """Import class from class path"""
    module_path, class_name = class_path.rsplit(".", 1)
    module = import_module(module_path)
    return getattr(module, class_name)

def import_custom_component(custom_component: str) -> CustomComponent:
    """Import custom component from custom component name"""
    return import_class("langflow.interface.custom.custom_component.CustomComponent")

(Source: src/backend/langflow/interface/importing/utils.py)

Given this, there are a few things you could check:

1. Make sure the PYTHONPATH environment variable is correctly set to include the path to your custom components. You can check this by running echo $PYTHONPATH in your terminal.

2. Check if the fully qualified name of your CustomComponent class is correct. The fully qualified name should include the module path and the class name, separated by a dot. For example, in your case, it should be something like {langflow_custom_components_path}.parrot_llm_component.ParrotLLMComponent.

3. Make sure the CustomComponent class is correctly imported in your custom component file. In your case, it should be from langflow import CustomComponent.

If you've checked all of these and the issue still persists, it might be helpful to add some logging in the import_class and import_custom_component functions to see what's going wrong. For example, you could log the module_path, class_name, and the imported module and class.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• tests/test_custom_component.py
• src/backend/langflow/interface/importing/utils.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[fenss]

🤖

Based on the information you've provided, it seems like your code for the custom components is correct. The issue you're experiencing might be related to how Langflow discovers and loads custom components.

Langflow uses the importlib module in Python to dynamically load modules at runtime. It uses the import_module function to import the module and the getattr function to get the class from the module. This is done in the import_class function, which is called by the import_custom_component function with the fully qualified name of the CustomComponent class.

Here's the relevant code from Langflow:

def import_module(module_path: str) -> Any:
    """Import module from module path"""
    if "from" not in module_path:
        # Import the module using the module path
        return importlib.import_module(module_path)
    # Split the module path into its components
    _, module_path, _, object_name = module_path.split()

    # Import the module using the module path
    module = importlib.import_module(module_path)

    return getattr(module, object_name)

def import_class(class_path: str) -> Any:
    """Import class from class path"""
    module_path, class_name = class_path.rsplit(".", 1)
    module = import_module(module_path)
    return getattr(module, class_name)

def import_custom_component(custom_component: str) -> CustomComponent:
    """Import custom component from custom component name"""
    return import_class("langflow.interface.custom.custom_component.CustomComponent")

(Source: src/backend/langflow/interface/importing/utils.py)

Given this, there are a few things you could check:

1. Make sure the `PYTHONPATH` environment variable is correctly set to include the path to your custom components. You can check this by running `echo $PYTHONPATH` in your terminal.

2. Check if the fully qualified name of your `CustomComponent` class is correct. The fully qualified name should include the module path and the class name, separated by a dot. For example, in your case, it should be something like `{langflow_custom_components_path}.parrot_llm_component.ParrotLLMComponent`.

3. Make sure the `CustomComponent` class is correctly imported in your custom component file. In your case, it should be `from langflow import CustomComponent`.

If you've checked all of these and the issue still persists, it might be helpful to add some logging in the import_class and import_custom_component functions to see what's going wrong. For example, you could log the module_path, class_name, and the imported module and class.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

* [tests/test_custom_component.py](https://github.com/logspace-ai/langflow/blob/dev/tests/test_custom_component.py)

* [src/backend/langflow/interface/importing/utils.py](https://github.com/logspace-ai/langflow/blob/dev/src/backend/langflow/interface/importing/utils.py)

About Dosu

It's not helpful, the components have tested, for example A, B and C, I ran a flow using A+B+C, It's worked. Then I add D, one of A/B/C disappeared, and I can see D in the menu bar, it's not the codes problem or environment variables problem I think.

[dosubot[bot]]

🤖

Based on the information provided, there is no limit set in the Langflow codebase that restricts the number of custom components that can be loaded at a time. The build_component_menu_list function in the DirectoryReader class reads all .py files in the given directory and processes each file to build a component, without any specified limit. You can find this in the directory_reader.py file.

As for the order of loading, Langflow does not have a specific order or priority in which custom components are loaded. The loading of custom components is handled by the load_flow method in the CustomComponent class. This method retrieves the flow data from the database using the provided flow_id. If a tweaks dictionary is provided, it is processed and applied to the graph_data. The graph_data is then passed to the build_sorted_vertices function along with the user_id. The build_sorted_vertices function is responsible for building the vertices of the flow graph in a sorted order. You can find this in the custom_component.py file.

Given this, it's unclear why only three components are being loaded at a time in your case. It might be helpful to check the logs for any errors or warnings during the component loading process. Also, ensure that all your custom components are correctly defined and that they are all in the directory specified by the PYTHONPATH environment variable.

If you've already checked these and the issue persists, could you provide more information about your setup? Specifically, it would be helpful to know:

• The exact command you're using to run Langflow
• The full directory structure of your custom components
• Any error messages or warnings you're seeing in the logs

This additional information will help us better understand the issue and provide a more accurate solution.

Sources

• src/backend/langflow/interface/custom/directory_reader/directory_reader.py
• src/backend/langflow/interface/custom/custom_component/custom_component.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[fenss]

I found the reason:

While logging to the langflow UI, I checked the requests, found sth like {your_langchain_url}/api/v1/all, the response contains the UI definition:

response = requests.get(
    '{your_langflow_url}/api/v1/all',
    headers={
        'x-api-key': '{your_api_key}',
    },
)

print(response.json().keys())

# dict_keys(['chains', 'agents', 'prompts', 'llms', 'memories', 'tools', 'toolkits', 'wrappers', 'embeddings', 'documentloaders', 'textsplitters', 'utilities', 'output_parsers', 'langflow_custom_components', 'retrievers', 'custom_components', 'vectorstores'])

Pick some keys to see more:

print(response.json()['chains'].keys())

# dict_keys(['ConversationalRetrievalChain', 'LLMCheckerChain', 'LLMMathChain', 'RetrievalQA', 'RetrievalQAWithSourcesChain', 'SQLDatabaseChain', 'CombineDocsChain', 'SeriesCharacterChain', 'MidJourneyPromptChain', 'TimeTravelGuideChain', 'Document', 'ConversationChain', 'BaseQAWithSourcesChain', 'LLMChain'])

print(response.json()['custom_components'].keys())
# dict_keys(['Data'])

We found that it use the component's return type to identify each component, in the upper cases, the official "custom_components" group contains only one component "CustomComponent", and the key is "Data", the return type of itself.

So if we create some components with the same return type, like LLM(langchain.llms.base), they overwrite each other(on the UI).

the temporary solution is wrap each CustomComponent's return value, but I think it's not a good way.