langflow crashes with SIGABRT or SIGSEV when connecting to vanilla postgres in custom component on MacOS, ver 1.0.9 and 1.0.10 tested

vertgo commented 1 month ago

Bug Description

Langflow process stops and restarts when attempting to fork if building code that connects to a running postgres instance

+[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES is already set as an environment variable, and when running the debugger we see it is also set in Langflow if it detects darwin at main.py::set_var_for_macos_issue

Reproduction

QueryDB-POC-2.json

Set up a running postgres DB service (by default on localhost at port 5432)
open the flow
build the custom component

Expected behavior

The query returns normally

Who can help?

@ogabrielluiz

Operating System

macOS

Langflow Version

1.0.9, 1.0.10

Python Version

3.12

Screenshot

Flow File

QueryDB-POC-2.json

dosubot[bot] commented 1 month ago

To address the issue of Langflow crashing with SIGABRT or SIGSEV when connecting to a PostgreSQL database on macOS, you can try the following steps:

Ensure Correct Database URL Format: Make sure the database URL is correctly formatted. The SQLExecutorComponent class in Langflow replaces "postgresql://" with "postgres://". Ensure your database URL follows this format.
```
def clean_up_uri(self, uri: str) -> str:
   if uri.startswith("postgresql://"):
       uri = uri.replace("postgresql://", "postgres://")
   return uri.strip()
```

Set Environment Variables: Ensure the OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable is set to YES. This is necessary to avoid errors when running Gunicorn on macOS.

def set_var_for_macos_issue():
   if platform.system() in ["Darwin"]:
       import os
       os.environ["OBJC_DISABLE_INITIALIZE_FORK_SAFETY"] = "YES"
       os.environ["no_proxy"] = "*"  # to avoid error with gunicorn
       logger.debug("Set OBJC_DISABLE_INITIALIZE_FORK_SAFETY to YES to avoid error")

Use Uvicorn Instead of Gunicorn: As an alternative to using Gunicorn, you can run the server using Uvicorn, which bypasses the need for fork() and avoids the related issues on macOS.

def run_langflow(host, port, log_level, options, app):
   try:
       if platform.system() in ["Windows", "Darwin"]:
           import uvicorn
           uvicorn.run(
               app,
               host=host,
               port=port,
               log_level=log_level.lower(),
               loop="asyncio",
           )
       else:
           from langflow.server import LangflowApplication
           LangflowApplication(app, options).run()
   except KeyboardInterrupt:
       pass
   except Exception as e:
       logger.exception(e)
       sys.exit(1)

Enable Detailed Logging: Set the logging level to debug to get more detailed information about the crash. This can help in diagnosing the issue further.
```
LANGFLOW_LOG_LEVEL=debug
```
Check the log file specified by the log_file option for detailed logs.

By following these steps, you should be able to mitigate the crashes related to PostgreSQL connections on macOS. If the issue persists, consider updating to the latest version of Langflow, as there may have been fixes or improvements related to this problem in newer releases ^[1]^[2]^[3].

To continue talking to Dosu, mention @dosu.

carlosrcoelho commented 3 weeks ago

@vertgo

Are you experiencing this issue in the current version?

Current version 1.0.17

guilhermeneves commented 3 weeks ago

@vertgo @carlosrcoelho I encountered the same issue when connecting to a postgres instance from langflow, it was resolved by adjusting the following settings. I'm using Langflow version 1.0.17 on a MacBook Air M2:

export DISABLE_SPRING=true
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

vertgo commented 3 weeks ago

@guilhermeneves so I had already set the fork safety env variable.

The spring env variable also is necessary? We will test that. Thank you.

vertgo commented 2 weeks ago

@guilhermeneves That doesn't work for v.16, on a macbook pro intel.

Hmm we haven't been able to install 17, as there are looping errors trying to install a bunch of packages but dying at this one when trying to pip update langflow (def a separate issue):

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/clevercsv/ Collecting clevercsv (from qianfan==0.3.5->langflow) Using cached clevercsv-0.8.2-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB) Downloading clevercsv-0.8.1-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB) Downloading clevercsv-0.8.0-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB) Downloading clevercsv-0.7.7-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB) Downloading clevercsv-0.7.6-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB)

Downloading clevercsv-0.7.5-cp311-cp311-macosx_10_9_x86_64.whl.metadata (19 kB) Downloading clevercsv-0.7.4.tar.gz (70 kB)

vertgo commented 2 weeks ago

The bug still persists in v.0.18 @carlosrcoelho , even with @guilhermeneves 's env variables set from above

italojohnny commented 1 week ago

I was able to reproduce the error: Screenshot 2024-09-16 at 13 41 04

I'm working on it!

italojohnny commented 1 week ago

Hi @vertgo ,

I hope you're doing well.

I've been quite busy running multiple tests and digging deep to find a solution to this issue. I did manage to get it working, but since I made several changes along the way, I'm not exactly sure which one did the trick LOL. That’s where I need your help.

Could you try running Langflow in the same environment where you reported the issue, using the following command?

OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES langflow run

Please let me know if this solves the problem in your environment.

Thanks!

Oops... I just re-read the issue and saw @guilhermeneves 's responses. Despite that, @vertgo , could you confirm that copying and pasting the command I mentioned doesn’t work for you?

I have a few more things to share... can you confirm this first?

italojohnny commented 1 week ago

Issues #598 #1295 #1831 #2721 #3819 are related to this issue

vertgo commented 1 week ago

Hi @vertgo ,

I hope you're doing well.

I've been quite busy running multiple tests and digging deep to find a solution to this issue. I did manage to get it working, but since I made several changes along the way, I'm not exactly sure which one did the trick LOL. That’s where I need your help.

Could you try running Langflow in the same environment where you reported the issue, using the following command?
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES langflow run
Please let me know if this solves the problem in your environment.

Thanks!

Oops... I just re-read the issue and saw @guilhermeneves 's responses. Despite that, @vertgo , could you confirm that copying and pasting the command I mentioned doesn’t work for you?

I have a few more things to share... can you confirm this first?

A member of my team just tried this on an Intel core i7 mac, still getting the error

ERROR - init - Worker (pid:55809) was sent SIGSEGV!

italojohnny commented 1 week ago

I don't doubt that your team member is genuinely facing this issue. However, can you see that the problem isn't caused by a bug in Langflow, but rather by a dependency that fails under very specific conditions?

If your team member has reasons to believe this is a Langflow issue, we'll need more details about their machine to assist further.

Additionally, a quick workaround would be to run Langflow inside a Docker container.

Since you’re no longer encountering this problem, I'll go ahead and close the issue.

vertgo commented 1 week ago

I don't doubt that your team member is genuinely facing this issue. However, can you see that the problem isn't caused by a bug in Langflow, but rather by a dependency that fails under very specific conditions?

If your team member has reasons to believe this is a Langflow issue, we'll need more details about their machine to assist further.

Additionally, a quick workaround would be to run Langflow inside a Docker container.

Since you’re no longer encountering this problem, I'll go ahead and close the issue.

I'm not sure I understand. You just asked us to try calling langflow in this way and tell you what it did. Did you want us to do it on a new version?

Tbh we searched for that bug and did this first:

OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

That was already mentioned in the original bug report

( I also found in langflow already checks if it's on Mac os and sets that variable explicitly already). Yet that sigsev weirdly still arises. Did you commit a change that fixes it?

vertgo commented 1 week ago

Issues #598 #1295 #1831 #2721 #3819 are related to this issue

Those other errors seem to occur running langflow. We can run langflow just fine, this only arises when we do the postgres call I included in the bug report.

codenprogressive commented 1 week ago

@vertgo one thing I want to mention. maybe it is similar to you problem. One of my workflows were resulting into crushing my vm (i'm running langflow in a VM and I got SIGKILL) was because I loaded 4M rows in one of components. during the build, the flow will insert in vertx_build and transaction tables to log the input and output, etc... due to this huge volume when running again the flow this crashed my vm et got SIGKILL message. there must be a limit to not reach when inserting in Postgresql.

question for you, are loading huge volume of data in your components? trying isolate the issue. if not related, I will open a new issue and add a flow to reproduce the issue! cc @italojohnny

italojohnny commented 1 week ago

@vertgo, In your last message, you didn’t mention any more issues, so I assumed yours was resolved. Now we need the error details from your colleague to move forward.

I didn’t commit any changes to fix the issue, but during my investigation, I found the code that sets OBJC_DISABLE_INITIALIZE_FORK_SAFETY. It works fine when running as a contributor (make backend/frontend), but fails with pip, likely due to arbitrary code execution in Langflow.

If you're setting the variable correctly and still encountering errors, it seems external to Langflow. If your colleague believes it's a Langflow issue, we’ll need more details from their machine

langflow-ai / langflow