firebase / firebase-tools

The Firebase Command Line Tools
MIT License
4.01k stars 925 forks source link

Emulator error `Error: socket hang up` #6628

Open jhuleatt opened 1 year ago

jhuleatt commented 1 year ago

On my m1 mac, I get the following error when emulating a function with the BigQuery python SDK:

>  objc[122]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
>  objc[122]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
⬢  functions: Failed to handle request for function us-central1-commontrees
⬢  functions: Failed to start functions in /Users/jhuleatt/projects/io-python/functions: Error: socket hang up

here's the function code:

from firebase_functions import https_fn
from google.cloud import bigquery
import json

@https_fn.on_request(timeout_sec=20)
def commontrees(req: https_fn.Request) -> https_fn.Response:
    client = bigquery.Client()
    query = """
        SELECT spc_latin, spc_common, COUNT(spc_latin) as num_specimens
        FROM `bigquery-public-data.new_york_trees.tree_census_2015`
        WHERE SPC_LATIN <> ''
        GROUP BY spc_latin, spc_common
        ORDER BY num_specimens DESC
        LIMIT 50
    """
    query_job = client.query(query)

    common_trees = [];
    for row in query_job:
        common_trees.append({
            "scientificName": row['spc_latin'],
            "commonName": row['spc_common'],
            "count": row['num_specimens']
        })
    return https_fn.Response(json.dumps(common_trees),  mimetype='application/json')
jhuleatt commented 1 year ago

It seems like this might be specific to MacOS. Following the advice in this StackOverflow answer fixes the issue, but I'm not sure if it is something we want to recommend.

jamescash commented 1 year ago

Having the same issue. Above solution did not work for me.

  >  objc[61886]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
  >  objc[61886]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. 
 We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on 
 objc_initializeAfterForkError to debug.

` from firebase_functions import firestore_fn, https_fn
from firebase_admin import initialize_app, firestore
import google.cloud.firestore
from langchain.llms import OpenAI

app = initialize_app()

@https_fn.on_request()
def askQuestion(req: https_fn.Request) -> https_fn.Response:

    projectId = req.args.get("projectId")
    prompt = req.args.get("prompt")

    if projectId is None:
        return https_fn.Response("No projectId", status=400)
    if prompt is None:
        return https_fn.Response("No projectId", status=400)

    llm = OpenAI(temperature=0)
    print(llm(prompt))

    return https_fn.Response(f"Success!", status=200) `
jamescash commented 1 year ago

@jhuleatt Do you mind sharing your fix? This is not working for me:

import os os.environ['OBJC_DISABLE_INITIALIZE_FORK_SAFETY'] = 'YES'

taeold commented 1 year ago

Starting the emulator with this option does the trick for me:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
exaby73 commented 1 year ago

@jhuleatt Could you provide a reproducible example so that I may test this further?

peterfriese commented 1 year ago

I'm running into the same issue.

To repro, create a new Firebase Function module (using firebase init functions, and then change the code for on_request_example like this:

import requests

@https_fn.on_request()
def on_request_example(req: https_fn.Request) -> https_fn.Response:
  response = requests.get("https://www.google.com")
  return https_fn.Response("Hello world!")

This results in the following output:

⬢  functions: Failed to handle request for function us-central1-on_request_example
⬢  functions: Failed to start functions in helloworld: Error: socket hang up
exaby73 commented 1 year ago

Hello @peterfriese. Unfortunately, I am not able to reproduce this issue. I tried the following code:

import requests
from firebase_functions import https_fn, options
from firebase_admin import initialize_app

options.set_global_options(max_instances=10)

initialize_app()

@https_fn.on_request()
def on_request_example(req: https_fn.Request) -> https_fn.Response:
    response = requests.get("https://www.example.com") # Tried with google.com as well
    return https_fn.Response("Hello world!")

The following is my requirements.txt:

firebase-functions==0.1.1

My python version is Python 3.11.4

arcoyk commented 11 months ago

Solved same kind of error by overwriting the unused function.

import requests

@https_fn.on_request()
def the_deleted_function_name(req: https_fn.Request) -> https_fn.Response:
  pass

It seems emulator tries to find cached function that doesn't exist.

antont commented 11 months ago

@exaby73

Unfortunately, I am not able to reproduce this issue. I tried the following code:

Did you try on a Mac? I think I'm also using that OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

exaby73 commented 11 months ago

@antont I am on an M1 mac, without the OBJC_DISABLE_INITIALIZE_FORK_SAFETY flag set

antont commented 11 months ago

@exaby73 I am on an M1 mac, without the OBJC_DISABLE_INITIALIZE_FORK_SAFETY flag set

Ok. After OS upgrade and related restart, I didn't have the env var back yet, and again got that:

>  objc[26098]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
>  objc[26098]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
i  Request to function failed: Error: socket hang up

Then setting the flag fixed it.

I'll keep this env now so can provide details about versions etc. later if that helps.

exaby73 commented 11 months ago

Hey @antont. I'm wondering if you can reproduce this using Node as well. Since I can't reproduce this myself, I am unable to confirm it

google-oss-bot commented 8 months ago

Hey @jhuleatt. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

google-oss-bot commented 8 months ago

Since there haven't been any recent updates here, I am going to close this issue.

@jhuleatt if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.

AgaMiko commented 6 months ago

Exporting those two variables fixed the problem for me:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
export no_proxy=*
b0ot commented 5 months ago

Still an issue with me and my M1 Mac.

Going to try a couple of the fixes, don't really have a short example to reproduce.

EDIT: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES Allowed the emulator python cloud function to work

gregfenton commented 5 months ago

Is this now documented somewhere?

Am I correct to summarize this as:

b0ot commented 5 months ago

Am I correct to summarize this as:

  • MacOS
  • M1 chip (M2? M3? Intel?)
  • running Python cloud functions in the Emulator Suite
  • need to set env var export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES in order for the (Python engine ??) to run properly

Yes that is correct, at least for my case (Macbook Air, M1, 2020) Until I set that export, I was unable to have my python cloud functions run successfully in the Firebase Emulator.