honeybadger-io / honeybadger-python

Send Python and Django errors to Honeybadger.
https://www.honeybadger.io/
MIT License
15 stars 25 forks source link

Honeybadger reports exception raised in thread, even when it is handled successfully by the caller. #168

Closed epgui closed 4 months ago

epgui commented 4 months ago

I have the following code (see comments):

# Uses threading: https://github.com/kata198/func_timeout/blob/master/func_timeout/StoppableThread.py
from func_timeout import FunctionTimedOut, func_timeout

def thread_timeout(
    seconds: DecimalSeconds = 10.0,
    f_handle_timeout: TimeoutHandler = _default_timeout_handler,
):
    """Intended to be used as a parameterized decorator."""

    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            try:
                return func_timeout(
                    timeout=seconds,
                    func=f,
                    args=args,
                    kwargs=kwargs,
                )
            except FunctionTimedOut:
                f_handle_timeout()

        return wrapper

    return decorator

def _handle_timeout():
    raise GoogleSheetTimeoutError(
        # ... (not the interesting part)
    )

class HttpError(Exception):
    def __init__(self, response: HTTPResponse):
        self.response = response
        self.content = _error_contents(response._body)  # Implementation not shown -- not interesting
        self.url = response.url

    @property
    def status_code(self):
        return self.response.status

@thread_timeout(seconds=90, f_handle_timeout=_handle_timeout)
def _execute_with_timeout(request: Callable[[], HTTPResponse]) -> Any:
    response: HTTPResponse = request()
    if response.status == 200:
        return response.json()
    else:
        # The error is raised here and is not caught within the thread but is caught in `perform_request(...)`
        # Honeybadger nonetheless reports the error, with a stack trace that shows only what happens inside the thread.
        raise HttpError(response)

@with_rate_limiting
def perform_request(
    request: Callable[[], HTTPResponse],
    retries: int = 5,
) -> Any:
    try:
        return _execute_with_timeout(request)

    except HttpError as e:
        if e.status_code == 503:
            # This error gets caught and handled correctly. My data pipeline completes successfully.
            # Honeybadger still reports the error that occurred inside the thread

Elsewhere I set up honeybadger as follows:

from honeybadger import honeybadger
from honeybadger.contrib.logger import HoneybadgerHandler

def setup_logging(environment: str, honeybadger_api_key: str):
    root = logging.getLogger()
    root.setLevel(logging.DEBUG)

    # ...

    # Add honeybadger to the logging handler so that we capture ERROR logs
    hh = HoneybadgerHandler(api_key=honeybadger_api_key)
    hh.honeybadger.configure(environment=environment)
    hh.setLevel(logging.ERROR)
    root.addHandler(hh)

    # Add honeybadger directly, so that we capture uncaught Exceptions
    honeybadger.configure(
        api_key=honeybadger_api_key,
        environment=environment,
        report_local_variables=True,
        params_filters=["password", "password_confirmation", "params_filters", "token"],
    )

    # ...

I am running this in AWS Fargate, in a docker container (image python:3.10.8, with build argument --platform linux/amd64), with python 3.10.8 and honeybadger 4.3.5.

Clearly I don't want Honeybadger to alert me when my exceptions are being handled by the parent thread, so what is the intended usage here?

subzero10 commented 4 months ago

Hey @epgui, can you please attach an example of the reported error? Specifically, I'd like to see if we can identify whether the error is reported from the logging handler or the uncaught exceptions handler.

epgui commented 4 months ago

Hey @epgui, can you please attach an example of the reported error? Specifically, I'd like to see if we can identify whether the error is reported from the logging handler or the uncaught exceptions handler.

This is embarrassing, but I think we can chalk this one up to user error, the user error being an explicit invocation of honeybadger.notify() (passing in the original exception from within the thread) in the except HttpError as e block... 😂

subzero10 commented 3 months ago

Oh, I see 😄 . No worries! I'm glad that you sorted it out. Thank you for using Honeybadger!