search

Miserlou / Zappa

Serverless Python
https://blog.zappa.io/
MIT License
11.89k stars 1.2k forks source link

Cannot invoke task: TypeError: Object of type 'LambdaContext' is not JSON serializable #1097

Open robwatkiss opened 7 years ago

robwatkiss commented 7 years ago

Context

When invoking tasks a TypeError is raised as zappa tries to JSON encode the LambdaContext object.

Expected Behavior

The task should be invoked

Actual Behavior

A TypeError is raised as context.identity (CognitoIdenity instance) cannot be serialized as it has no '__dict__' attribute and hence cannot be serialized.

Object of type 'LambdaContext' is not JSON serializable: TypeError
Traceback (most recent call last):
  File "/var/task/handler.py", line 505, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 242, in lambda_handler
    return handler.handler(event, context)
  File "/var/task/handler.py", line 341, in handler
    result = self.run_function(app_function, event, context)
  File "/var/task/handler.py", line 272, in run_function
    result = app_function(event, context) if varargs else app_function()
  File "/var/task/zappa/async.py", line 344, in _run_async
    aws_region=aws_region).send(task_path, args, kwargs)
  File "/var/task/zappa/async.py", line 143, in send
    self._send(message)
  File "/var/task/zappa/async.py", line 151, in _send
    payload = json.dumps(message).encode('utf-8')
  File "/var/lang/lib/python3.6/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/lang/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'LambdaContext' is not JSON serializable

Possible Fix

Remove unserializable types before serialization or define custom encoder. It's a horrible solution, I admit.

# create dict from context with serializable types (this list almost certainly missing many valid types!)
context_serializable = {k:v for k, v in context.__dict__.items() if type(v) in [int, float, bool, str, list, dict]}
json.dumps(context_serializable)

Steps to Reproduce

  1. Create test task 
    
from zappa.async import task

@task def my_func(): print("All good!")

2. Deploy it
3. Invoke it `zappa invoke bugtest 'test.my_func'`

## Your Environment
* Zappa version used: 0.43.2
* Operating System and Python version: Ubuntu, Python 3.6
* The output of `pip freeze`:

argcomplete==1.8.2 base58==0.2.4 boto3==1.4.5 botocore==1.5.40 certifi==2017.7.27.1 chardet==3.0.4 click==6.7 docutils==0.14 durationpy==0.5 future==0.16.0 futures==3.1.1 hjson==3.0.0 idna==2.6 jmespath==0.9.3 kappa==0.6.0 lambda-packages==0.16.1 placebo==0.8.1 python-dateutil==2.6.1 python-slugify==1.2.4 PyYAML==3.12 requests==2.18.4 s3transfer==0.1.11 six==1.10.0 toml==0.9.2 tqdm==4.15.0 troposphere==1.9.5 Unidecode==0.4.21 urllib3==1.22 Werkzeug==0.12 wsgi-request-logger==0.4.6 zappa==0.43.2

* Your `zappa_settings.py`:

{ "bugtest": { "app_function": "test.my_func", "aws_region": "eu-west-2", "profile_name": "turbogram", "s3_bucket": "zappa-bug-test" } }

mcrowson commented 7 years ago

I'm not opposed to this solution. I was attempting to return another raised error type and its lack of JSON-afinity made it super ugly.

robwatkiss commented 7 years ago

So I have been playing around with this today to try and come to a solution, however, I believe I may have stumbled on another issue.

First, my solution:

import json
from zappa.async import task, ASYNC_CLASSES, LambdaAsyncResponse

# Define a custom encoder which casts non base types to string
class LambdaMessageEncoder(json.JSONEncoder):
    def default(self, obj):
        if not type(obj) in [int, float, complex, dict, tuple, list, bool] and obj is not None:
            return str(obj)

        return json.JSONEncoder.default(self, obj)

# Extend LambdaAsyncResponse to rework the _send method
# The send method is only included due to a bug where response_id is not set
class TestLambdaAsyncResponse(LambdaAsyncResponse):
    def send(self, task_path, args, kwargs):
        """
        Create the message object and pass it to the actual sender.
        """
        message = {
                'task_path': task_path,
                'capture_response': getattr(self, 'capture_response', False),
                'response_id': getattr(self, 'response_id', None),
                'args': args,
                'kwargs': kwargs
            }
        self._send(message)
        return self

    def _send(self, message):
        """
        Given a message, directly invoke the lamdba function for this task.
        """
        message['command'] = 'zappa.async.route_lambda_task'

        # Use custom encoder class
        payload = json.dumps(message, cls=LambdaMessageEncoder).encode('utf-8')

        if len(payload) > 128000: # pragma: no cover
            raise AsyncException("Payload too large for async Lambda call")

        self.response = self.client.invoke(
                                    FunctionName=self.lambda_function_name,
                                    InvocationType='Event', #makes the call async
                                    Payload=payload
                                )
        self.sent = (self.response.get('StatusCode', 0) == 202)

# Add service to ASYNC_CLASSES
ASYNC_CLASSES['test'] = TestLambdaAsyncResponse

@task()
def my_func_default_encoder(*args, **kwargs):
    print("All good, default encoder!")

@task(service='test')
def my_func_with_encoder(*args, **kwargs):
    print("All good, new encoder!")

This fixes our issues with LambdaContext being unserializable but raises a new issue as Zappa is trying to call the method asynchronously when we invoke it through the command line. The response object, TestLambdaAsyncResponse, is in turn unserializable. Output below:

Calling invoke for stage bugtest..
[DEBUG] 2017-09-12T11:52:17.97Z d327d303-97b0-11e7-86c5-114d325c08e9 StringToSign:
AWS4-HMAC-SHA256
20170912T115217Z
20170912/eu-west-2/lambda/aws4_request..............
[DEBUG] 2017-09-12T11:52:17.97Z d327d303-97b0-11e7-86c5-114d325c08e9 Signature:
..................
[DEBUG] 2017-09-12T11:52:17.98Z d327d303-97b0-11e7-86c5-114d325c08e9 Sending http request: <PreparedRequest [POST]>
[INFO]  2017-09-12T11:52:17.98Z d327d303-97b0-11e7-86c5-114d325c08e9    Starting new HTTPS connection (2): lambda.eu-west-2.amazonaws.com
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 "POST /2015-03-31/functions/zappabug-bugtest/invocations HTTP/1.1" 2020
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Response headers: {'content-type': '', 'date': 'Tue, 12 Sep 2017 11:52}
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Response body:
<botocore.response.StreamingBody object at 0x7f72a708df98>
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Event needs-retry.lambda.Invoke: calling handler <botocore.retryhandle>
[DEBUG] 2017-09-12T11:52:17.212Z d327d303-97b0-11e7-86c5-114d325c08e9 No retry needed.
Result of test.my_func_with_encoder:
<test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30>
An error occurred during JSON serialization of response: <test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30> is not JSON serializable
Traceback (most recent call last):
  File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/runtime/awslambda/bootstrap.py", line 110, in decimal_serializer
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30> is not JSON serializable
[END] RequestId: d327d303-97b0-11e7-86c5-114d325c08e9
[REPORT] RequestId: d327d303-97b0-11e7-86c5-114d325c08e9
Duration: 131.44 ms
Billed Duration: 200 ms 
Memory Size: 512 MB
Max Memory Used: 34 MB

I'm not sure my understanding of the internals of Zappa is quite strong enough to tackle this one, would appreciate some guidance/thoughts on how to tackle this.

robwatkiss commented 7 years ago

Further to the above, I believe an error in the TestLambdaAsyncResponse method was causing Zappa to retry calling our method.

I need to find the time to work through this issue as right now I have very little confidence in my solution.

abdulwahid24 commented 6 years ago

@robwatkiss any update on this?

mesuutt commented 6 years ago

I am getting same error.

I added task decorator to function and I am calling the function from zappa_settings.json shown as below:

"events": [
    {
        "function": "tasks.print_for_test_lambda",
        "expression": "rate(1 minute)"
    }
],

Project directory structure:


rootDir/
   tasks.py
   zappa_settings.json
  ...
robwatkiss commented 6 years ago

@mesuutt to have a method execute from an event it can't have the task decorator (based on my last tests ~2 months ago)

mcrowson commented 6 years ago

I believe that when the event calls a function with a task decoy or the event launches a separate lambda function to do the task.

Sent from my iPhone

On Nov 27, 2017, at 11:52 AM, Rob Watkiss notifications@github.com wrote:

@mesuutt to have a method execute from an event it can't have the task decorator (based on my last tests ~2 months ago)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

abdulwahid24 commented 6 years ago

@mcrowson Yes, I do agree with you but I am still confused that If the task function has some dependencies on imported modules then how would it going to run.

Please look at below snippet.

import os
from flask import Flask
from flask_restful import Api as FlaskRestfulAPI, Resource,reqparse
from elasticsearch import Elasticsearch
from zappa.async import task

from config import config
from analyzer import Paradigmatic

app = Flask(__name__)
app.config.from_object(config['dev'])

class ParadigmaticResource(Resource):
    post_parser = reqparse.RequestParser()
    post_parser.add_argument('word1', required=True, location='json')
    post_parser.add_argument('word2', required=True, location='json')

    @task
    def calculate_paradigmatic(self, *args, **kwargs):
        paradigmatic = Paradigmatic()
        score = paradigmatic.similarity(kwargs['word1'], kwargs['word2'])
        return {'score': score }

    def post(self):
        args = self.post_parser.parse_args()
        data = self.calculate_paradigmatic(word1=args['word1'], word2=args['word2'])
        return {'score': data}

with app.app_context():
    api = FlaskRestfulAPI(app)
    api.add_resource(ParadigmaticResource, '/paradigmatic')

if __name__ == '__main__':
    app.run(debug=True)

Now When I execute this endpoint then I am getting the below-mentioned error.

[1521814433242] [DEBUG] 2018-03-23T14:13:53.226Z 6a8d7647-2ea4-11e8-8f8b-3f7ca6a6d141 Zappa Event: {'resource': '/{proxy+}', 'path': '/paradigmatic', 'httpMethod': 'POST', 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Accept-Language': 'en-GB,en-US;q=0.9,en;q=0.8', 'cache-control': 'no-cache', 'CloudFront-Forwarded-Proto': 'https', 'CloudFront-Is-Desktop-Viewer': 'true', 'CloudFront-Is-Mobile-Viewer': 'false', 'CloudFront-Is-SmartTV-Viewer': 'false', 'CloudFront-Is-Tablet-Viewer': 'false', 'CloudFront-Viewer-Country': 'IN', 'content-type': 'application/json', 'Host': 'voxvx70ba3.execute-api.ap-south-1.amazonaws.com', 'origin': 'chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop', 'postman-token': '0c8ebf3f-22b9-c825-e54a-0325e486171d', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36', 'Via': '2.0 affc54995ded64e4c5647f6363ed884e.cloudfront.net (CloudFront)', 'X-Amz-Cf-Id': 'zt9m6CYbOqvOWZOPE9BQftFnEqD4qDK2WTkE5k3mIJ5gnr19MogfYA==', 'X-Amzn-Trace-Id': 'Root=1-5ab50ba1-a06594aae7a25b8e84eefedc', 'X-Forwarded-For': '103.19.39.2, 54.182.245.48', 'X-Forwarded-Port': '443', 'X-Forwarded-Proto': 'https'}, 'queryStringParameters': None, 'pathParameters': {'proxy': 'paradigmatic'}, 'stageVariables': None, 'requestContext': {'requestTime': '23/Mar/2018:14:13:53 +0000', 'path': '/dev/paradigmatic', 'accountId': '445420586144', 'protocol': 'HTTP/1.1', 'resourceId': 'ne01c4', 'stage': 'dev', 'requestTimeEpoch': 1521814433148, 'requestId': '6a85fbf4-2ea4-11e8-9c0b-3f97e930b286', 'identity': {'cognitoIdentityPoolId': None, 'accountId': None, 'cognitoIdentityId': None, 'caller': None, 'sourceIp': '103.19.39.2', 'accessKey': None, 'cognitoAuthenticationType': None, 'cognitoAuthenticationProvider': None, 'userArn': None, 'userAgent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36', 'user': None}, 'resourcePath': '/{proxy+}', 'httpMethod': 'POST', 'apiId': 'voxvx70ba3'}, 'body': 'ewoJIndvcmQxIjogIm15c3FsIiwKCSJ3b3JkMiI6ICJqcXVlcnkiCn0=', 'isBase64Encoded': True}
[1521814433243] Object of type 'ParadigmaticResource' is not JSON serializable

I am not sure what is missing, I followed as per the documentation.

I want to execute this function asynchronously.

Please do let me know if someone has any idea about it.

Thanks

scoates commented 6 years ago

Have you tried creating a wrapper function that's not in the class? The async function call stuff needs a function name, not a method of a class.

abdulwahid24 commented 6 years ago

Yes, I got it. It should an independent method.

charlax commented 6 years ago

I'm getting this error when trying to call a task from inside a task.

@task(...)
def test():
    inside_task()

@task(...)
def inside_task():
    pass

With the following traceback:

Traceback (most recent call last):
  File "/var/task/handler.py", line 567, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 240, in lambda_handler
    return handler.handler(event, context)
  File "/var/task/handler.py", line 372, in handler
    result = self.run_function(app_function, event, context)
  File "/var/task/handler.py", line 275, in run_function
    result = app_function(event, context) if varargs else app_function()
  File "/var/task/zappa/async.py", line 424, in _run_async
    capture_response=capture_response).send(task_path, args, kwargs)
  File "/var/task/zappa/async.py", line 170, in send
    self._send(message)
  File "/var/task/zappa/async.py", line 178, in _send
    payload = json.dumps(message).encode('utf-8')
  File "/var/lang/lib/python3.6/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/lang/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'LambdaContext' is not JSON serializable
MunsuPark commented 4 years ago

I'm getting this error when trying to call a task from inside a task.

@task(...)
def test():
    inside_task()

@task(...)
def inside_task():
    pass

With the following traceback:

Traceback (most recent call last):
  File "/var/task/handler.py", line 567, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 240, in lambda_handler
    return handler.handler(event, context)
  File "/var/task/handler.py", line 372, in handler
    result = self.run_function(app_function, event, context)
  File "/var/task/handler.py", line 275, in run_function
    result = app_function(event, context) if varargs else app_function()
  File "/var/task/zappa/async.py", line 424, in _run_async
    capture_response=capture_response).send(task_path, args, kwargs)
  File "/var/task/zappa/async.py", line 170, in send
    self._send(message)
  File "/var/task/zappa/async.py", line 178, in _send
    payload = json.dumps(message).encode('utf-8')
  File "/var/lang/lib/python3.6/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/lang/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'LambdaContext' is not JSON serializable

you should not use @taskdecorator to the function that used for events in zappa_settings.json

hafneem commented 2 years ago

So I have been playing around with this today to try and come to a solution, however, I believe I may have stumbled on another issue.

First, my solution:

import json
from zappa.async import task, ASYNC_CLASSES, LambdaAsyncResponse

# Define a custom encoder which casts non base types to string
class LambdaMessageEncoder(json.JSONEncoder):
    def default(self, obj):
        if not type(obj) in [int, float, complex, dict, tuple, list, bool] and obj is not None:
            return str(obj)

        return json.JSONEncoder.default(self, obj)

# Extend LambdaAsyncResponse to rework the _send method
# The send method is only included due to a bug where response_id is not set
class TestLambdaAsyncResponse(LambdaAsyncResponse):
    def send(self, task_path, args, kwargs):
        """
        Create the message object and pass it to the actual sender.
        """
        message = {
                'task_path': task_path,
                'capture_response': getattr(self, 'capture_response', False),
                'response_id': getattr(self, 'response_id', None),
                'args': args,
                'kwargs': kwargs
            }
        self._send(message)
        return self

    def _send(self, message):
        """
        Given a message, directly invoke the lamdba function for this task.
        """
        message['command'] = 'zappa.async.route_lambda_task'

        # Use custom encoder class
        payload = json.dumps(message, cls=LambdaMessageEncoder).encode('utf-8')

        if len(payload) > 128000: # pragma: no cover
            raise AsyncException("Payload too large for async Lambda call")

        self.response = self.client.invoke(
                                    FunctionName=self.lambda_function_name,
                                    InvocationType='Event', #makes the call async
                                    Payload=payload
                                )
        self.sent = (self.response.get('StatusCode', 0) == 202)

# Add service to ASYNC_CLASSES
ASYNC_CLASSES['test'] = TestLambdaAsyncResponse

@task()
def my_func_default_encoder(*args, **kwargs):
    print("All good, default encoder!")

@task(service='test')
def my_func_with_encoder(*args, **kwargs):
  print("All good, new encoder!")

This fixes our issues with LambdaContext being unserializable but raises a new issue as Zappa is trying to call the method asynchronously when we invoke it through the command line. The response object, TestLambdaAsyncResponse, is in turn unserializable. Output below:

Calling invoke for stage bugtest..
[DEBUG] 2017-09-12T11:52:17.97Z d327d303-97b0-11e7-86c5-114d325c08e9 StringToSign:
AWS4-HMAC-SHA256
20170912T115217Z
20170912/eu-west-2/lambda/aws4_request..............
[DEBUG] 2017-09-12T11:52:17.97Z d327d303-97b0-11e7-86c5-114d325c08e9 Signature:
..................
[DEBUG] 2017-09-12T11:52:17.98Z d327d303-97b0-11e7-86c5-114d325c08e9 Sending http request: <PreparedRequest [POST]>
[INFO]    2017-09-12T11:52:17.98Z d327d303-97b0-11e7-86c5-114d325c08e9    Starting new HTTPS connection (2): lambda.eu-west-2.amazonaws.com
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 "POST /2015-03-31/functions/zappabug-bugtest/invocations HTTP/1.1" 2020
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Response headers: {'content-type': '', 'date': 'Tue, 12 Sep 2017 11:52}
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Response body:
<botocore.response.StreamingBody object at 0x7f72a708df98>
[DEBUG] 2017-09-12T11:52:17.211Z d327d303-97b0-11e7-86c5-114d325c08e9 Event needs-retry.lambda.Invoke: calling handler <botocore.retryhandle>
[DEBUG] 2017-09-12T11:52:17.212Z d327d303-97b0-11e7-86c5-114d325c08e9 No retry needed.
Result of test.my_func_with_encoder:
<test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30>
An error occurred during JSON serialization of response: <test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30> is not JSON serializable
Traceback (most recent call last):
  File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/var/runtime/awslambda/bootstrap.py", line 110, in decimal_serializer
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <test.TestLambdaAsyncResponse object at 0x7f72a7cf1d30> is not JSON serializable
[END] RequestId: d327d303-97b0-11e7-86c5-114d325c08e9
[REPORT] RequestId: d327d303-97b0-11e7-86c5-114d325c08e9
Duration: 131.44 ms
Billed Duration: 200 ms 
Memory Size: 512 MB
Max Memory Used: 34 MB

I'm not sure my understanding of the internals of Zappa is quite strong enough to tackle this one, would appreciate some guidance/thoughts on how to tackle this.

I'm facing same issue, any update on this?

monkut commented 1 year ago

As mentioned in the comment, a scheduled event cannot use a @task decorator.

If the function is called in code other than the scheduled event, and you still want it to run as a scheduled event, I believe if you create a simple handler (my_async_task_handler() below) to manage the scheduled event you can work around this issue. (same when calling a task in a task).

...
"events": [
    {
        "function": "tasks.my_async_task_handler",
        "expression": "rate(1 minute)"
    }
],
...

tasks.py:

@task(...)
def my_async_task():
    print("code here")

def my_async_task_handler():
    my_async_task()