brefphp / symfony-messenger

Bridge to use Symfony Messenger on AWS Lambda with Bref
MIT License
72 stars 22 forks source link

Invoke error #46

Closed JKetelaar closed 3 years ago

JKetelaar commented 3 years ago

With a setup using a function to use SNS, I am getting the following error:

{
    "errorType": "Symfony\\Component\\Messenger\\Exception\\MessageDecodingFailedException",
    "errorMessage": "Could not decode message using PHP serialization: {\"notificationType\":\"Bounce\",\"bounce\":{\"feedbackId\":\"01070177b7c56aa1-117088ce-e70c-42c6-9120-42aacdde7907-000000\",\"bounceType\":\"Permanent\",\"bounceSubType\":\"General\",\"bouncedRecipients\":[{\"emailAddress\":\"bounce@simulator.amazonses.com\",\"action\":\"failed\",\"status\":\"5.1.1\",\"diagnosticCode\":\"smtp; 550 5.1.1 user unknown\"}],\"timestamp\":\"2021-02-19T00:51:44.000Z\",\"remoteMtaIp\":\"3.226.40.239\",\"reportingMTA\":\"dsn; b224-13.smtp-out.eu-central-1.amazonses.com\"},\"mail\":{\"timestamp\":\"2021-02-19T00:51:42.822Z\",\"source\":\"jketelaar@intercube.io\",\"sourceArn\":\"arn:aws:ses:eu-central-1:544387708820:identity/jketelaar@intercube.io\",\"sourceIp\":\"178.132.211.234\",\"sendingAccountId\":\"544387708820\",\"messageId\":\"01070177b7c56466-279ebebc-6da2-4a1c-af23-b78eac23a6da-000000\",\"destination\":[\"bounce@simulator.amazonses.com\"]}}.",
    "stack": [
        "#0 /var/task/vendor/symfony/messenger/Transport/Serialization/PhpSerializer.php(38): Symfony\\Component\\Messenger\\Transport\\Serialization\\PhpSerializer->safelyUnserialize()",
        "#1 /var/task/vendor/bref/symfony-messenger/src/Service/Sns/SnsConsumer.php(40): Symfony\\Component\\Messenger\\Transport\\Serialization\\PhpSerializer->decode()",
        "#2 /var/task/vendor/bref/bref/src/Event/Sns/SnsHandler.php(18): Bref\\Symfony\\Messenger\\Service\\Sns\\SnsConsumer->handleSns()",
        "#3 /var/task/vendor/bref/bref/src/Runtime/Invoker.php(29): Bref\\Event\\Sns\\SnsHandler->handle()",
        "#4 /var/task/vendor/bref/bref/src/Runtime/LambdaRuntime.php(102): Bref\\Runtime\\Invoker->invoke()",
        "#5 /opt/bref/bootstrap.php(43): Bref\\Runtime\\LambdaRuntime->processNextEvent()",
        "#6 {main}"
    ]
}

Any clue of what I'm doing wrong?

Here's some more information about the application:

config/packages/messenger.yaml:

framework:
    messenger:
        transports:
            async: '%env(MESSENGER_TRANSPORT_DSN)%'
        routing:
            'App\Message\SesMessage': async

src/Message/SesMessage.php

<?php
namespace App\Message;

class SesMessage
{
    private $content;

    public function __construct(string $content)
    {
        $this->content = $content;
    }

    public function getContent(): string
    {
        return $this->content;
    }
}

src/MessageHandler/SesNotificationHandler.php

<?php

namespace App\MessageHandler;

use App\Message\SesMessage;
use Symfony\Component\Messenger\Handler\MessageHandlerInterface;

class SesNotificationHandler implements MessageHandlerInterface
{
    public function __invoke(SesMessage $message)
    {

    }
}

bin/consumer.php

<?php declare(strict_types=1);

use App\Kernel;
use Bref\Symfony\Messenger\Service\Sns\SnsConsumer;
use Symfony\Component\Dotenv\Dotenv;

require dirname(__DIR__) . '/vendor/autoload.php';

(new Dotenv())->bootEnv(dirname(__DIR__) . '/.env');

$kernel = new Kernel($_SERVER['APP_ENV'], (bool)$_SERVER['APP_DEBUG']);
$kernel->boot();

// Return the Bref consumer service
return $kernel->getContainer()->get(SnsConsumer::class);

serverless.yml

...

functions:
  worker:
    handler: bin/consumer.php
    timeout: 20
    reservedConcurrency: 5 
    layers:
      - ${bref:layer.php-74}
    events:
      - sns:
          arn: arn:aws:sns:eu-central-1:1234567890:Test
mnapoli commented 3 years ago

Could not decode message using PHP serialization

It seems like Symfony is trying to use unserialize() (the PHP serialization format) over the message content, but the message content is JSON.

Maybe it's a matter of configuring Symfony Messenger so that it uses JSON as a format?

JKetelaar commented 3 years ago

Thanks so much, you put me in the right direction! Since Symfony 4.3 Symfony has moved from its serializer (JSON) to the PhpSerializer.

Symfony 4.3 & 4.4:

framework:
    messenger:
        serializer:
            id: 'messenger.transport.symfony_serializer'

Though this doesn't work yet on Symfony 5, there you need to make the change as follows:

framework:
    messenger:
        serializer:
            default_serializer: messenger.transport.symfony_serializer
            symfony_serializer:
                format: json
                context: { }

Should I make a PR with this and add it to the readme?

JKetelaar commented 3 years ago

After this, a lot more errors occur that have to do with the old integration of this module and the latest Symfony. In the meantime, I've also written my own SnsConsumer and created my own Serializer. This works (sketchy) as of now, though am I doing something completely wrong, or is the integration with Symfony 5.x not fully tested yet?

mnapoli commented 3 years ago

I'm confused: is your Symfony Lambda worker handling events that were created by the Symfony application?

The principle is like this:

So the format of the message is defined by Symfony: if you publish it as JSON, your app will receive it as JSON. If you publish it as serialized, then your app will receive it as serialized.

Are you using another architecture?

JKetelaar commented 3 years ago

Ah... That would explain a lot... My bad, sorry.

So the approach I'm taking and using this bridge/bundle for is to receive SES messages in SNS and process them with a Lambda function. So in my case Symfony isn't creating a message in SNS, but SES is (or anything else I want to hook SES on).

Would you recommend another module/approach for this? Or is this uncommon and does it indeed require custom built software?

mnapoli commented 3 years ago

In that case I would recommend using a typed handler: https://bref.sh/docs/function/handlers.html#sns-events

Note that this typed handler can be a Symfony service, you can use the same trick to load a symfony service:

<?php declare(strict_types=1);

use Bref\Symfony\Messenger\Service\Sqs\SqsConsumer;

require dirname(__DIR__) . '/config/bootstrap.php';

$kernel = new \App\Kernel($_SERVER['APP_ENV'], (bool) $_SERVER['APP_DEBUG']);
$kernel->boot();

return $kernel->getContainer()->get(MySnsHandlerService::class);
JKetelaar commented 3 years ago

This explains a lot, thanks so much!

I'll probably write a blog post about it or something, in case others ever have the same situation.

mnapoli commented 3 years ago

Please do!