php-enqueue / enqueue-dev

Message Queue, Job Queue, Broadcasting, WebSockets packages for PHP, Symfony, Laravel, Magento. DEVELOPMENT REPOSITORY - provided by Forma-Pro
https://enqueue.forma-pro.com/
MIT License
2.18k stars 435 forks source link

Configuration with Amazon SQS and Symfony #619

Closed noniagriconomie closed 6 years ago

noniagriconomie commented 6 years ago

Hi

I plan to migrate from swarrot lib over rabbitmq to enqueue over amazon sqs.

I have found some documentation on the symfony bundle (I use version 3.4 lts) and the sqs library, and trying to configure it properly, but i face some problems:

enqueue:
    transport:
        default: 'sqs'
        sqs:
            key: '%app.amazonsqs.key%'
            secret: '%app.amazonsqs.secret%'
            token: ~
            region: 'eu-west-1'
            retries: 3
            version: '2012-11-05'
            lazy: true
    client: ~
$producer->sendEvent(
            'QUEUE_URL.fifo',
            \json_encode(['id' => 42, 'a' => 'b'])
        );

And i got The specified queue does not exist for this wsdl version

What do i miss?

Many thanks for your time

makasim commented 6 years ago

Do you really need fifo?

makasim commented 6 years ago

fyi https://github.com/php-enqueue/enqueue-dev/pull/175

noniagriconomie commented 6 years ago

@makasim hi, yes for the majority of existing queues, some are not mandatory and we do have exactly one consume for such queues

i have check this PR already, but could not find the correct config

here i understand that the queue need to be in the client config, but i have a lot of queues, means i have to create lot of clients?

i just want to find proper config to get the job done like having the service which publish a message in the correct sqs queue and after for the command to consume this queue

makasim commented 6 years ago

You could use fifo queues with sqs transport (without using Enqueue Client higher abstraction).

Long story short. Client does not support SQS fifi queues. The client uses a router to dispatch messages to their final destinations and it breaks fifo cuz after router messages could go in different order. That's why a PR with such a feature was declined. It won't simply work.

makasim commented 6 years ago

FYI: Enqueue Client works with SQS generic (not fifo) queues out of the box.

noniagriconomie commented 6 years ago

ok so if i understood correctly, we have to create a standard queue on sqs and

what if i use the amazon sdq to publish message in a fifo queue, and use your bundle to consume (with a processor and the consume command) will it works properly ?

makasim commented 6 years ago

You can use SQS fifo queues if you use a transport directly. Here's an article that covers the setup process. It was written for RabbitMQ but should work for SQS too with some fairly simple changes.

Enqueue client could be used with standard queues. Here's how you can configure it https://github.com/php-enqueue/enqueue-dev/blob/0.8/docs/bundle/quick_tour.md