uecode / qpush-bundle

Asynchronous processing for Symfony using Push Queues
qpush-bundle.readthedocs.org
Apache License 2.0
168 stars 54 forks source link

When queue is aws fifo (name needs to end with .fifo) and push notification is true, sns topic cant be created #131

Open 2Taps opened 7 years ago

2Taps commented 7 years ago

The error ive got is: InvalidParameter

Invalid parameter: Topic Name

I have debugged and saw that the method getNameWithPrefix of AbstractProvider.php should have a way to preserve the ".fifo" extension to be used in the sqs queue create method, QueueName parameter

Did quick adjustment just make it pass, maybe you will find more elegant solution. public function getNameWithPrefix($removeFifoExtension=true) { if (!empty($this->options['queue_name'])) { if($removeFifoExtension) { return str_replace('.fifo', '', $this->options['queue_name']); } else { return $this->options['queuename']; } } return sprintf("%s%s", self::QPUSH_PREFIX, $this->name); }

Then in AwsProvider.php file in line 405 add the false parameter preserve fifo extension in aws sqs queue name creation $result = $this->sqs->createQueue(['QueueName' => $this->getNameWithPrefix(false), 'Attributes' => $attributes]);

Hope you can incorporate this to the project soon, thank you the bundle looks very good!

2Taps commented 7 years ago

Hey Guys,

Also on this matter, even applying the fix above i could not receive the message on my service using the fifo queue. After a lot of debug i guess i found the problem.

In the AwsProvider.php publish method there is a return $result->get('MessageId'); on line 232 that is stoping the method when push_notifications is true. Because of this return, the $this->sqs->sendMessage on line 256 never get called and the message is never added to the sqs queue. So when sns calls my endpoint and try to fetch the message from sqs, there is no message there.

I have commented out the line 232 and was able to receive the message on my endpoint. Please confirm to me that i have made the right adjustments if you can.

Thank you!

danmurf commented 7 years ago

Hi @GuilhermeMoura1 ,

Unfortunately I believe this is a limitation of SQS, not this bundle:

http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-subscribe-queue-sns-topic.html

Amazon SNS isn't currently compatible with FIFO queues.

2Taps commented 7 years ago

Hi @danmurf,

Guess i am a bit confused now.


So are you telling me that this return in the $this->sqs->sendMessage on line 256 is intentional. And when push notification is true we only publish the message to SNS, then SNS forward the message to SQS and SNS start to send notification to our endpoint defined in the bundle configuration.

Then our endpoint receive the notification and from what i could understand in the code, it calls SQS to retrieve messages?

And as amazon says that fifo queues can not receive the message from automatically SNS we cant use push_notification alongside fifo queues in this bundle right?

danmurf commented 7 years ago

Hi GuilhermeMoura1, that's my understanding (that you can't use push_notifications alongside fifo queues).

I've created a PR to update the docs so that others are aware of this: https://github.com/uecode/qpush-bundle/pull/134