How do you know if a message was successfully added to a queue?

[phuze]

Which service(blob, file, queue, table) does this issue concern?

queue

Which version of the SDK was used?

dev-master

What's the PHP/OS version?

7.2.9

I encountered a situation where a message was not written to queue, yet no errors were triggered. It just simply wasn't created. This leaves me with a question --- how do I REALLY know if the message was successfully added?

If you refer to my function addToQueue(), you'll note its inside a try block, and assuming all goes well, simply returns true (successful). However, there's no actual check in place (I didn't think I needed one until today). Noting that there was no exception thrown/caught.

use MicrosoftAzure\Storage\Common\Middlewares\RetryMiddlewareFactory;
use MicrosoftAzure\Storage\Common\Exceptions\ServiceException;
use MicrosoftAzure\Storage\Queue\QueueRestProxy;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Monolog\Formatter\LineFormatter;

class Azure {

    /**
     * Logging
     */
    protected static function logger() {
        $logger = new Logger('Azure');
        $handler = new StreamHandler('/var/log/daemons/CLASS.log');
        $handler->setFormatter(new LineFormatter("[azure] %datetime%: %message% %context%\n", null, false, true));
        $logger->pushHandler($handler);
        return $logger;
    }

    /**
     * Create a retry factory. Example:
     * 
     *      RetryMiddlewareFactory::create(
     *          $type = self::GENERAL_RETRY_TYPE,
     *          $numberOfRetries = Resources::DEFAULT_NUMBER_OF_RETRIES,
     *          $interval = Resources::DEFAULT_RETRY_INTERVAL,
     *          $accumulationMethod = self::LINEAR_INTERVAL_ACCUMULATION,
     *          $retryConnect = false
     *      )
     * 
     * Retry factory is configured to retry 3 times, every 3 seconds
     * $retryConnect has been set to true to enable retries on
     * connection failures -- by default it is false.
     */
    protected static function retryFactory() {
        $retryMiddleware = RetryMiddlewareFactory::create(
            RetryMiddlewareFactory::GENERAL_RETRY_TYPE,
            3,
            3,
            RetryMiddlewareFactory::LINEAR_INTERVAL_ACCUMULATION,
            true
        );
        return $retryMiddleware;
    }

    /**
     * Add message to a message queue
     * @param string $queueName - the queue name
     * @param object $json - the queue data
     * @return boolean true
     */
    public static function addToQueue($queueName, $json)
    {
        try {
            # create queue service
            $queueClient = QueueRestProxy::createQueueService(AZURE_QUEUE);
            # add the retry middleware
            $queueClient->pushMiddleware(self::retryFactory());
            # create message
            $queueClient->createMessage($queueName, $json);
            # return true if successful
            return true;
        } catch (Exception $e) {
            # catch both Azure and Guzzle exceptions
            if ($e instanceof ServiceException OR $e instanceof RequestException) {
                self::logger()->error("[Azure::addToQueue][$queueName][".$e->getCode()."] ".$e->getMessage());
            }
        }        
    }

}

[XiaoningLiu]

Hi @purplekrayons

This should never happen when HTTP request is successful but no queue message is added. If so, there should be some bugs. To help address the root cause, here are 2 steps you can do:

1> $queueClient->createMessage will return CreateMessageResult object including the inserted message properties. Please check the result is empty or not by calling CreateMessageResult::getQueueMessage(). https://github.com/Azure/azure-storage-php/blob/master/azure-storage-queue/src/Queue/Models/CreateMessageResult.php

2> Enable SDK logging by adding a HistoryMiddleware, find out and send us the request and response of the buggy createMessage request.

https://github.com/Azure/azure-storage-php/blob/master/azure-storage-common/src/Common/Middlewares/HistoryMiddleware.php

[phuze]

Closing this issue. Unfortunately I did not have time to explore the avenues you recommended with regards to implementing the HistoryMiddleware and I have never been able to determine why queue messages randomly don't get created yet no errors were thrown.

Since the project was time-sensitive, I opted to address this by reducing the rate at which the application was creating messages. My hunch is that we were simply overloading the queue service.