Gae influencing on beanstalkd queue?

[rulrok]

Hello,

I'm trying to setup a Laravel 5.0 project on GAE that uses the push queue for all the normal actions which you could do, and one beanstalkd queue that is running in a VM compute engine.

I need to use this beanstalkd queue for only one very specific task that MUST be run on the VM.

So, inside my Command class I have this method to push it to my desired queue.

class ProcessSceneImage extends Command implements SelfHandling, ShouldBeQueued {

    use InteractsWithQueue, SerializesModels;

... (other content)

  public function queue($queue, $command) {

        $queue->pushOn('images_queue', $command);

    }

... (other content)

}

In my controller, I just dispatch the process like that

        $this->dispatch(
            new ProcessSceneImage(
                $scene_id
                , Request::get('type')
            )
        );

The problem is that the exception that I'm getting suggests that this process was tried to be queued into GAE queue instead of beanstalkd. Below, the stack trace (most recent at the top). The lines 05~07 make me believe that Gae queue is getting in the way somehow.

TaskQueueException in PushQueue.php line 90:
Invalid queue name
01 in PushQueue.php line 90
02 at PushQueue::errorCodeToException('6') in PushQueue.php line 189
03 at PushQueue->addTasks(array(object(PushTask))) in PushTask.php line 292
04 at PushTask->add('images_queue') in GaeQueue.php line 85
05 at GaeQueue->pushRaw('{"job":"Illuminate\\Queue\\CallQueuedHandler@call","data":{"command":"O:30:\"App\\Commands\\ProcessSceneImage\":3:{s:42:\"\u0000App\\Commands\\ProcessSceneImage\u0000i_scene_id\";i:13;s:38:\"\u0000App\\Commands\\ProcessSceneImage\u0000s_type\";s:4:\"flat\";s:6:\"\u0000*\u0000job\";N;}"},"attempts":1,"queue":"images_queue"}', 'images_queue') in GaeQueue.php line 67
06 at GaeQueue->push(object(ProcessSceneImage), '', 'images_queue') in Queue.php line 30
07 at Queue->pushOn('images_queue', object(ProcessSceneImage)) in ProcessSceneImage.php line 59
08 at ProcessSceneImage->queue(object(GaeQueue), object(ProcessSceneImage)) in Dispatcher.php line 256
09 at Dispatcher->dispatchToQueue(object(ProcessSceneImage)) in Dispatcher.php line 181
10 at Dispatcher->dispatch(object(ProcessSceneImage)) in DispatchesCommands.php line 15
11 at WebController->dispatch(object(ProcessSceneImage)) in SceneController.php line 112
12 at SceneController->store('2')
13 at call_user_func_array(array(object(SceneController), 'store'), array('pano' => '2')) in Controller.php line 246
14 at Controller->callAction('store', array('pano' => '2')) in ControllerDispatcher.php line 162
15 at ControllerDispatcher->call(object(SceneController), object(Route), 'store') in ControllerDispatcher.php line 107
16 at ControllerDispatcher->Illuminate\Routing\{closure}(object(Request))
17 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 141
18 at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
19 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 101
20 at Pipeline->then(object(Closure)) in ControllerDispatcher.php line 108
21 at ControllerDispatcher->callWithinStack(object(SceneController), object(Route), object(Request), 'store') in ControllerDispatcher.php line 67
22 at ControllerDispatcher->dispatch(object(Route), object(Request), 'App\Http\Controllers\Web\SceneController', 'store') in Route.php line 204
23 at Route->runWithCustomDispatcher(object(Request)) in Route.php line 134
24 at Route->run(object(Request)) in Router.php line 701
25 at Router->Illuminate\Routing\{closure}(object(Request))
26 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 141
27 at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
28 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 101
29 at Pipeline->then(object(Closure)) in Router.php line 703
30 at Router->runRouteWithinStack(object(Route), object(Request)) in Router.php line 670
31 at Router->dispatchToRoute(object(Request)) in Router.php line 628
32 at Router->dispatch(object(Request)) in Kernel.php line 214
33 at Kernel->Illuminate\Foundation\Http\{closure}(object(Request))
34 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 141
35 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in VerifyCsrfToken.php line 43
36 at VerifyCsrfToken->handle(object(Request), object(Closure)) in VerifyCsrfToken.php line 20
37 at VerifyCsrfToken->handle(object(Request), object(Closure)) in Pipeline.php line 125
38 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in ShareErrorsFromSession.php line 55
39 at ShareErrorsFromSession->handle(object(Request), object(Closure)) in Pipeline.php line 125
40 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in StartSession.php line 61
41 at StartSession->handle(object(Request), object(Closure)) in Pipeline.php line 125
42 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in AddQueuedCookiesToResponse.php line 36
43 at AddQueuedCookiesToResponse->handle(object(Request), object(Closure)) in Pipeline.php line 125
44 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in EncryptCookies.php line 40
45 at EncryptCookies->handle(object(Request), object(Closure)) in Pipeline.php line 125
46 at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in CheckForMaintenanceMode.php line 42
47 at CheckForMaintenanceMode->handle(object(Request), object(Closure)) in Pipeline.php line 125
48 at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
49 at call_user_func(object(Closure), object(Request)) in Pipeline.php line 101
50 at Pipeline->then(object(Closure)) in Kernel.php line 115
51 at Kernel->sendRequestThroughRouter(object(Request)) in Kernel.php line 84
52 at Kernel->handle(object(Request)) in index.php line 53

My .env file looks like this (for debugging)

APP_ENV=production
APP_DEBUG=true
APP_KEY=random_string
BEANSTALKD_HOST=xxx.xxx.xx.xxx (my VM IPv4 IP)
DB_CONNECTION=cloudsql
CLOUD_SQL_SOCKET=/cloudsql/<project-id>
CLOUD_SQL_DATABASE=<database_name>
CLOUD_SQL_USERNAME=root
CLOUD_SQL_PASSWORD=
CACHE_DRIVER=memcached
SESSION_DRIVER=memcached
QUEUE_DRIVER=gae
MAIL_DRIVER=gae
MAIL_HOST=mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=
MAIL_PASSWORD=
LOG_HANDLER=syslog
FILESYSTEM=gae
CACHE_SERVICES_FILE=false
CACHE_CONFIG_FILE=false
CACHE_ROUTES_FILE=false
CACHE_COMPILED_VIEWS=false

My queue config file looks like this

    'default' => env('QUEUE_DRIVER', 'sync'),

    'connections' => [

        'gae' => [
            'driver'    => 'gae',
            'queue'     => 'default',
            'url'       => '/tasks',
            'encrypt'   => true,
        ],

        'sync' => [
            'driver' => 'sync',
        ],

        'beanstalkd' => [
            'driver' => 'beanstalkd',
            'host'   => env('BEANSTALKD_HOST'),
            'queue'  => 'default',
            'ttr'    => 60,
        ],

        'images_queue' => [
            'driver' => 'beanstalkd',
            'host'   => env('BEANSTALKD_HOST'),
            'queue'  => 'images_queue',
            'ttr'    => 60,
        ],

    ],

    'failed' => [
        'database' => 'mysql', 'table' => 'failed_jobs',
    ],

I'm getting out of hope at this point. Could someone give me a hand here?

[shpasser]

First of all, thanks for a very thorough description. I think that the problem is that you are trying to push the job to images_queue queue using the default connection(GAE). In order to use a different connection('images_queue') I would suggest to try the following code:

Queue::connection('images_queue')->pushOn('images_queue', $command);

Please notice that both the connection and the queue are sharing the same name(according to the configuration file). I would suggest changing the connection name to something like images_connection. Please let me know if this helps.

[rulrok]

That was it!

This is what was making me confused too. The gae connection has the same name as the driver. So I though that by saying which queue you wanted, you were actually saying which connection to use. Moreover, I was already in a scavenger hunt into Laravel documentation and they've never told about that Queue::connection method. Now I have changed the names as you suggested.

Gosh. This is so frustrating! :disappointed:

Thank you a lot. You saved me here.