laravel-doctrine / orm

An integration library for Laravel and Doctrine ORM
MIT License
828 stars 178 forks source link

[BUG] [QUESTION] Laravel Scheduler not able to find database connection #505

Closed ukenpachi closed 3 years ago

ukenpachi commented 3 years ago

Please prefix your issue with one of the following: [BUG] [PROPOSAL] [QUESTION].

class Kernel extends ConsoleKernel
{
    /**
     * The Artisan commands provided by your application.
     *
     * @var array
     */
    protected $commands = [
        'App\Console\Commands\Start',
    ];

    /**
     * Define the application's command schedule.
     *
     * @param  \Illuminate\Console\Scheduling\Schedule  $schedule
     * @return void
     */
    protected function schedule(Schedule $schedule)
    {
        $schedule->command('send:sale-emails')->everyMinute()->runInBackground();
    }
}
[2021-11-11 17:14:40] local.ERROR: Connection [DATABASECONNECTION] has no configuration in [database.connections.DATABASECONNECTION]
{"exception":"[object] (InvalidArgumentException(code: 0): Connection [DATABASECONNECTION] has no configuration in
[database.connections.DATABASECONNECTION] at /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/EntityManagerFactory.php:470)
[stacktrace]
#0 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/EntityManagerFactory.php(112): LaravelDoctrine\\ORM\\EntityManagerFactory->getConnectionDriver()
#1 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/IlluminateRegistry.php(81): LaravelDoctrine\\ORM\\EntityManagerFactory->create()
#2 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(873): LaravelDoctrine\\ORM\\IlluminateRegistry->LaravelDoctrine\\ORM\\{closure}()
#3 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(758): Illuminate\\Container\\Container->build()
#4 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(694): Illuminate\\Container\\Container->resolve()
#5 /home/vagrant/code/email-api/vendor/laravel/lumen-framework/src/Application.php(300): Illuminate\\Container\\Container->make()
#6 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/IlluminateRegistry.php(396): Laravel\\Lumen\\Application->make()
#7 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/IlluminateRegistry.php(209): LaravelDoctrine\\ORM\\IlluminateRegistry->getService()
#8 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/IlluminateRegistry.php(242): LaravelDoctrine\\ORM\\IlluminateRegistry->getManager()
#9 /home/vagrant/code/email-api/vendor/laravel-doctrine/extensions/src/GedmoExtensionsServiceProvider.php(23): LaravelDoctrine\\ORM\\IlluminateRegistry->getManagers()
#10 /home/vagrant/code/email-api/vendor/illuminate/events/Dispatcher.php(404): LaravelDoctrine\\Extensions\\GedmoExtensionsServiceProvider->LaravelDoctrine\\Extensions\\{closure}()
#11 /home/vagrant/code/email-api/vendor/illuminate/events/Dispatcher.php(249): Illuminate\\Events\\Dispatcher->Illuminate\\Events\\{closure}()
#12 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/DoctrineServiceProvider.php(288): Illuminate\\Events\\Dispatcher->dispatch()
#13 /home/vagrant/code/email-api/vendor/laravel-doctrine/orm/src/DoctrineServiceProvider.php(158): LaravelDoctrine\\ORM\\DoctrineServiceProvider->bootExtensionManager()
#14 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(1269): LaravelDoctrine\\ORM\\DoctrineServiceProvider->LaravelDoctrine\\ORM\\{closure}()
#15 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(1234): Illuminate\\Container\\Container->fireCallbackArray()
#16 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(1219): Illuminate\\Container\\Container->fireAfterResolvingCallbacks()
#17 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(778): Illuminate\\Container\\Container->fireResolvingCallbacks()
#18 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(694): Illuminate\\Container\\Container->resolve()
#19 /home/vagrant/code/email-api/vendor/laravel/lumen-framework/src/Application.php(300): Illuminate\\Container\\Container->make()
#20 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(1031): Laravel\\Lumen\\Application->make()
#21 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(951): Illuminate\\Container\\Container->resolveClass()
#22 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(912): Illuminate\\Container\\Container->resolveDependencies()
#23 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(758): Illuminate\\Container\\Container->build()
#24 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(694): Illuminate\\Container\\Container->resolve()
#25 /home/vagrant/code/email-api/vendor/laravel/lumen-framework/src/Application.php(300): Illuminate\\Container\\Container->make()
#26 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(1031): Laravel\\Lumen\\Application->make()
#27 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(951): Illuminate\\Container\\Container->resolveClass()
#28 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(912): Illuminate\\Container\\Container->resolveDependencies()
#29 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(758): Illuminate\\Container\\Container->build()
#30 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(694): Illuminate\\Container\\Container->resolve()
#31 /home/vagrant/code/email-api/vendor/laravel/lumen-framework/src/Application.php(300): Illuminate\\Container\\Container->make()
#32 /home/vagrant/code/email-api/vendor/illuminate/container/BoundMethod.php(175): Laravel\\Lumen\\Application->make()
#33 /home/vagrant/code/email-api/vendor/illuminate/container/BoundMethod.php(124): Illuminate\\Container\\BoundMethod::addDependencyForCallParameter()
#34 /home/vagrant/code/email-api/vendor/illuminate/container/BoundMethod.php(36): Illuminate\\Container\\BoundMethod::getMethodDependencies()
#35 /home/vagrant/code/email-api/vendor/illuminate/container/Util.php(40): Illuminate\\Container\\BoundMethod::Illuminate\\Container\\{closure}()
#36 /home/vagrant/code/email-api/vendor/illuminate/container/BoundMethod.php(93): Illuminate\\Container\\Util::unwrapIfClosure()
#37 /home/vagrant/code/email-api/vendor/illuminate/container/BoundMethod.php(37): Illuminate\\Container\\BoundMethod::callBoundMethod()
#38 /home/vagrant/code/email-api/vendor/illuminate/container/Container.php(653): Illuminate\\Container\\BoundMethod::call()
#39 /home/vagrant/code/email-api/vendor/illuminate/console/Command.php(136): Illuminate\\Container\\Container->call()
#40 /home/vagrant/code/email-api/vendor/symfony/console/Command/Command.php(299): Illuminate\\Console\\Command->execute()
#41 /home/vagrant/code/email-api/vendor/illuminate/console/Command.php(121): Symfony\\Component\\Console\\Command\\Command->run()
#42 /home/vagrant/code/email-api/vendor/symfony/console/Application.php(978): Illuminate\\Console\\Command->run()
#43 /home/vagrant/code/email-api/vendor/symfony/console/Application.php(295): Symfony\\Component\\Console\\Application->doRunCommand()
#44 /home/vagrant/code/email-api/vendor/symfony/console/Application.php(167): Symfony\\Component\\Console\\Application->doRun()
#45 /home/vagrant/code/email-api/vendor/illuminate/console/Application.php(94): Symfony\\Component\\Console\\Application->run()
#46 /home/vagrant/code/email-api/vendor/laravel/lumen-framework/src/Console/Kernel.php(116): Illuminate\\Console\\Application->run()
#47 /home/vagrant/code/email-api/artisan(35): Laravel\\Lumen\\Console\\Kernel->handle()
#48 {main}
"}

this does not work also with supervisord

eigan commented 3 years ago

I think we need some more info because the error message is pretty clear about what the issue is. Since this doesn't help you then we need to know why it doesn't.

ukenpachi commented 3 years ago

I think we need some more info because the error message is pretty clear about what the issue is. Since this doesn't help you then we need to know why it doesn't.

Hi, I am just creating a job which runs using the laravel schedule:run same way I have done in the last 8 years from laravel 4. Except I am using laravel doctrine. That error message only happens if artisan if calling any method which communicate with the DB. In My case the send:sale-emails method calls a method which uses doctrine repository to run a select query one of the 3 databases and should use mailchimp to sent the emails.

If I run the method using postman it works fine, no errors but for some reason doctrine is unable to find the database connection. If I var dump the array in config/Doctrine.php I can see all connections.

I am not sure what specific information you are after, can you let me know. Thank you

dpslwk commented 3 years ago

can you access the db from the same terminal / context with which you are calling php artisan ...

issue here is php-cli can not access the db using the same host/port/user details and wha the php web server can classic example is trying to use php artisan out side a homestead dev box

eigan commented 3 years ago

This specific stack trace does not complain about config/doctrine.php but config/database.php.

If it works in HTTP and not CLI, then I would also make sure that you are loading the correct cached configuration. It can be different depending on which environment you are using when calling artisan (as @dpslwk pointed out). Calling artisan config:clear (or config:cache to re-cache).

ukenpachi commented 3 years ago

can you access the db from the same terminal / context with which you are calling php artisan ...

issue here is php-cli can not access the db using the same host/port/user details and wha the php web server can classic example is trying to use php artisan out side a homestead dev box

I do not think it is the php-cli because php artisan migrate and other commands we have set up which uses pure PDO works. Just fine. I think the might be something in the laravel configuration doctrine.

 "require": {
        "php": "^7.3|^8.0",
        "ext-json": "*",
        "box/spout": "^3.3",
        "gedmo/doctrine-extensions": "^3.1",
        "laravel-doctrine/extensions": "^1.4",
        "laravel-doctrine/orm": "^1.7",
        "laravel/lumen-framework": "8.2.3",
        "phpmailer/phpmailer": "^6.5",
        "rbdwllr/reallysimplejwt": "^2.1",
        "sentry/sentry-laravel": "2.7.0",
        "vlucas/phpdotenv": "^5.3"
    },
ukenpachi commented 3 years ago

This specific stack trace does not complain about config/doctrine.php but config/database.php.

If it works in HTTP and not CLI, then I would also make sure that you are loading the correct cached configuration. It can be different depending on which environment you are using when calling artisan (as @dpslwk pointed out). Calling artisan config:clear (or config:cache to re-cache).

has I am using lumen 8, config:cache andconfig:clear do not exist with lumen

dpslwk commented 3 years ago

having tried a quick test install locally I have not issue, which make me believe this is very much a config or setup issue not a code bug

check your code for DATABASECONNECTION (especially .env, config/doctrine.php and config/database.php)

try dumping\loggin config('database.connections') with both a route\view and say an artisan common or even a scheduled job

ukenpachi commented 3 years ago

config('database.connections')

found the issue and it is definitely a bug

https://github.com/laravel-doctrine/orm/blob/1.7/src/EntityManagerFactory.php#L469

Has I have a multiple database set up connecting to 9 databases. So my config/database.php looks like this

<?php
// Default database configuration
$compiledDatabases = [
    'default' => 'mysql',
    'migrations' => 'migrations',
    'connections' => [
        'mysql' => [
            'driver' => env('DB_CONNECTION'),
            'host' => env('DB_HOST'),
            'port' => env('DB_PORT'),
            'database' => env('DB_DATABASE'),
            'username' => env('DB_USERNAME'),
            'password' => env('DB_PASSWORD'),
            'charset' => 'utf8',
            'collation' => 'utf8_unicode_ci',
            'prefix' => '',
            'strict' => false,
        ],
    ]
];

// Determine the names of any custom databases
$envs = $_ENV;
$databaseNameList = [];
foreach ($envs as $envName => $envValue) {
    if (
        substr($envName, 0, 3) == 'DB_'
        && !in_array($envName, ['DB_CONNECTION', 'DB_HOST', 'DB_PORT', 'DB_DATABASE', 'DB_USERNAME', 'DB_PASSWORD'])
    ) {
        if (preg_match("/^DB_(.+?)_.+?$/", $envName, $match)) {
            if (!in_array(strtolower($match[1]), $databaseNameList)) {
                $databaseNameList[] = strtolower($match[1]);
            }
        }
    }
}

// Assign the custom database data to the databases array used in lumen
foreach ($databaseNameList as $databaseName) {
    $compiledDatabases['connections'][$databaseName]['driver'] = env('DB_' . strtoupper($databaseName) . '_CONNECTION');
    $compiledDatabases['connections'][$databaseName]['host'] = env('DB_' . strtoupper($databaseName) . '_HOST');
    $compiledDatabases['connections'][$databaseName]['port'] = env('DB_' . strtoupper($databaseName) . '_PORT');
    $compiledDatabases['connections'][$databaseName]['database'] = env('DB_' . strtoupper($databaseName) . '_DATABASE');
    $compiledDatabases['connections'][$databaseName]['username'] = env('DB_' . strtoupper($databaseName) . '_USERNAME');
    $compiledDatabases['connections'][$databaseName]['password'] = env('DB_' . strtoupper($databaseName) . '_PASSWORD');
    $compiledDatabases['connections'][$databaseName]['charset'] = 'utf8';
    $compiledDatabases['connections'][$databaseName]['collation'] = env('DB_' . strtoupper($databaseName) . '_COLLATION',
        'utf8_unicode_ci');
    $compiledDatabases['connections'][$databaseName]['prefix'] = '';
    $compiledDatabases['connections'][$databaseName]['strict'] = false;
}

return $compiledDatabases;

EntityManagerFactory.php I added var_dump($this->config->get('database')); and I got only

array(3) {
  ["default"]=>
  string(5) "mysql"
  ["migrations"]=>
  string(10) "migrations"
  ["connections"]=>
  array(1) {
    ["mysql"]=>
    array(10) {
      ["driver"]=>
      string(5) "mysql"
      ["host"]=>
      string(13) "192.168.10.10"
      ["port"]=>
      string(4) "3306"
      ["database"]=>
      string(9) "lumen-api"
      ["username"]=>
      string(9) "homestead"
      ["password"]=>
      string(6) "secret"
      ["charset"]=>
      string(4) "utf8"
      ["collation"]=>
      string(15) "utf8_unicode_ci"
      ["prefix"]=>
      string(0) ""
      ["strict"]=>
      bool(false)
    }
  }
}

This should be a total of 7 connections 5 of which are external connections(outside homestead). Looks like the database.php does not get executed at all with you run kernal runs the commands for the sales compare to if you are running the application via http(see image)

image

dpslwk commented 3 years ago

yeah this is not a bug with laravel-doctrine but an issue with lumen or your complex config\database.php

I suspect $envs = $_ENV; is at the heart of the issue and how lumen might be handling that different between http and console

EntityManagerFactory is just being passed the config from the framework

ukenpachi commented 3 years ago

yeah this is not a bug with laravel-doctrine but an issue with lumen or your complex config\database.php

I suspect $envs = $_ENV; is at the heart of the issue and how lumen might be handling that different between http and console

EntityManagerFactory is just being passed the config from the framework

Might be, it works now if the database.php like https://github.com/laravel/laravel/blob/8.x/config/database.php#L36 it works fine. Any suggestion how to fix it?

eigan commented 3 years ago
  1. Where do you declare your DB_ env variables?
  2. Does calling config:cache then testing via http break the http method?
  3. Is the connections present in config('database') at any other non-related laravel-doctrine contexts (cli commands/tinker)?

In laravel, there is a file called Illuminate\Foundation\Bootstrap\LoadEnvironmentVariables. If you have something similar in Lumen could you put a breakpoint here and see if it properly loads the environment before laravel-doctrine uses it.

ukenpachi commented 3 years ago
  1. Where do you declare your DB_ env variables?
  2. Does calling config:cache then testing via http break the http method?
  3. Is the connections present in config('database') at any other non-related laravel-doctrine contexts (cli commands/tinker)?

In laravel, there is a file called Illuminate\Foundation\Bootstrap\LoadEnvironmentVariables. If you have something similar in Lumen could you put a breakpoint here and see if it properly loads the environment before laravel-doctrine uses it.

(1) .env file (2) all http requests works has expected, the issue is when running php artisan command which uses any of the laravel doctrine repository entity. I know this because if I run php artisan migrate this command works without any issue, unfortunately the custom command we have created uses Laravel doctrine and all of it breaks (3) yes, in migratecommand class I added var_dump(config('database')); ran php artisan migrate and I can see all the database connections

Lumen has LoadEnvironmentVariables it gets called in line 3 in bootstrap/app.php file

eigan commented 3 years ago

(2) not all http requests works has expected,

When a config is cached will laravel no longer read the config/* files and instead use the cached config. The idea was to see if there actually was different env-variables in http vs cli.

If LoadEnvironmentVariables is loaded before the call in laravel-doctrine, and it loads the proper environment variables, then you might have found a bug. Please open a testcase and we can look into it. Otherwise I can't really help you further.

ukenpachi commented 3 years ago

(2) not all http requests works has expected,

When a config is cached will laravel no longer read the config/* files and instead use the cached config. The idea was to see if there actually was different env-variables in http vs cli.

If LoadEnvironmentVariables is loaded before the call in laravel-doctrine, and it loads the proper environment variables, then you might have found a bug. Please open a testcase and we can look into it. Otherwise I can't really help you further.

sorry I meant to say "all http request" works has expected and "NOT" was a typo

ukenpachi commented 3 years ago

$_ENV does not work in php cli I have to use Dotenv\Dotenv::createArrayBacked(base_path())->load(); instead