code-distortion / adapt

A Laravel package that builds databases for your tests, improving their speed
MIT License
24 stars 3 forks source link

Multiple databases support #6

Closed manu144x closed 1 year ago

manu144x commented 1 year ago

I did not find any other way to ask this, I'm asking it here.

We have a Laravel application that uses 2 databases, some for some models, others for other. Is there a way to have both databases initialized and up and running in the exact same way with the same hashing/versioning features?

Thank you.

code-distortion commented 1 year ago

Hi @manu144x. Thanks for your question. Yes, Adapt can manage multiple databases.

When building a single database, you would configure Adapt by setting its config values or adding properties to your test class. Behind the scenes, Adapt builds a CodeDistortion\Adapt\DatabaseDefinition object which contains these settings. It then builds the database based on that.

Adding the method databaseInit(…) to your test class lets you define more of these DatabaseDefinition objects, in a programmatic way. Each one represents a connection that a database will be built for.

You can configure a second one with different settings to the first, including if you have a separate directory of migrations to run instead. Have a look at Building Extra Databases for the settings you can change.

You might like to put databaseInit(…) in a trait so it can be shared between tests more easily. However your code might look something like this:

// tests/Feature/MyFeatureTest.php

use CodeDistortion\Adapt\DatabaseDefinition;
use CodeDistortion\Adapt\AdaptDatabase;
use Tests\TestCase;

class MyFeatureTest extends TestCase
{
    use AdaptDatabase;

    protected function databaseInit(DatabaseDefinition $database): void
    {
        // $database contains the settings for your first db, based on
        // Adapt's config file and class properties

        // prepareConnection(…) creates another DatabaseDefinition.
        // It's also based on on your configuration, however you would
        // then tweak its settings
        $this->prepareConnection('mysql-2')
            // 'mysql-2' is your additional connection from config/database.php
            ->migrations('database/other_migrations_dir');
            // … add extra configuration tweaks as needed
    }

    // … test methods
}

I hope that all makes sense. Let me know if you have any questions.

manu144x commented 1 year ago

Beautiful, thank you.

I will implement this probably in the main TestCase class since we use it everywhere.

I will try to implement it but seems 99% on point what we needed.

Thank you very much for your quick response.

manu144x commented 1 year ago

Hey, sorry to come back with another issue, I tried to set this up but I can't get it to work in any way. I get this error:

Only one connection can be specified as being the default connection

My setup is hilariously simple:

protected function databaseInit(DatabaseDefinition $database): void
    {
        $this->prepareConnection('mysql_reporting')
            ->migrations('database/migrations/mysql_reporting');

    }

Am I missing anything here?

Also, another info would be that if I move the method in the an actual test case, not the base TestCase method, it won't seed the database.

If I don't override that method, everything works as expected, but of course, using only the default connection.

code-distortion commented 1 year ago

Hi @manu144x. Good questions, I'll have a look to see if I can replicate the issues

manu144x commented 1 year ago

Sure, let me know if you need help with anything.

code-distortion commented 1 year ago

Hi @manu144x, I found the bug that caused the error message you saw, and released 0.12.11 with the fix.

Regarding the second part, where you found that your database isn't getting seeded. I haven't been able to reproduce this one yet.

If this is still happening, would you mind opening a new issue? Including log output (https://www.code-distortion.net/docs/adapt/logging/) and any other details that might help me re-produce it will be helpful!

manu144x commented 1 year ago

I still haven't managed to get it working.

If I create the migrations table, in the secondary database, it complains that it exists already. If I remove it then it crashes complaining that it doesn't exist. I don't understand what's happening at all.

$this->prepareConnection('mysql_reporting')
            ->noSeeders()
            ->isNotABrowserTest()
            ->migrations('database/migrations/mysql_reporting');

With the error:

ADAPT DEBUG: ┌── Build Settings ──────────────────────────────────────────┐
ADAPT DEBUG: │ Using scenarios?    Yes                                    │
ADAPT DEBUG: │ Re-use method:      None, it will be rebuilt for each test │
ADAPT DEBUG: │ Verify db after?    No                                     │
ADAPT DEBUG: │ For a browser test? No                                     │
ADAPT DEBUG: │ Parallel testing?   No                                     │
ADAPT DEBUG: │ Build-checksum:     "60cb04acc136aa9665b88fdacc62c332"     │
ADAPT DEBUG: │ Snapshot-checksum:  "7ccb75c7afa8e97f3c9555d5bae0a2de"     │
ADAPT DEBUG: │ Scenario-checksum:  "81ada17271bd56ca2da57fb1a9996617"     │
ADAPT DEBUG: └────────────────────────────────────────────────────────────┘
ADAPT DEBUG: ┌── Resolved Database ──────────────────────────────────────┐
ADAPT DEBUG: │ Connection: "mysql_reporting"                             │
ADAPT DEBUG: │ Driver:     "mysql"                                       │
ADAPT DEBUG: │ Host:       "mysql_reporting"                             │
ADAPT DEBUG: │ Database:   "cia_reporting_testing_1_60cb04_81ada17271bd" │
ADAPT DEBUG: └───────────────────────────────────────────────────────────┘
ADAPT DEBUG: Changed the database for connection "mysql_reporting" to "cia_reporting_testing_1_60cb04_81ada17271bd"
ADAPT DEBUG: Dropped the existing database (4ms)
ADAPT DEBUG: Created a new database (5ms)
ADAPT DEBUG: Snapshot import: "database/adapt-test-storage/snapshots/snapshot.cia-reporting-testing-1.60cb04-7ccb75c7afa8.mysql" - not found (1ms)
ADAPT DEBUG: Set up the re-use meta-data (28ms)
ADAPT ERROR: ┌── An Exception Occurred - AdaptBuildException ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
ADAPT ERROR: │ An error occurred when running the migrations "/var/www/html/database/migrations/mysql_reporting"                                                                                                                                                                                                                       │
ADAPT ERROR: │ /var/www/html/vendor/code-distortion/adapt/src/Exceptions/AdaptBuildException.php on line 129                                                                                                                                                                                                                           │
ADAPT ERROR: │                                                                                                                                                                                                                                                                                                                         │
ADAPT ERROR: │ Previous Exception - Illuminate\Database\QueryException                                                                                                                                                                                                                                                                 │
ADAPT ERROR: │ SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'migrations' already exists (SQL: create table `migrations` (`id` int unsigned not null auto_increment primary key, `migration` varchar(255) not null, `batch` int not null) default character set utf8mb4 collate 'utf8mb4_unicode_ci' engine = InnoDB) │
ADAPT ERROR: │ /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php on line 760                                                                                                                                                                                                                               │
ADAPT ERROR: │                                                                                                                                                                                                                                                                                                                         │
ADAPT ERROR: │ Previous Exception - PDOException                                                                                                                                                                                                                                                                                       │
ADAPT ERROR: │ SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'migrations' already exists                                                                                                                                                                                                                              │
ADAPT ERROR: │ /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php on line 545               

Then if I remove the migrations table:

ADAPT DEBUG: ┌── Build Settings ──────────────────────────────────────────┐
ADAPT DEBUG: │ Using scenarios?    Yes                                    │
ADAPT DEBUG: │ Re-use method:      None, it will be rebuilt for each test │
ADAPT DEBUG: │ Verify db after?    No                                     │
ADAPT DEBUG: │ For a browser test? No                                     │
ADAPT DEBUG: │ Parallel testing?   No                                     │
ADAPT DEBUG: │ Build-checksum:     "60cb04acc136aa9665b88fdacc62c332"     │
ADAPT DEBUG: │ Snapshot-checksum:  "7ccb75c7afa8e97f3c9555d5bae0a2de"     │
ADAPT DEBUG: │ Scenario-checksum:  "81ada17271bd56ca2da57fb1a9996617"     │
ADAPT DEBUG: └────────────────────────────────────────────────────────────┘
ADAPT DEBUG: ┌── Resolved Database ──────────────────────────────────────┐
ADAPT DEBUG: │ Connection: "mysql_reporting"                             │
ADAPT DEBUG: │ Driver:     "mysql"                                       │
ADAPT DEBUG: │ Host:       "mysql_reporting"                             │
ADAPT DEBUG: │ Database:   "cia_reporting_testing_1_60cb04_81ada17271bd" │
ADAPT DEBUG: └───────────────────────────────────────────────────────────┘
ADAPT DEBUG: Changed the database for connection "mysql_reporting" to "cia_reporting_testing_1_60cb04_81ada17271bd"
ADAPT DEBUG: Dropped the existing database (4ms)
ADAPT DEBUG: Created a new database (5ms)
ADAPT DEBUG: Snapshot import: "database/adapt-test-storage/snapshots/snapshot.cia-reporting-testing-1.60cb04-7ccb75c7afa8.mysql" - not found (1ms)
ADAPT DEBUG: Set up the re-use meta-data (28ms)
ADAPT ERROR: ┌── An Exception Occurred - AdaptBuildException ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
ADAPT ERROR: │ An error occurred when running the migrations "/var/www/html/database/migrations/mysql_reporting"                                                                                                                                                                                                                       │
ADAPT ERROR: │ /var/www/html/vendor/code-distortion/adapt/src/Exceptions/AdaptBuildException.php on line 129                                                                                                                                                                                                                           │
ADAPT ERROR: │                                                                                                                                                                                                                                                                                                                         │
ADAPT ERROR: │ Previous Exception - Illuminate\Database\QueryException                                                                                                                                                                                                                                                                 │
ADAPT ERROR: │ SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'migrations' already exists (SQL: create table `migrations` (`id` int unsigned not null auto_increment primary key, `migration` varchar(255) not null, `batch` int not null) default character set utf8mb4 collate 'utf8mb4_unicode_ci' engine = InnoDB) │
ADAPT ERROR: │ /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php on line 760                                                                                                                                                                                                                               │
ADAPT ERROR: │                                                                                                                                                                                                                                                                                                                         │
ADAPT ERROR: │ Previous Exception - PDOException                                                                                                                                                                                                                                                                                       │
ADAPT ERROR: │ SQLSTATE[42S01]: Base table or view already exists: 1050 Table 'migrations' already exists                                                                                                                                                                                                                              │
ADAPT ERROR: │ /var/www/html/vendor/laravel/framework/src/Illuminate/Database/Connection.php on line 545               

It's like schroedinger's migrations table :)

code-distortion commented 1 year ago

Well we're moving forward at least.

Thanks for the logs. I'll have a look to see if I can reproduce this one

manu144x commented 1 year ago

I managed to get it working.

You should update the documentation I think :)

In reality if I override the function, I need to manage everything myself, the default database will not be used from the config.

Also the migrations with schema dumps don't work either, I need to use your schema import feature.

It's an all or nothing, which is nothing wrong, I like it, it's just not clear that it's that way. Here is the version that works fine:


    protected function databaseInit(DatabaseDefinition $database): void
    {
        // The default one needs to be fully managed here too, it will not use the config
        $database->connection('mysql')
            ->initialImports([
                'mysql' => ['database/schema/mysql-schema.dump'], //the normal migration will not recognize the squashed migrations in the schema
            ])
            ->migrations('database/migrations')
            ->seeders(['DatabaseSeeder'])
            ->isNotABrowserTest()
            ->transaction()
            ->noJournal()
            ->snapshots('!afterSeeders')
            ->makeDefault();

        $this->prepareConnection('mysql_reporting')
            ->initialImports([
                'mysql' => ['database/schema/mysql_reporting-schema.dump'],
            ])->migrations('database/migrations/mysql_reporting')
            ->noSeeders()
            ->isNotABrowserTest()
            ->transaction()
            ->noJournal()
            ->snapshots('!afterSeeders');
    }