Cannot run a test without a database connection even when `$usesDatabase = false`

[blueo]

Affected Version

5.0.8

Description

If I create a test for a module that doesn't require any DB connection, I'll get a connection error when running the test. Note that the error itself isn't the problem here - the problem is that there should be no attempt to connect to a database at all.

Steps to Reproduce

If I setup a basic test

<?php

namespace My\NameSpace;

use SilverStripe\Dev\SapphireTest;

class ExampleTest extends SapphireTest
{

    protected $usesDatabase = false;

    public function testExample(): void
    {
        $this->assertNotNull(null);
    }

}

it will result in the following error:

PHPUnit 9.6.9 by Sebastian Bergmann and contributors.

F                                                                   1 / 1 (100%)F

Time: 00:00.060, Memory: 16.00 MB

There were 2 failures:

1) My\NameSpace\Tests\ExampleTest::testExample
Failed asserting that null is not null.

/home/user/myproject/tests/ExampleTest.php:15
/home/user/myproject/vendor/phpunit/phpunit/phpunit:107
/home/user/myproject/vendor/bin/phpunit:97

2) My\NameSpace\Tests\ExampleTest::tearDownAfterClass
Exception in My\NameSpace\Tests\ExampleTest::tearDownAfterClass
Connection refused

/home/user/myproject/vendor/silverstripe/framework/src/ORM/Connect/MySQLiConnector.php:120
/home/user/myproject/vendor/silverstripe/framework/src/ORM/Connect/MySQLDatabase.php:91
/home/user/myproject/vendor/silverstripe/framework/src/ORM/DB.php:292
/home/user/myproject/vendor/silverstripe/framework/src/ORM/DB.php:106
/home/user/myproject/vendor/silverstripe/framework/src/ORM/Connect/TempDatabase.php:72
/home/user/myproject/vendor/silverstripe/framework/src/ORM/Connect/TempDatabase.php:82
/home/user/myproject/vendor/silverstripe/framework/src/ORM/Connect/TempDatabase.php:168
/home/user/myproject/vendor/silverstripe/framework/src/Dev/State/FixtureTestState.php:117
/home/user/myproject/vendor/silverstripe/framework/src/Dev/State/SapphireTestState.php:98
/home/user/myproject/vendor/silverstripe/framework/src/Dev/SapphireTest.php:420
/home/user/myproject/vendor/phpunit/phpunit/phpunit:107
/home/user/myproject/vendor/bin/phpunit:97

ideally there is no DB connection attempted if we're not using the DB at all. This probably only applies to simpler modules.

I can work around it with:

public static function setUpBeforeClass(): void
    {  
        // workaround to disable test database connection 
        parent::setUpBeforeClass();
        $states = static::$state->getStates();
        unset($states['fixtures']);
        static::$state->setStates($states);

        static::$tempDB = null;

    }

[GuySartorelli]

Labelled as affecting CMS 4 as well, since we didn't change the way test db stuff works for CMS 5.