mysite should have a sample test that works out of the box

[axyr]

When you want to create tests for a clean install of Silverstripe with this installer, there are quite some steps necessary before you can run tests.

This might be combination of documentation, unclear requirements and operator error.

These are the steps I needed to perform before being able to run some UnitTests for a module in a clean SS install :

1. Install Silverstripe
   • composer create-project silverstripe/installer myproject
   • running myproject.dev/install.php
2. Adding required files
   • phpunit.xml.dist
     • I did not got that file with the installer
     • This is required to find SapphireTest
   • _ss_environment.php
     • This is also needed to find SapphireTest
     • You need to add a $_FILE_TO_URL_MAPPING
     • Here I add define('SS_ENVIRONMENT_TYPE', 'dev'); as well
3. Again set dev mode in mysite/_config.php
   • I already have set dev mode in _ss_environment.php, but need to set it again?
4. Run cli commands
   • php framework/cli-script.php dev/build flush=all
     • needed even when the installation has created a manifest file.
   • vendor/bin/phpunit mymodule/tests
     • Yes, we got green!
   • php framework/cli-script.php dev/tests/cleanupdb
     • Ever seen like 500 ss_tmpdb123456xx databases pop up in your mysql browser, because you did not know there is a db create every testrun but not cleaned up?

Now, there might be some issues/comments due to operator error. Above steps are done on a vanilla MAMP PRO environment. And yes I read the documentation.

But when you do a clean install of Laravel, testing just works out of the box with nice config settings for testing like selecting another (type of) database. And it has a SampleTest present from where you can go on and create your own tests.

I really think everyone will benefit if testing is working out of the box and there will be a sample test in mysite for newcomers to see how easy it is to add tests to their project.

I would really like to help with this, but I think there are some framework core inner workings that might make this work not very easy?..

[tractorcow]

1. If you have a proper _ss_environment.php, you don't need to run install.php.
2. You should be getting that unless you're installing from dist. Try --prefer-source so that non-dist files are included.
3. If your _ss_environment.php is setup then you don't need to set dev mode in _config.php. I've never done so at least.
4. Tests will build their own DB, so you shouldn't need to have to explicitly create the DB.

Point taken on the cleanupdb issue... there's obviously some leakage there that needs fixing!

This is what I did just now to setup a test site.

cd ~/Sites # Where I have an _ss_environment.php
composer create-project silverstripe/installer ./ss40hackday 4.0.x-dev --prefer-source
cd ./ss40hackday
./vendor/bin/phpunit ./framework/tests/

And it ran my tests.

A basic ss_environment file can be seen at https://docs.silverstripe.org/en/3.2/contributing/making_a_silverstripe_core_release/ if you need a simple example.

[axyr]

Thanks for your reply.

I tried your way and that seems to work, with exception of an Out of Memory error while php.ini has 2048MB set for cli... but I will figure that out I hope..

Can it be that the order and amount of settings matter in the _ss_environment.php file?

I use this :

<?php

global $_FILE_TO_URL_MAPPING;
$_FILE_TO_URL_MAPPING['/Users/martijn/www/tests/silverstripe/test'] = 'http://test.silverstripe.tests.site';

define('SS_ENVIRONMENT_TYPE', 'dev');

since all other settings are in config.yml and mysite/_config.php

In your example $_FILE_TO_URL_MAPPING is set as:

$_FILE_TO_URL_MAPPING[__DIR__] = "http://localhost";

That seems so minimal that it might not be a config setting but a core value like BASE_PATH and THEME_DIR etc ?

When running tests from a command line, does it not make sense to or :

• set environment to dev whatever the setting might be. we are on the cli anyway.
• or have it set in the phpunit.xml like:

<env name="SS_ENVIRONMENT_TYPE" value="dev"/>

I understand that running and writing tests might require some more knowledge than average, but I think it is so essential that it should work just as easy as visiting the CMS after installation.

So that after doing a composer create-project, you run mysite/tests and show that in installation documentation, rather then having to setup extra configs.

Does that make sense?

[tractorcow]

Can it be that the order and amount of settings matter in the _ss_environment.php file?

I don't think so, since we should just be setting constants.

set environment to dev whatever the setting might be. we are on the cli anyway.

I agree with this... we should probably force "env=dev" when running phpunit and CLI. I think having it set in the phpunit.xml is probably a nice way to inject it.

I understand that running and writing tests might require some more knowledge than average, but I think it is so essential that it should work just as easy as visiting the CMS after installation.

I would consider a mock /mysite/tests/MyTest.php a possible improvement to the default installer. Worth checking to see what the @silverstripe/core-team feel about that though. ;)

I wonder if we should include some default sanity-checks in this test that validates any user-defined models for common errors. E.g. mis-matched db relationships is a common one. :P

[dhensby]

I would consider a mock /mysite/tests/MyTest.php a possible improvement to the default installer. Worth checking to see what the @silverstripe/core-team feel about that though. ;)

Wouldn't this mean dev requirements were now production requirements if we included tests out of the box?

[tractorcow]

Maybe we could exclude mysite/tests via gitattributes then?

[dhensby]

Good point

[axyr]

Please review my take on command line installation with a Sample Unit Test:

https://groups.google.com/d/msg/silverstripe-dev/YQ9ru7UuUV0/zWc-1pvwCQAJ

https://github.com/axyr/silverstripe-cli-installer

[sminnee]

Wouldn't this mean dev requirements were now production requirements if we included tests out of the box?

No. require and require-dev are parks of the scaffold of a new project and are unrelated to the development workflow of the silverstripe project.

• The require-dev requirements are the requirements to run development processes, such as test, on a new site.
• The require requirements are the requirements to run the site as it is intended in production, which excludes unit tests.

In particular require does not mean "anything that users of SilverStripe should use".

[sminnee]

I would consider a mock /mysite/tests/MyTest.php a possible improvement to the default installer. Worth checking to see what the @silverstripe/core-team feel about that though. ;)

I'd be happy with this. I would also modify phpunit.xml.dist to only run mysite/tests, and I would amend our Travis scripts to use something else, if they don't already.

[sminnee]

@axyr I think this issue covers too many disparate topics. It would be good to see a PR for silverstripe/installer that includes a default test and phpunit.xml.dist.

I'll open a separate ticket for ensuring that environments are configured on install.

[axyr]

For the record. Most of the existing code of the framework is used for the installation, besides the checking and writing of the config files and asking the questions.

I use the Symfony console command to interact with the user.

This is how it looks like :

and ain't silverstipe new <foldername> much nicer then composer create-project silverstripe/installer <foldername> ;)

[sminnee]

and ain't silverstipe new much nicer then composer create-project silverstripe/installer

This is a spurious example, because you've excluded the step to install the silverstripe CLI tool.

I think a silverstripe CLI tool makes sense as an optional addition, but people shouldn't be forced to download a 2nd tool to use silverstripe for the first time. To support that, silverstripe should just be a simple wrapper. It could even be a bash script. ;-)

The screenshot above looks good; I think it just needs to be refactored to a post-install hook that composer create-project can run, which is what #123 was intended to cover.

[GuySartorelli]

This issue is ancient and the discussion mostly revolves around functionality that no longer exists, or which has changed significantly.