yiisoft / yii2

Yii 2: The Fast, Secure and Professional PHP Framework
http://www.yiiframework.com
BSD 3-Clause "New" or "Revised" License
14.23k stars 6.91k forks source link

Testing support #45

Closed qiangxue closed 10 years ago

qiangxue commented 11 years ago

Better support for testing models and controllers.

klimov-paul commented 11 years ago

I am trying to say that first of all the framework should be testable. Then you can use any tool you wish to perform the test. Particular test framework or test tool should not be important. Anyone can write a unit test even without any test framework at all, like following:

$user = new ArUser();
$user->save();
if (ArUser::model()->findAll()==1) {
    echo ‘Test passed’;
} else {
    echo ‘Test failed!!!’;
}

And this will be unit test.

What I want to see in Yii is the ability to create simple test for the controllers and entire application. For example:


public function testContactUs() {
    $request = new HttpRequest(); // create empty request
    $application = new MyWebApplication(); // create application to be tested
    try {
        $application->processRequest($request); // process test request
    } catch (TestRenderException $e) {
        $this->assertEquals($e->viewFile, ‘views/site/contact.php’); // check correct view file rendered
        $this->assertContains($e->renderResult, ‘Contact’); // check correct content
        return;
    }
    $this->assertTrue(false, ‘No view render!’);
}
public function testContactUsSubmitSuccess() {
    $request = new HttpRequest(); 
    // Fill up request:
    $request->post = array(
         ‘email’=>’some@doamin.com’, 
         ‘content’=>’Test content’
    );
    $application = new MyWebApplication(); // create application to be tested
    try {
        $application->processRequest($request); // process test request
    } catch (TestRedirectException $e) {
        $this->assertEquals($e->redirectUrl, ‘site/contact/success’); // check correct redirect
        $email = $application->email->getLast();
        $this->assertEquals(‘admin@mysite.com’, $email->receiver); // check email receiver
        $this->assertContains($email->content, ’Test content’); // check email content
        return;
    }
    $this->assertTrue(false, ‘No redirect!’);
}

If framework allow me such things like test exceptions, isolated request processing and so on, who cares, which syntax will be used to actually write a test?

Many people here say Codeception is usefull and powerfull tool. I believe you, fine, it is. But its integration is not a big deal. It could be done anytime. Nothing is preventing you of its usage. Once Yii2 will proceed at least to “alpha” stage you are welcome to submit pull request, which will add Codeception support (if it would not be there already). Now I ask you to concentrate on other matters related to the testability. It is time to move forward.

I want to see the framework testable by simple “if” statement. If this will be achieved it will be suitable for any test platform (even without it at all). For me this is a noble goal, which implementation example I have provided above.

jacmoe commented 11 years ago

@klimov-paul what you still fail to get is the fact that Codeception has everything pretty much set up and good to go - out of the box! And a full range of choices, methodologies, whatever. It doesn't lock you into one particular school of testing.

I think one of the goals also should be to offer options which just works.

I can't count the number of topics in the Yii forum which deals with how to set up Selenium..

Codeception will not get in your way, either. You can use PHPUnit as before. Even with or without Codeception.

And, also: it is distributed as a phar, which is awesome. Easy to grab. PEAR has issues IMO. Even PEAR2.

Ragazzo commented 11 years ago

@klimov-paul if you are talking about Yii2 and testing Yii2 code then i got to say that codeception is not applied for that, Yii2 as i see developing with unit-tests, that is good. We talking about support testing applications build with Yii2 not Yii2 core-code.

But its integration is not a big deal. It could be done anytime.

agree, so i think it is a time to just stop discussion for now, and wait @DavertMik and @samdark collaboration on Hot-Code.

P.S. btw, the test that you provided for testing controllers is a little bit weird, i think it is better not to do so, anyway maybe it looks to me so because i dont like phpunit+selenium integration))

klimov-paul commented 11 years ago

The test I provided has nothing to do with Selenium or functional test at all. As I said, it could be written using plain PHP without any additional tools! Framework itself should have a "test gaps", which could be used.

The only thing, which such tool as Selenium is justified to be used, is JavaScript testing.

jacmoe commented 11 years ago

Your unit test looks totally useless. It should use fixtures, construct and tear down hooks. And, also: front end testing, ie functional testing, is equally important to unit testing IMO. We are not really discussing how testing should be done for the framework itself, are we?

DavertMik commented 11 years ago

@klimov-paul I definitely agree with you.

Framework itself should provide interfaces for testing. And here are the requiremenets. https://github.com/yiisoft/yii2/issues/45#issuecomment-15769203

If we all agree on that. I'd create issue for each point in my list. And we could start more productive discussion

samdark commented 11 years ago

Good plan. It will help Yii a lot even if we'll not adapt Codeception as an official solution.

DavertMik commented 11 years ago

At least you can adopt Codeception as recommended solution ;)

DavertMik commented 11 years ago

Here are the tickets for Yii2 testing design.

126

127

129

130

samdark commented 11 years ago

Thanks. These are really helpful suggestions.

schmunk42 commented 11 years ago

Here's my codeception setup with Yii 1.x and travis-ci:

It would be awesome to find a unified way for Yii2 to grab the tests of all installed extensions and run them.

nurcahyo commented 11 years ago
My opinion: the program should be designed in the way in could be tested via unit test only. At least this is valid for the PHP part of the application. I suppose we should concentrate on the matter to create enough “test gaps” in the Yii core, so it could allow writing unit tests for the most possible scenarios and features. For example: I should be able to easily check if the correct controller action was invoked for particular request data, I should be able to mock database operations (sometimes it is enough to check the data insertion has been invoked and not necessary to actually insert row into database), I should be able to mock Active Record classes (which is impossible in Yii 1.1.x) and so on. For me Yii core should support the PHPUnit and its official extensions, because this is definitely a standard in the PHP code testing. Anything beyond this is an additional feature.

:+1: for this

Like in Symfony Http Foundation we can test controller by create controller instance and call it action ID.

Ragazzo commented 11 years ago

@qiangxue first of 3 parts of that article that i was talking about, there are some comparisons with Behat/Behat+Mink and phpunit. http://codeception.com/05-06-2013/specification-testing-coparison.html

qiangxue commented 11 years ago

Very informative. Thank you!

Ragazzo commented 11 years ago

@qiangxue i will bump this issue one last time before HotCode conference (Samdark and DavertMik will also discuss it there), just to get your attention to some new informative codeception blog posts, that can be useful for you also, just see http://codeception.com/blog there are also second part of my article named (Playing with Cests), and some other good articles about phantomjs and nice summary Codeception 1.6.1: Laravel and Groups containing new release features and some links to articles where PageObject pattern usage in codeception is explained.

qiangxue commented 11 years ago

Thanks for the links. We just need time to align our priorities.

yolopreux commented 11 years ago

It will be nice to have ability like FactoryGirl in ruby (prepare db test data) and truncation strategy after run scenario or unit test.

resurtm commented 11 years ago

FactoryGirl in ruby

Correct me if i'm wrong, but FactoryGirl offers same functionality as fixtures does, isn't it? If so, fixtures mechanism will definitely be implemented in 2.0.

yolopreux commented 11 years ago

Yes it is mutch better than fixtures.

dmill-bz commented 11 years ago

I don't know if this has been mentioned or not but the ability to have multiple types of fixtures would be nice. Like file system fixtures and so forth (or possibility of making our own). I believe codeception supports these(at least partially) .

Ragazzo commented 11 years ago

@PommeVerte codeception support it in one-another way, but this topic not only about codeception. Also there are some good solutions for phpunit fixtures already, for FS too.

tonydspaniard commented 11 years ago

In order to provide Javascript test functionality, some references:

It is a bridge to use composer with npm packages, allowing us to install javascript test libraries.

radzserg commented 11 years ago

Right now functional tests doesn't work in basic app yii\base\InvalidConfigException: Object configuration must be an array containing a "class" element. according to yii2 module for codeception entry script should return config array. But now index-test.php looks similar to yii1 version. It's just entry script with testing config.

I tried yii with codeception in version 1 few months ago. We had discussion with @Ragazzo . But not much have been changed since the last time. My own opinion is that codeception is very nice for functional and acceptance testing. As for unit I don't see any benefits at this point. Moreover we have nice Fixture manager in yii1 but now we don't have it in yii2. (Codeception provide similar functionality, but IMO yii1 variant is more elegant) And we still can't make work it with yii noramally. I'd like to like to try yii2 with some of my toy project but testing is very important. I get used to write tests with my apps. But now this question is stuck. I believe it's the issue of concern to many developers.

So do we have any decision/progress according tests? Thanks

Ragazzo commented 11 years ago

As for unit I don't see any benefits at this point.

Codeception can run phpunit tests, the main thing here is that we have one entry point for all. Now in Yii2 there is no CDbFixture manager and other because of it is not done yet. You can use Yii2 with CDbTestCase/CTestCase when it will be done, as you see now there is not so much done yet. Also there were some problems with Yii1 and Codeception integration due that Yii1 do not provide some integrations for testing fw and other (headers, sessions, env. levels, etc). You are very welcome to provide some support/integrations too.

I also stand for CDbFixtureManager and CDbTestCase as they are very useful, the only thing that codeception will provide for unit-tests for yii is simple one-entry-point to run: php codecept.phar run unit users/UserSpecTest.php for example, and UserSpecTest < DbTestCase.

Thanks.

iJackUA commented 10 years ago

Btw what about Functional tests with Codeception (is it supposed at least to start somehow now) ? It fails to run right now on Basic app as test config provided with

        Yii2:
            entryScript: 'web/index-test.php'
            ...

and Yii2 connector expect to receive configuration array for Yii::createObject() even it to add in the index-test.php something like return \yii\helpers\ArrayHelper::merge($config, ['class' => 'yii\web\Application']); functional tests will start launching 2/3 are even passing, but fails on form submission as I understand it could faili due accessControl (it can't handle correctly HttpException) but I dont get why exactly it does happen

 1) Failed to ensure that login works in LoginCept.php
Sorry, I couldn't submit form "#login-form",{"LoginForm[username]":"admin","LoginForm[password]":"admin"}:
yii\web\HttpException: You are not allowed to perform this action.
qiangxue commented 10 years ago

Fixed tests and added readme file: https://github.com/yiisoft/yii2/blob/master/apps/basic/tests/README.md

Ragazzo commented 10 years ago

@qiangxue what do you mean by 2 Update tests ?

qiangxue commented 10 years ago

It shouldn't be there. I have removed it. Thanks.

Ragazzo commented 10 years ago

i think step 2 must be redeclared in this way:

php codecept.phar bootstrap

as in codeception tutorial (http://codeception.com/quickstart), because some people may think that they need manually create this .yml configs, and it is not so. So step 2: bootstrap command + simple explanation (very short) that already is there.

qiangxue commented 10 years ago

Since we already have the tests directory and initial configurations, why do we still need to bootstrap it?

Ragazzo commented 10 years ago

right, sorry, forgot that this are dist files.

qiangxue commented 10 years ago

DB fixture support added.

Ragazzo commented 10 years ago

@qiangxue i think it is better to keep this issue open for other suggestions from other developers, also this one https://github.com/yiisoft/yii2/issues/149 can be close i guess. Also what do you think about fixtures with rollback transactions? can this be implemented? will speed up tests.

qiangxue commented 10 years ago

What are the remaining action items for this issue? If there are new requests, it's better they can be described in new issues.

I'm not clear about the requirement for "fixtures with rollback transactions"? Could you create a new issue and give more details about it? Thanks.