Closed qiangxue closed 10 years ago
I am trying to say that first of all the framework should be testable. Then you can use any tool you wish to perform the test. Particular test framework or test tool should not be important. Anyone can write a unit test even without any test framework at all, like following:
$user = new ArUser();
$user->save();
if (ArUser::model()->findAll()==1) {
echo ‘Test passed’;
} else {
echo ‘Test failed!!!’;
}
And this will be unit test.
What I want to see in Yii is the ability to create simple test for the controllers and entire application. For example:
public function testContactUs() {
$request = new HttpRequest(); // create empty request
$application = new MyWebApplication(); // create application to be tested
try {
$application->processRequest($request); // process test request
} catch (TestRenderException $e) {
$this->assertEquals($e->viewFile, ‘views/site/contact.php’); // check correct view file rendered
$this->assertContains($e->renderResult, ‘Contact’); // check correct content
return;
}
$this->assertTrue(false, ‘No view render!’);
}
public function testContactUsSubmitSuccess() {
$request = new HttpRequest();
// Fill up request:
$request->post = array(
‘email’=>’some@doamin.com’,
‘content’=>’Test content’
);
$application = new MyWebApplication(); // create application to be tested
try {
$application->processRequest($request); // process test request
} catch (TestRedirectException $e) {
$this->assertEquals($e->redirectUrl, ‘site/contact/success’); // check correct redirect
$email = $application->email->getLast();
$this->assertEquals(‘admin@mysite.com’, $email->receiver); // check email receiver
$this->assertContains($email->content, ’Test content’); // check email content
return;
}
$this->assertTrue(false, ‘No redirect!’);
}
If framework allow me such things like test exceptions, isolated request processing and so on, who cares, which syntax will be used to actually write a test?
Many people here say Codeception is usefull and powerfull tool. I believe you, fine, it is. But its integration is not a big deal. It could be done anytime. Nothing is preventing you of its usage. Once Yii2 will proceed at least to “alpha” stage you are welcome to submit pull request, which will add Codeception support (if it would not be there already). Now I ask you to concentrate on other matters related to the testability. It is time to move forward.
I want to see the framework testable by simple “if” statement. If this will be achieved it will be suitable for any test platform (even without it at all). For me this is a noble goal, which implementation example I have provided above.
@klimov-paul what you still fail to get is the fact that Codeception has everything pretty much set up and good to go - out of the box! And a full range of choices, methodologies, whatever. It doesn't lock you into one particular school of testing.
I think one of the goals also should be to offer options which just works.
I can't count the number of topics in the Yii forum which deals with how to set up Selenium..
Codeception will not get in your way, either. You can use PHPUnit as before. Even with or without Codeception.
And, also: it is distributed as a phar, which is awesome. Easy to grab. PEAR has issues IMO. Even PEAR2.
@klimov-paul if you are talking about Yii2 and testing Yii2 code then i got to say that codeception is not applied for that, Yii2 as i see developing with unit-tests, that is good. We talking about support testing applications build with Yii2 not Yii2 core-code.
But its integration is not a big deal. It could be done anytime.
agree, so i think it is a time to just stop discussion for now, and wait @DavertMik and @samdark collaboration on Hot-Code.
P.S. btw, the test that you provided for testing controllers is a little bit weird, i think it is better not to do so, anyway maybe it looks to me so because i dont like phpunit+selenium integration))
The test I provided has nothing to do with Selenium or functional test at all. As I said, it could be written using plain PHP without any additional tools! Framework itself should have a "test gaps", which could be used.
The only thing, which such tool as Selenium is justified to be used, is JavaScript testing.
Your unit test looks totally useless. It should use fixtures, construct and tear down hooks. And, also: front end testing, ie functional testing, is equally important to unit testing IMO. We are not really discussing how testing should be done for the framework itself, are we?
@klimov-paul I definitely agree with you.
Framework itself should provide interfaces for testing. And here are the requiremenets. https://github.com/yiisoft/yii2/issues/45#issuecomment-15769203
If we all agree on that. I'd create issue for each point in my list. And we could start more productive discussion
Good plan. It will help Yii a lot even if we'll not adapt Codeception as an official solution.
At least you can adopt Codeception as recommended solution ;)
Here are the tickets for Yii2 testing design.
Thanks. These are really helpful suggestions.
Here's my codeception setup with Yii 1.x and travis-ci:
It would be awesome to find a unified way for Yii2 to grab the tests of all installed extensions and run them.
My opinion: the program should be designed in the way in could be tested via unit test only. At least this is valid for the PHP part of the application. I suppose we should concentrate on the matter to create enough “test gaps” in the Yii core, so it could allow writing unit tests for the most possible scenarios and features. For example: I should be able to easily check if the correct controller action was invoked for particular request data, I should be able to mock database operations (sometimes it is enough to check the data insertion has been invoked and not necessary to actually insert row into database), I should be able to mock Active Record classes (which is impossible in Yii 1.1.x) and so on. For me Yii core should support the PHPUnit and its official extensions, because this is definitely a standard in the PHP code testing. Anything beyond this is an additional feature.
:+1: for this
Like in Symfony Http Foundation we can test controller by create controller instance and call it action ID.
@qiangxue first of 3 parts of that article that i was talking about, there are some comparisons with Behat/Behat+Mink and phpunit. http://codeception.com/05-06-2013/specification-testing-coparison.html
Very informative. Thank you!
@qiangxue i will bump this issue one last time before HotCode conference (Samdark and DavertMik will also discuss it there), just to get your attention to some new informative codeception blog posts, that can be useful for you also, just see http://codeception.com/blog there are also second part of my article named (Playing with Cests
), and some other good articles about phantomjs
and nice summary Codeception 1.6.1: Laravel and Groups
containing new release features and some links to articles where PageObject pattern usage in codeception is explained.
Thanks for the links. We just need time to align our priorities.
It will be nice to have ability like FactoryGirl in ruby (prepare db test data) and truncation strategy after run scenario or unit test.
FactoryGirl in ruby
Correct me if i'm wrong, but FactoryGirl offers same functionality as fixtures does, isn't it? If so, fixtures mechanism will definitely be implemented in 2.0.
Yes it is mutch better than fixtures.
I don't know if this has been mentioned or not but the ability to have multiple types of fixtures would be nice. Like file system fixtures and so forth (or possibility of making our own). I believe codeception supports these(at least partially) .
@PommeVerte codeception support it in one-another way, but this topic not only about codeception. Also there are some good solutions for phpunit fixtures already, for FS too.
In order to provide Javascript test functionality, some references:
It is a bridge to use composer with npm
packages, allowing us to install javascript test libraries.
Right now functional tests doesn't work in basic app yii\base\InvalidConfigException: Object configuration must be an array containing a "class" element. according to yii2 module for codeception entry script should return config array. But now index-test.php looks similar to yii1 version. It's just entry script with testing config.
I tried yii with codeception in version 1 few months ago. We had discussion with @Ragazzo . But not much have been changed since the last time. My own opinion is that codeception is very nice for functional and acceptance testing. As for unit I don't see any benefits at this point. Moreover we have nice Fixture manager in yii1 but now we don't have it in yii2. (Codeception provide similar functionality, but IMO yii1 variant is more elegant) And we still can't make work it with yii noramally. I'd like to like to try yii2 with some of my toy project but testing is very important. I get used to write tests with my apps. But now this question is stuck. I believe it's the issue of concern to many developers.
So do we have any decision/progress according tests? Thanks
As for unit I don't see any benefits at this point.
Codeception can run phpunit tests, the main thing here is that we have one entry point for all. Now in Yii2 there is no CDbFixture manager and other because of it is not done yet. You can use Yii2 with CDbTestCase/CTestCase when it will be done, as you see now there is not so much done yet. Also there were some problems with Yii1 and Codeception integration due that Yii1 do not provide some integrations for testing fw and other (headers, sessions, env. levels, etc). You are very welcome to provide some support/integrations too.
I also stand for CDbFixtureManager and CDbTestCase as they are very useful, the only thing that codeception will provide for unit-tests for yii is simple one-entry-point to run:
php codecept.phar run unit users/UserSpecTest.php
for example, and UserSpecTest < DbTestCase
.
Thanks.
Btw what about Functional tests with Codeception (is it supposed at least to start somehow now) ? It fails to run right now on Basic app as test config provided with
Yii2:
entryScript: 'web/index-test.php'
...
and Yii2 connector expect to receive configuration array for Yii::createObject()
even it to add in the index-test.php
something like
return \yii\helpers\ArrayHelper::merge($config, ['class' => 'yii\web\Application']);
functional tests will start launching
2/3 are even passing, but fails on form submission
as I understand it could faili due accessControl (it can't handle correctly HttpException
) but I dont get why exactly it does happen
1) Failed to ensure that login works in LoginCept.php
Sorry, I couldn't submit form "#login-form",{"LoginForm[username]":"admin","LoginForm[password]":"admin"}:
yii\web\HttpException: You are not allowed to perform this action.
Fixed tests and added readme file: https://github.com/yiisoft/yii2/blob/master/apps/basic/tests/README.md
@qiangxue what do you mean by 2 Update tests
?
It shouldn't be there. I have removed it. Thanks.
i think step 2 must be redeclared in this way:
php codecept.phar bootstrap
as in codeception tutorial (http://codeception.com/quickstart), because some people may think that they need manually create this .yml configs, and it is not so. So step 2: bootstrap command + simple explanation (very short) that already is there.
Since we already have the tests directory and initial configurations, why do we still need to bootstrap it?
right, sorry, forgot that this are dist
files.
DB fixture support added.
@qiangxue i think it is better to keep this issue open for other suggestions from other developers, also this one https://github.com/yiisoft/yii2/issues/149 can be close i guess. Also what do you think about fixtures with rollback transactions? can this be implemented? will speed up tests.
What are the remaining action items for this issue? If there are new requests, it's better they can be described in new issues.
I'm not clear about the requirement for "fixtures with rollback transactions"? Could you create a new issue and give more details about it? Thanks.
Better support for testing models and controllers.