search

playframework / play1

Play framework
https://www.playframework.com/documentation/1.4.x/home
Other
1.58k stars 683 forks source link

Play should run its own tests in precompiled mode #1154

Open asolntsev opened 7 years ago

asolntsev commented 7 years ago

Currently play own tests are executed in DEV mode. Thus they will not catch any errors that happen only in PROD mode (for example, see issue https://github.com/playframework/play1/pull/1152).

I believe we should run play tests in PROD mode. It will give stronger confidence that application works in production as in test.

flybyray commented 7 years ago

and why not just run

play precompile <appdir> --%test
play [auto-]test <appdir> -Dprecompiled=true

? The main reason why Play! runs tests in DEV is the default of jpa ddl update, which makes sense on an empty DB for a whole auto-test run.

The above approach is used by us on jenkins runs but we also use our own liquibase module

play precompile <appdir> --%test
play lb:install <appdir> --%test
play lb:update <appdir> --%test
play [auto-]test <appdir> -Dprecompiled=true

The lb:install will just apply fresh install migrations, the lb:update will do more or less nothing on a fresh install, but creation of views (for reports) is also handled by lb:update.

asolntsev commented 7 years ago

@flybyray Yes, you can run tests of some application this way. But I meant tests of Play itself. To build Play framework, we use command play test. And this command should run tests in precompiled mode - this is my point.

flybyray commented 7 years ago

do you mean ant test ?

asolntsev commented 7 years ago

Oh, sorry. Yes! I meant ant test (and ant autotest).

flybyray commented 7 years ago

just updated the first comment. but maybe you see reason for DEV-mode and empty db. that applies to most of the tests to. you should add jpa ddl update for prod in this case too

flybyray commented 7 years ago

some tests have bootstrap jobs which load data, in case the data yaml cotains templates you will need to precompile them too (this was just fixed some tickets ago). but you need to specify those templates in application.conf

flybyray commented 7 years ago

ok just checked none of the tests seem to have initial data with templates. from my point of view this is a missing code coverage

flybyray commented 7 years ago

@kill will not work. you may also want to set system property -Dwritepid=true to have a pid file to kill the server afterwards

Fraserhardy commented 6 years ago

Hi, We're trying to get coverage working on our play tests with JaCoCo. Are you aware of any way to get this working with Play framework? We're running 1.4.2 at the moment. I'm also interested to know if we can use ant directly to run tests too.

asolntsev commented 6 years ago

@Fraserhardy As much as I know, it's not possible. The problem is that JaCoCo needs to modify bytecode of your application, but Play also modifies the bytecode of your application. Those two frameworks conflict with each other.

Fraserhardy commented 6 years ago

@asolntsev Are you aware of any coverage tools that work with Play 1?

asolntsev commented 6 years ago

The problem I mentioned is actual for any other coverage tool. So I think it's basically not possible with Play 1.

On Feb 22, 2018 15:36, "Fraserhardy" notifications@github.com wrote:

@asolntsev https://github.com/asolntsev Are you aware of any coverage tools that work with Play 1?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/playframework/play1/issues/1154#issuecomment-367682601, or mute the thread https://github.com/notifications/unsubscribe-auth/AARE3b7kLI-YzJDU6SyU1tv3SKhZvxpSks5tXW1agaJpZM4Nn7OS .

flybyray commented 6 years ago

Should the codecoverage only be used interactively or should it be evaluated automatically? I noticed that Play1 uses a very outdated template for Intellij. If you have to adjust this, you can test it directly from Intellij including codecoverage (probably only Intellij Pro). https://drive.google.com/file/d/1dNAc55tz0YpC8BywIxeLi97qBmGcxYrG/view?usp=sharing

Fraserhardy commented 6 years ago

@flybyray Even within IntelliJ would be a massive bonus at the moment. We have had mixed success with test running direct in IntelliJ so far but I'll certainly try those steps from the video. Sounds like we need a PR to fix the Intellij template for Play then?

flybyray commented 6 years ago

the templating for various ides is outdated. at all the global movement to real modular systems should be better reflected in play.

take the eclipse templates for example. create a project with a module. add tests to your module. from command line tools of play it is valid to have module tests. in eclipse not???