search

appsody / stacks

Appsody application stacks. This repo will be archived soon.
https://appsody.dev
Apache License 2.0
90 stars 120 forks source link

Request: Be able to run tests on existing server #159

Open aguibert opened 5 years ago

aguibert commented 5 years ago

Is your feature request related to a problem? Please describe. Currently appsody run runs the container project in the foreground of the shell. If I open a new terminal tab and do appsody test, it kills the appsody run process and the appsody test process fails too.

My current understanding is that if I want to run a test, I first have to kill the running container and then issue appsody test. Doing appsody test boots up an entire new container, starts a server from scrach, and then runs the tests. With the basic generated starter project this took 60 seconds on my machine (newish macbook pro).

Describe the solution you'd like We should be able to run tests if the server is already running. The actual time it takes for the starter tests to run is about 1 second, so this would greatly cut down on the "inner dev/test loop" by not having to stop->start->stop containers/servers each time the developer wants to run a test.

Describe alternatives you've considered I briefly consulted the "local development" doc page but did not see any mention of a way to run tests on an existing container.

ebullient commented 5 years ago

appsody test runs maven pre-set maven commands in both the Spring and microprofile stacks. The time/duration/restart/whatever depends a lot on the kind of tests and how they run (and whether or not they are running against the server at all).

The Spring stack runs the OOTB tests.. and those take 7s total (Spring round-trips the server start, too). I'm not sure that I would expect appsody test to do anything other than run the test target (so it can do whatever it does).

Would an appsody shell suffice(such that you could run appsody commands in any order within the stack environment)?

aguibert commented 5 years ago

If I run mvn verify for a baisc Liberty maven project that has been warmed up (i.e. run before to download dependencies) it only takes ~8s on my machine, so as long as we get down to that ballpark time-wise I'm OK. I don't really care how we get there.

For comparison, if I run appsody test for the basic Liberty maven project that has also been warmed up, it takes 1m51s (I originally thought this was 1m because I didn't notice that 2 maven commands were being run and I just looked at the duration of the last one).

If I do an appsody test it looks like 2 commands are run:

  1. mvn -Dmaven.repo.local=/mvn/repository install -DskipTests
  2. ../validate.sh && mvn -Dmaven.repo.local=/mvn/repository verify

A breakdown of the time shows:

  1. 11s for mvn install -DskipTests
  2. 44s for ../validate.sh (not certain about this, but I assume it's where the remainder of the time is spent)
  3. 56s for mvn verify

For "inner dev loop" activities, this needs to be much faster, ideally comparable to running bare-metal. The only suggestion I can think of right now for improving the time is to perhaps cut out mvn install and ../validate.sh for repeated runs if pom.xml/config has not changed. Perhaps there are other things we can do though?