search

vaadin / flow

Vaadin Flow is a Java framework binding Vaadin web components to Java. This is part of Vaadin 10+.
Apache License 2.0
609 stars 167 forks source link

Enable maven parallel execution of modules #9887

Closed miguelatvaadin closed 2 years ago

miguelatvaadin commented 3 years ago

Description of the bug / feature

Enable maven parallel execution of modules (e.g. by using the T1C parameter).

As a first step, we should find the way to automatically assign a free port to the jetty (and to the webpack-dev-server, when involved). Also, some plugins are not marked as thread safe, so we should also check that there are not more conflicts which prevent us from running the modules in parallel.

Minimal reproducible example

run mvn -T1C -B -e -V -Pvalidation -Pnpm -Dcom.vaadin.testbench.Parameters.maxAttempts=2 clean verify

Expected behavior

build ends with SUCCESS and we experience a 15-20 minutes build time improvement

Actual behavior

integration test modules build fail when using the T1C parameter:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Flow 6.0-SNAPSHOT:
[INFO] 
[INFO] Flow ............................................... SUCCESS [  0.715 s]
[INFO] vaadin-buildhelpers ................................ SUCCESS [  1.253 s]
[INFO] Flow Push .......................................... SUCCESS [  0.271 s]
[INFO] Flow Generic Test Utilities ........................ SUCCESS [  0.784 s]
[INFO] Flow Server ........................................ SUCCESS [04:15 min]
[INFO] Flow Client ........................................ SUCCESS [01:28 min]
[INFO] Flow Data .......................................... SUCCESS [01:30 min]
[INFO] Flow HTML Components ............................... SUCCESS [ 19.769 s]
[INFO] Flow ............................................... SUCCESS [  0.011 s]
[INFO] Flow Polymer Templates Support ..................... SUCCESS [ 22.838 s]
[INFO] Flow Lit Templates Support ......................... SUCCESS [  6.385 s]
[INFO] Testing utilities for Flow ......................... SUCCESS [  1.095 s]
[INFO] TestBench elements for Flow HTML Components ........ SUCCESS [  2.576 s]
[INFO] Parent project for Flow themes ..................... SUCCESS [  0.050 s]
[INFO] Lumo theme for Flow ................................ SUCCESS [  1.587 s]
[INFO] Material theme for Flow ............................ SUCCESS [  1.587 s]
[INFO] Generic HTML 5 Drag and Drop support for Flow ...... SUCCESS [  4.318 s]
[INFO] Flow tests ......................................... SUCCESS [  0.065 s]
[INFO] Flow Test Resources ................................ SUCCESS [  0.720 s]
[INFO] Flow common test UI classes ........................ SUCCESS [  0.706 s]
[INFO] Fusion Endpoint .................................... SUCCESS [02:01 min]
[INFO] Flow Maven plugin .................................. SKIPPED
[INFO] Flow tests in pnpm and development mode ............ SKIPPED
[INFO] Flow tests in npm and development mode ............. SKIPPED
[INFO] Flow tests for dev mode ............................ SKIPPED
[INFO] Flow tests for PWA annotation ...................... SKIPPED
[INFO] Flow tests for PWA annotation (production mode) .... SKIPPED
[INFO] Flow root ui context tests ......................... SKIPPED
[INFO] Flow Router on custom context test ................. SKIPPED
[INFO] Flow tests for live reload in dev mode ............. SKIPPED
[INFO] Flow tests for Jetty reload in dev mode ............ SKIPPED
[INFO] Flow scalability tests ............................. FAILURE [  2.394 s]
[INFO] Flow memory leak tests ............................. SUCCESS [  5.311 s]
[INFO] Flow servlet registration test ..................... SKIPPED
[INFO] Flow Application Theme Test ........................ SUCCESS [  0.086 s]
[INFO] Flow reusable application theme .................... SUCCESS [  0.635 s]
[INFO] Flow application theme reusable tests .............. SKIPPED
[INFO] Test reusable application theme as parent .......... SKIPPED
[INFO] Flow tests for application theme live reload in dev mode SKIPPED
[INFO] Flow themes tests in NPM mode ...................... SKIPPED
[INFO] Flow Lumo theme tests .............................. SUCCESS [  2.053 s]
[INFO] Flow Material theme tests .......................... SUCCESS [  2.038 s]
[INFO] flow-servlet-containers-test ....................... SUCCESS [  2.144 s]
[INFO] Embedding Flow tests ............................... SUCCESS [  0.092 s]
[INFO] Flow Embedding test assets ......................... SUCCESS [  0.889 s]
[INFO] Flow Embedding, generic tests ...................... SKIPPED
[INFO] Flow Embedding, theme variant ...................... SKIPPED
[INFO] Flow Embedding, production tests ................... SKIPPED
[INFO] Flow Embedding application theme tests ............. SKIPPED
[INFO] Reusable custom theme for Flow Embedding test ...... SUCCESS [  0.674 s]
[INFO] Flow Embedding resuable application theme tests .... SKIPPED
[INFO] Flow Miscelaneous tests in npm ..................... SKIPPED
[INFO] Flow tests in pnpm and production mode ............. SKIPPED
[INFO] Flow tests in npm and production mode .............. SKIPPED
[INFO] Flow npm-only feature tests ........................ SUCCESS [  0.431 s]
[INFO] test-default-theme ................................. SKIPPED
[INFO] Flow general tests for npm only features ........... SKIPPED
[INFO] Flow test npm without BuildFrontendMojo ............ SKIPPED
[INFO] Flow test with custom frontend directory ........... SKIPPED
[INFO] Flow test of full class scanning in dev mode ....... SKIPPED
[INFO] Flow Server Production Mode ........................ SUCCESS [  0.175 s]
[INFO] Flow test of bytecode scanner in production mode ... SKIPPED
[INFO] Flow test fallback chunk in production mode ........ SKIPPED
[INFO] Startup performance regression test for dev mode ... SKIPPED
[INFO] Flow tests with more than one war deployed at the same time SUCCESS [  0.247 s]
[INFO] First war for multi war tests ...................... SKIPPED
[INFO] First war for multi war tests ...................... SKIPPED
[INFO] Bundle testing multiple war deployment ............. FAILURE [  7.398 s]
[INFO] Flow CCDM tests (dev mode) ......................... SKIPPED
[INFO] Flow CCDM tests (production mode) .................. SKIPPED
[INFO] Test navigation between ts and java views .......... SKIPPED
[INFO] Test navigation between ts and java views (production mode) SKIPPED
[INFO] Flow root context tests in NPM dev mode ............ SKIPPED
[INFO] Flow tests for no theme in NPM mode ................ SKIPPED
[INFO] Flow component demo helpers ........................ SUCCESS [  2.133 s]
[INFO] Flow Bill of Materials ............................. SUCCESS [  0.029 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:20 min (Wall Clock)
[INFO] Finished at: 2021-01-27T10:42:50+01:00
[INFO] ------------------------------------------------------------------------
miguelatvaadin commented 3 years ago

Here follows a link related to this task, pointed out by Soroosh: https://www.hascode.com/2014/05/allocating-available-random-ports-in-a-maven-build/

mcollovati commented 3 years ago

If it can be of interest, I use build-helper-maven-plugin for getting random ports for servlet containers and embedded databases during integration tests https://www.mojohaus.org/build-helper-maven-plugin/reserve-network-port-mojo.html

miguelatvaadin commented 3 years ago

If it can be of interest, I use build-helper-maven-plugin for getting random ports for servlet containers and embedded databases during integration tests https://www.mojohaus.org/build-helper-maven-plugin/reserve-network-port-mojo.html

Many thanks, Marco. I'll have a look at it

joheriks commented 2 years ago

Speeding up the Flow validation by parallel test execution is currently underway by @manolo .