Invictum / serenity-reportportal-integration

Serenity TAF integration with Report Portal
Apache License 2.0
20 stars 19 forks source link

Can't activate the merging of the executed batches #136

Closed Jazzyekim closed 3 years ago

Jazzyekim commented 3 years ago

Hi, We execute the tests using BATCH strategy (in parallel). I experience the same issues as it was metioned here - each batch execution is reported as a separate launch. I tried to use suggested merge approach, by providing -Dserenity.rp.communication.dir=REPO/rp -Dserenity.rp.modules.count=3, but it doesn't seems to have any effect.

Here are the example of mvn command that I am using: mvn clean verify -Dserenity.rp.communication.dir=REPO/rp -Dserenity.rp.modules.count=3 -Dserenity.batch.count=3 -Dfailsafe.rerunFailingTestsCount=0 -Dserenity.batch.number=1 -Denvironment=test -Dcucumber.filter.tags=@virtual-endpoint -Dscreenshot.mode=none -DgridUrl=http://selenoid-grid.test-auto.aws.fisv.cloud:4444/wd/hub -Dcucumber.options=featureFiles -Dwebdriver.driver=provided -Dwebdriver.provided.type=mydriver -Dwebdriver.provided.mydriver=com.fd.taf.core.ui.DriverFactory the rest two are exactly the same, just serenity.batch.number is different - 2 and 3 respectively.

Could you please suggest something how to activate the merging mechanism?

Dependencies: serenity-reportportal-integration: 1.5.5 serenity-cucumber5: 2.2.6

Jazzyekim commented 3 years ago

Interesting...i have just tried to debug it locally and got an exception in the Shutdown hook set in get method of ReportLaunchProvider class. It failed here: Long id = reportPortal.getClient().getLaunchByUuid(uuid).blockingGet().getLaunchId(); with exception:

ERROR com.epam.reportportal.service.analytics.GoogleAnalytics - sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Exception in thread "Thread-1" java.lang.IllegalStateException: Executor pool shut down
    at com.epam.reportportal.restendpoint.http.HttpClientRestEndpoint.executeInternal(HttpClientRestEndpoint.java:491)
    at com.epam.reportportal.restendpoint.http.HttpClientRestEndpoint.executeRequest(HttpClientRestEndpoint.java:411)
    at com.epam.reportportal.restendpoint.http.proxy.RestEndpointInvocationHandler.executeRestMethod(RestEndpointInvocationHandler.java:59)
    at com.epam.reportportal.restendpoint.http.proxy.RestEndpointInvocationHandler.invoke(RestEndpointInvocationHandler.java:43)
    at com.sun.proxy.$Proxy21.getLaunchByUuid(Unknown Source)
    at com.github.invictum.reportportal.injector.ReportLaunchProvider.lambda$get$0(ReportLaunchProvider.java:45)
    at java.lang.Thread.run(Thread.java:748)

I do not see such exception when i run it on Jenkins though

v1-wizard commented 3 years ago

Hi @Jazzyekim if you just want to execute tests in parallel you could try the next sing:

  1. Don't setup BATCH strategy. Remove all this configurations.
  2. Add the next parameters to reportportal.properties
    rp.client.join=true
    rp.client.join.lock.file.name=<PATH_TO_SYNC_DIR>/reportportal.lock
    rp.client.join.sync.file.name=<PATH_TO_SYNC_DIR>/reportportal.sync

    This configuration will use mechanism to join test results from report portal java client

Invictum commented 3 years ago

Interesting...i have just tried to debug it locally and got an exception in the Shutdown hook set in get method of ReportLaunchProvider class. It failed here: Long id = reportPortal.getClient().getLaunchByUuid(uuid).blockingGet().getLaunchId(); with exception:

ERROR com.epam.reportportal.service.analytics.GoogleAnalytics - sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Exception in thread "Thread-1" java.lang.IllegalStateException: Executor pool shut down
  at com.epam.reportportal.restendpoint.http.HttpClientRestEndpoint.executeInternal(HttpClientRestEndpoint.java:491)
  at com.epam.reportportal.restendpoint.http.HttpClientRestEndpoint.executeRequest(HttpClientRestEndpoint.java:411)
  at com.epam.reportportal.restendpoint.http.proxy.RestEndpointInvocationHandler.executeRestMethod(RestEndpointInvocationHandler.java:59)
  at com.epam.reportportal.restendpoint.http.proxy.RestEndpointInvocationHandler.invoke(RestEndpointInvocationHandler.java:43)
  at com.sun.proxy.$Proxy21.getLaunchByUuid(Unknown Source)
  at com.github.invictum.reportportal.injector.ReportLaunchProvider.lambda$get$0(ReportLaunchProvider.java:45)
  at java.lang.Thread.run(Thread.java:748)

I do not see such exception when i run it on Jenkins though

@Jazzyekim Seems like native RP client is unable to handshake with Google analytics on HTTPS connection. It might be blocked or inaccessible from your local machine, but whitelisted on CI. Another reason could be different java versions on local machine and CI, more precise different cacert files content.

The simplest way to solve it is to disable Google analytics, but after quick search I don't see an option to disable it in java client.

As for another option you may copy cacert file from Jenkins to local machine, then it should work the same as it does on CI.

As for merge issue, it is better to try it one more time after resolving of SSL handshake issue.

Jazzyekim commented 3 years ago

Thank you guys, I will try the suggested hints however I suspect that we can't get rid of batching. We had issues with forking - the alternative method of parallelization, but it is a different story :)