search

opengeospatial / teamengine

TEAM Engine (Test, Evaluation, And Measurement Engine) is an engine for testing web services and other resources written in JAVA. It executes test scripts written in Compliance Test Language (CTL), TestNG and other languages. It is lightweight and easy to run as a command line or to setup as a service. It can be used to test any type of service or encoding. It is also the official tool used by the Open Geospatial Consortium (OGC) for compliance testing.
Apache License 2.0
45 stars 41 forks source link

TestNG test suites are instable with current master branch #531

Closed dstenger closed 2 years ago

dstenger commented 2 years ago

Describe the bug When a TestNG test suite (e.g. ets-ogcapi-features10) is put into the same tomcat as the deps artifact of ets-kml22 (https://search.maven.org/remotecontent?filepath=org/opengis/cite/ets-kml22/1.14/ets-kml22-1.14-deps.zip), the execution of the TestNG test suite fails with error:

Error in call to extension function {public java.lang.Object com.occamlab.te.TECore.callFunction(net.sf.saxon.expr.XPathContext,java.lang.String,java.lang.String,net.sf.saxon.om.NodeInfo) throws java.lang.Exception}: Exception in extension function net.sf.saxon.s9api.SaxonApiException: Error in call to extension function {public javax.xml.transform.Source org.opengis.cite.ogcapifeatures10.TestNGController.doTestRun(org.w3c.dom.Document) throws java.lang.Exception}: Exception in extension function java.util.ServiceConfigurationError: org.testng.ITestNGListener: Provider com.occamlab.te.spi.executors.testng.EarlReporter could not be instantiated

This error did not occur with TEAM Engine v5.4.1.

To Reproduce Steps to reproduce the behavior:

  1. https://github.com/opengeospatial/teamengine-docker project can be used to easily reproduce the error.
  2. Build TEAM Engine v5.5-SNAPSHOT locally.
  3. Add v5.5-SNAPSHOT to pom file of teamengine-docker project.
  4. Build and start TEAM Engine: mvn clean install && docker run --rm -p 8081:8080 --name beta ogccite/teamengine-beta
  5. TEAM Engine can be accessed via http://localhost:8081/te2/.
  6. Error can be reproduced by running ets-ogcapi-features10.
  7. Remove ets-kml22 dependency from teamengine-docker project.
  8. Error does not occur anymore when running ets-ogcapi-features10.

Expected behavior It should be analyzed why this error is occurring now and was not occurring previously (v5.4.1). Afterwards, a fix either in TEAM Engine or ets-kml22 shall be implemented.

bpross-52n commented 2 years ago

The reason for this is the slf4j-log4j12-1.7.6.jar library.

dstenger commented 2 years ago

I can confirm that removing dependency slf4j-log4j12-1.7.6.jar solves the problem described in this issue. Also ets-kml22 is still working.

Issue was created in ets-kml22 repository: https://github.com/opengeospatial/ets-kml22/issues/30

dstenger commented 2 years ago

@bpross-52n Could you please provide some more details why this library is breaking TEAM Engine? Afterwards, the issue can be closed.

bpross-52n commented 2 years ago

With Teamengine 5.5-SNAPSHOT, there is a ClassNotFoundException: org.apache.log4j.Level. With Teamengine 5.4.1 another library is present: slf4j-jdk14-1.7.21.jar. This contains an implementation of the org.apache.log4j.Level.