search

opengeospatial / teamengine

TEAM Engine (Test, Evaluation, And Measurement Engine) is an engine for testing web services and other resources written in JAVA. It executes test scripts written in Compliance Test Language (CTL), TestNG and other languages. It is lightweight and easy to run as a command line or to setup as a service. It can be used to test any type of service or encoding. It is also the official tool used by the Open Geospatial Consortium (OGC) for compliance testing.
Apache License 2.0
45 stars 41 forks source link

Empty Earl-report #480

Closed jyrkka closed 2 years ago

jyrkka commented 2 years ago

Earl report shows zero count for all test result types ('testsPassed', 'testFailed' etc.) in report when running tests in ubuntu.

Root cause for the problem is the if-statement in here: https://github.com/opengeospatial/teamengine/blob/a3a52cec199907df50223601ff52ddb19cbc26a9/teamengine-core/src/main/java/com/occamlab/te/CtlEarlReporter.java#L612

The temporary directory name is 'tmp' instead of 'temp'. When adding branch for 'tmp' everything works fine.

jyrkka commented 2 years ago

Just figured out a workaround. Overriding "java.io.tmpdir" system property with a directory containing "temp" solves the problem also.

dstenger commented 2 years ago

Thank you for reporting. We will check if the referenced code can be improved.

bpross-52n commented 2 years ago

@jyrkka I am trying to reproduce the behavior. Could you tell me how you tested, so I can try out a possible fix? Thanks!

jyrkka commented 2 years ago

I was running tests using those "all-in-one"-jars. For example "java -jar ets-wms13-1.3-aio.jar test-run-props.xml"

Content of the props.xml is following:

<?xml version="1.0" encoding="UTF-8"?>
<properties version="1.0">
    <comment>Test run arguments for WMS 1.3</comment>
    <entry key="acceptMediaType">application/zip</entry>
    <entry key="capabilities-url">PUT_CABABILITY_URL_HERE</entry>
    <entry key="updatesequence">auto</entry>
    <entry key="high-updatesequence">NA</entry>
    <entry key="low-updatesequence">NA</entry>
    <entry key="basic">basic</entry>
    <entry key="queryable">false</entry>
    <entry key="raster_elevation">false</entry>
    <entry key="vector_elevation">false</entry>
    <entry key="time">false</entry>
    <entry key="recommended">false</entry>
</properties>

With the above command, tests are run but the resulting report is empty. But when running tests for example with command "java -Djava.io.tmpdir=/tmp/temp_123454 -jar ets-wms13-1.3-aio.jar test-run-props.xml", report generated just fine.

And the reason is that the line which i pointed earlier, it does to parse the log files since in my system there is "tmp" folder instead of "temp" folder.