opengeospatial / ets-sta10

Repository for the Executable Test Suite for OGC Sensor Things API
Other
6 stars 8 forks source link

Differences between online test suite and local code at GitHub #47

Closed mariakrommyda closed 5 years ago

mariakrommyda commented 5 years ago

Hello everyone,

I am dealing with a mind-blowing issue. I have as a base URI, http://147.102.5.101:8090/SensorThing/v1.0/ When I run the test online, at http://cite.opengeospatial.org/te2/ I get a failed test.

read Entities With Filter Q O |  java.lang.AssertionError: An Exception occurred during testing!: A JSONObject text must begin with '{' at character 0

When I run locally the code available here, master, tag 1.0 or tag 1.2, I pass all tests without any reported problems.

Can you please help me identify the cause of this problem and/or how I can reproduce it locally so as to do the appropriate debugging?

Thank you for your time, Maria.

hylkevds commented 5 years ago

Can you post the full stack trace, so we can see where exactly the problem is?

When I try to insert the required starting entities (from http://cite.opengeospatial.org/te2/about/sta10/1.0/site/) your service throws an exception:

org.json.JSONException: JSONObject["result"] not found.
    at org.json.JSONObject.get(JSONObject.java:473)
    at sensorthing.validation.Observation.isValid(Observation.java:94)
    at sensorthing.apis.Things.Thing(Things.java:227)

Which is odd, since all Observations in that json have result values.

mariakrommyda commented 5 years ago

I am not sure how I can get the full stack trace from the online test. Please let me know how I can find it, so that I can post it.

Not sure why you got this error, I just tried the post and worked just fine. Can you please try again?

Maria.

hylkevds commented 5 years ago

Odd, now it works.

You have a bug in your time parsing, it doesn't recognise Z as a time zone: http://147.102.5.101:8090/SensorThing/v1.0/Observations?$filter=phenomenonTime%20ge%202015-03-03T00:00:00.000Z

mariakrommyda commented 5 years ago

Thank you for that, I have corrected it, but I am still getting the same error.

Is there a way to run that locally? So as not to take so much of your time?

Maria.

hylkevds commented 5 years ago

What is the problem with running the tests locally? It should generate an XML file in a folder named testng in either your home directory, or in the local directory. You should also see it perform the same queries as the on-line test does, by looking in the logfile of your server.

mariakrommyda commented 5 years ago

Actually running the code locally never gave me any issue regarding the filtering test. I could pass all without any problem.

I think that there is some difference between the code here and the one running at the test suite.

Anyway, after looking further into my date parsing the issue has been resolved.

Thank you for your help. Maria.

taniakhalafbeigi commented 5 years ago

There shouldn't be any change between the code. Every time there is a release, CITE people just get the code from here and deploy it in OGC Team Engine. However, I agree with you that sometimes running the tests on Team Engine have some odds. There were times I ran the test suite on Team Engine, got failed for the first time and the second time, no change and exact same initial data, it passed. So not quite sure what is happening! But I know it will pass after the second or third time of running!! But I think I can assure you that the releases are done exactly from this repo and as tagged. You can even see the commits for the releases that do the preparation.

mariakrommyda commented 5 years ago

Thank you both for your time and responses.