search

spalacios23 / HAPI-Server-Tester

a script written in python to check random datasets on HAPI servers
0 stars 2 forks source link

Consider use of Bob's verifier in this framework #3

Open jbfaden opened 1 year ago

jbfaden commented 1 year ago

Bob and I showed that the verifier could be used with this. 

time curl "https://hapi-server.org/verify-dev/?url=http://hapi-server.org/servers/TestData2.0/hapi&id=dataset1&output=json"

This takes around three seconds to run, which meets the spec of running within three minutes for all servers. This request returns a json object containing number of successes and failures.

jbfaden commented 1 year ago

Also there's talk of an aliveness test url within the about json, and this could be used to identify the dataset. See https://github.com/hapi-server/data-specification/issues/135.

rweigel commented 1 year ago

For faster test, can specify only one parameter (~4x faster for this case; 1s vs 3.8s):

curl "http://hapi-server.org/verify-dev/?url=http://hapi-server.org/servers/TestData2.0/hapi&id=dataset1&parameter=scalar&output=json"
jbfaden commented 1 year ago

I think there might be an error when only the time is requested:

curl "https://hapi-server.org/verify-dev/?url=https://cottagesystems.com/HapiServerDemo/hapi&id=Iowa%20City%20Forecast&parameters=Time&output=json"

If I bring up the html version in a browser (https://hapi-server.org/verify-dev/?url=https://cottagesystems.com/HapiServerDemo/hapi&id=Iowa%20City%20Forecast&parameters=Time) it appears to be stuck in a loop.

rweigel commented 1 year ago

fixed.

jbfaden commented 1 year ago

Thanks. I started to play with this today, and I'll see if I can finish it off tomorrow. Note this is intended to be an aliveness test more than a verifier, so I might ask that we can further simplify/constrain the test. 

time wget -O foo.json "https://hapi-server.org/verify-dev/?url=https://cottagesystems.com/HapiServerDemo/hapi&id=Iowa%20City%20Forecast&parameters=Time&&output=json"

takes about 12 seconds to complete.

rweigel commented 1 year ago

I was thinking about that too. No need to check if YYYY-DOY syntax works, for example.

rweigel commented 1 year ago

Test is now minimal.

I could omit comparison that column(s) from all param request matches column(s) from single param request. Not sure if that is a good idea.

jbfaden commented 1 year ago

I'd suggest that if a test is trivial in time to perform, then you do it. I'll probably ignore the result of the test when reporting aliveness, but it doesn't hurt to perform it. I'll still print the results for reference.