RDFLib / sparqlwrapper

A wrapper for a remote SPARQL endpoint
https://sparqlwrapper.readthedocs.io/
Other
513 stars 121 forks source link

Fix CI pipeline #179

Closed aucampia closed 2 years ago

aucampia commented 2 years ago
aucampia commented 2 years ago

making it draft until they all actually pass

aucampia commented 2 years ago

A lot of test failures, will look at it when I have time. Good to at least have this on GitHub Actions thought.

nicholascar commented 2 years ago

Yes, happy to have the failures there as we can work through them and either fix any code bugs or, most likely, update the way testing is done to make things work. I guess the ens result will be pytest-updated tests for everything and perhaps much more mocking of endpoints than use of actual remote endpoints which I think are the slow things.

eggplants commented 2 years ago

It seems that POST is not supported in some type of query. Skippable?

$ grep -o 'ERROR: test.*test.virtuoso__v7' log
ERROR: testAskByPOSTinCSV (test.virtuoso__v7
ERROR: testAskByPOSTinCSV_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinJSON (test.virtuoso__v7
ERROR: testAskByPOSTinJSON_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinTSV (test.virtuoso__v7
ERROR: testAskByPOSTinTSV_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinUnknow (test.virtuoso__v7
ERROR: testAskByPOSTinUnknow_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinXML (test.virtuoso__v7
ERROR: testAskByPOSTinXML_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinJSONLD (test.virtuoso__v7
ERROR: testConstructByPOSTinJSONLD_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinN3 (test.virtuoso__v7
ERROR: testConstructByPOSTinN3_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinTURTLE (test.virtuoso__v7
ERROR: testConstructByPOSTinTURTLE_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinJSONLD (test.virtuoso__v7
ERROR: testDescribeByPOSTinJSONLD_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinN3 (test.virtuoso__v7
ERROR: testDescribeByPOSTinN3_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinTURTLE (test.virtuoso__v7
ERROR: testDescribeByPOSTinTURTLE_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinCSV (test.virtuoso__v7
ERROR: testSelectByPOSTinCSV_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinJSON (test.virtuoso__v7
ERROR: testSelectByPOSTinJSON_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinTSV (test.virtuoso__v7
ERROR: testSelectByPOSTinTSV_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinUnknow (test.virtuoso__v7
ERROR: testSelectByPOSTinUnknow_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinXML (test.virtuoso__v7
ERROR: testSelectByPOSTinXML_Conneg (test.virtuoso__v7

$ grep -o 'ERROR: test.*test.virtuoso__v8' log
ERROR: testAskByPOSTinCSV (test.virtuoso__v8
ERROR: testAskByPOSTinCSV_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinJSON (test.virtuoso__v8
ERROR: testAskByPOSTinJSON_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinTSV (test.virtuoso__v8
ERROR: testAskByPOSTinTSV_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinUnknow (test.virtuoso__v8
ERROR: testAskByPOSTinUnknow_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinXML (test.virtuoso__v8
ERROR: testAskByPOSTinXML_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinJSONLD (test.virtuoso__v8
ERROR: testConstructByPOSTinJSONLD_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinN3 (test.virtuoso__v8
ERROR: testConstructByPOSTinN3_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinTURTLE (test.virtuoso__v8
ERROR: testConstructByPOSTinTURTLE_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinJSONLD (test.virtuoso__v8
ERROR: testDescribeByPOSTinJSONLD_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinN3 (test.virtuoso__v8
ERROR: testDescribeByPOSTinN3_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinTURTLE (test.virtuoso__v8
ERROR: testDescribeByPOSTinTURTLE_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinCSV (test.virtuoso__v8
ERROR: testSelectByPOSTinCSV_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinJSON (test.virtuoso__v8
ERROR: testSelectByPOSTinJSON_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinTSV (test.virtuoso__v8
ERROR: testSelectByPOSTinTSV_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinUnknow (test.virtuoso__v8
ERROR: testSelectByPOSTinUnknow_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinXML (test.virtuoso__v8
ERROR: testSelectByPOSTinXML_Conneg (test.virtuoso__v8
eggplants commented 2 years ago
$ grep -o 'FAIL: test.*test.virtuoso__v7' log
FAIL: testConstructByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v7
FAIL: testDescribeByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v7
FAIL: testSelectByPOSTinJSONLD_Unexpected (test.virtuoso__v7
FAIL: testSelectByPOSTinJSONLD_Unexpected_Conneg (test.virtuoso__v7

$ grep -o 'FAIL: test.*test.virtuoso__v8' log
FAIL: testConstructByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v8
FAIL: testDescribeByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v8
FAIL: testSelectByPOSTinJSONLD_Unexpected (test.virtuoso__v8
FAIL: testSelectByPOSTinJSONLD_Unexpected_Conneg (test.virtuoso__v8
nicholascar commented 2 years ago

It is true that not all SPARQL endpoints implement POST. If we are to skip POST tests for those implementations, we need a very careful reporting mechanism to indicate which tests are skipped when and why.

Have a look at the SPARQL Endpoint Implementations in the README. I think we should turn those implementation notes into data objects (RDF graphs?) and then run tests per configuration. This is sort of happening now but the configuration is per testing file and it would be better to have a single data object with all tested implementations represented in it. We may then be able to reduce the number of testing files down a lot and just have one set of comprehensive tests that are then configured for use (changed parameter names; some skipped etc) with each tested implementation. Then we can just add new configuration for new tested implementations, e.g. AWS Neptune, rather than writing a new test file.

eggplants commented 2 years ago

https://dbpedia.org's rate limit is harsh. https://github.com/RDFLib/sparqlwrapper/runs/4751129579?check_suite_focus=true

aucampia commented 2 years ago

Made an issue for broken tests and re-opened the slowness issue:

aucampia commented 2 years ago

abandoning this, will make a new one when I'm working on it.