Closed aucampia closed 2 years ago
making it draft until they all actually pass
A lot of test failures, will look at it when I have time. Good to at least have this on GitHub Actions thought.
Yes, happy to have the failures there as we can work through them and either fix any code bugs or, most likely, update the way testing is done to make things work. I guess the ens result will be pytest
-updated tests for everything and perhaps much more mocking of endpoints than use of actual remote endpoints which I think are the slow things.
It seems that POST is not supported in some type of query. Skippable?
$ grep -o 'ERROR: test.*test.virtuoso__v7' log
ERROR: testAskByPOSTinCSV (test.virtuoso__v7
ERROR: testAskByPOSTinCSV_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinJSON (test.virtuoso__v7
ERROR: testAskByPOSTinJSON_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinTSV (test.virtuoso__v7
ERROR: testAskByPOSTinTSV_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinUnknow (test.virtuoso__v7
ERROR: testAskByPOSTinUnknow_Conneg (test.virtuoso__v7
ERROR: testAskByPOSTinXML (test.virtuoso__v7
ERROR: testAskByPOSTinXML_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinJSONLD (test.virtuoso__v7
ERROR: testConstructByPOSTinJSONLD_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinN3 (test.virtuoso__v7
ERROR: testConstructByPOSTinN3_Conneg (test.virtuoso__v7
ERROR: testConstructByPOSTinTURTLE (test.virtuoso__v7
ERROR: testConstructByPOSTinTURTLE_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinJSONLD (test.virtuoso__v7
ERROR: testDescribeByPOSTinJSONLD_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinN3 (test.virtuoso__v7
ERROR: testDescribeByPOSTinN3_Conneg (test.virtuoso__v7
ERROR: testDescribeByPOSTinTURTLE (test.virtuoso__v7
ERROR: testDescribeByPOSTinTURTLE_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinCSV (test.virtuoso__v7
ERROR: testSelectByPOSTinCSV_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinJSON (test.virtuoso__v7
ERROR: testSelectByPOSTinJSON_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinTSV (test.virtuoso__v7
ERROR: testSelectByPOSTinTSV_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinUnknow (test.virtuoso__v7
ERROR: testSelectByPOSTinUnknow_Conneg (test.virtuoso__v7
ERROR: testSelectByPOSTinXML (test.virtuoso__v7
ERROR: testSelectByPOSTinXML_Conneg (test.virtuoso__v7
$ grep -o 'ERROR: test.*test.virtuoso__v8' log
ERROR: testAskByPOSTinCSV (test.virtuoso__v8
ERROR: testAskByPOSTinCSV_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinJSON (test.virtuoso__v8
ERROR: testAskByPOSTinJSON_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinTSV (test.virtuoso__v8
ERROR: testAskByPOSTinTSV_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinUnknow (test.virtuoso__v8
ERROR: testAskByPOSTinUnknow_Conneg (test.virtuoso__v8
ERROR: testAskByPOSTinXML (test.virtuoso__v8
ERROR: testAskByPOSTinXML_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinJSONLD (test.virtuoso__v8
ERROR: testConstructByPOSTinJSONLD_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinN3 (test.virtuoso__v8
ERROR: testConstructByPOSTinN3_Conneg (test.virtuoso__v8
ERROR: testConstructByPOSTinTURTLE (test.virtuoso__v8
ERROR: testConstructByPOSTinTURTLE_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinJSONLD (test.virtuoso__v8
ERROR: testDescribeByPOSTinJSONLD_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinN3 (test.virtuoso__v8
ERROR: testDescribeByPOSTinN3_Conneg (test.virtuoso__v8
ERROR: testDescribeByPOSTinTURTLE (test.virtuoso__v8
ERROR: testDescribeByPOSTinTURTLE_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinCSV (test.virtuoso__v8
ERROR: testSelectByPOSTinCSV_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinJSON (test.virtuoso__v8
ERROR: testSelectByPOSTinJSON_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinTSV (test.virtuoso__v8
ERROR: testSelectByPOSTinTSV_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinUnknow (test.virtuoso__v8
ERROR: testSelectByPOSTinUnknow_Conneg (test.virtuoso__v8
ERROR: testSelectByPOSTinXML (test.virtuoso__v8
ERROR: testSelectByPOSTinXML_Conneg (test.virtuoso__v8
$ grep -o 'FAIL: test.*test.virtuoso__v7' log
FAIL: testConstructByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v7
FAIL: testDescribeByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v7
FAIL: testSelectByPOSTinJSONLD_Unexpected (test.virtuoso__v7
FAIL: testSelectByPOSTinJSONLD_Unexpected_Conneg (test.virtuoso__v7
$ grep -o 'FAIL: test.*test.virtuoso__v8' log
FAIL: testConstructByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v8
FAIL: testDescribeByPOSTinCSV_Unexpected_Conneg (test.virtuoso__v8
FAIL: testSelectByPOSTinJSONLD_Unexpected (test.virtuoso__v8
FAIL: testSelectByPOSTinJSONLD_Unexpected_Conneg (test.virtuoso__v8
It is true that not all SPARQL endpoints implement POST. If we are to skip POST tests for those implementations, we need a very careful reporting mechanism to indicate which tests are skipped when and why.
Have a look at the SPARQL Endpoint Implementations in the README. I think we should turn those implementation notes into data objects (RDF graphs?) and then run tests per configuration. This is sort of happening now but the configuration is per testing file and it would be better to have a single data object with all tested implementations represented in it. We may then be able to reduce the number of testing files down a lot and just have one set of comprehensive tests that are then configured for use (changed parameter names; some skipped etc) with each tested implementation. Then we can just add new configuration for new tested implementations, e.g. AWS Neptune, rather than writing a new test file.
Made an issue for broken tests and re-opened the slowness issue:
abandoning this, will make a new one when I'm working on it.
MANIFEST.in
so it goes into sdist