search

eXist-db / exist-xqts-runner

W3C XQTS driver for eXist-db
GNU Lesser General Public License v3.0
5 stars 6 forks source link

[BUG] Error comparison seems to report false negatives #15

Closed line-o closed 4 years ago

line-o commented 4 years ago

In Class XQTS_3_1.xs-base64Binary

case base64-902 is reported as a Failure Judging from the data this should have been marked as a success instead. FORG0001 was expected and FORG0001 returned.

Expected: 'Error(FORG0001)', but query returned an error: QueryError(ERROR,exerr:ERROR FORG0001: Invalid base64 data [at line 1, column 1])

line-o commented 4 years ago

Another example in prod-PathExpr

expected AnyOf(List(Error(XPST0008), Error(XPST0003))) actual QueryError(ERROR,exerr:ERROR org.exist.xquery.XPathException: err:XPST0003 expecting "else", found ']' [at line 1, column 40] )

marked as Failure

line-o commented 4 years ago

There is a lot of those. Fixing this bug alone would give us slightly better overall results.

adamretter commented 4 years ago

Expected: 'Error(FORG0001)', but query returned an error: QueryError(ERROR,exerr:ERROR FORG0001: Invalid base64 data [at line 1, column 1])

This is a bug in eXist-db Error reporting. You can see above the exerr:ERROR which is the Error Code used by eXist-db when it doesn't know the correct error, the FORG0001 part only appears as a string within the Error Message returned by eXist-db. This is an eXist-db bug as FORG0001 should be returned as the Error Code and not (within) the Error Message.

expected AnyOf(List(Error(XPST0008), Error(XPST0003))) actual QueryError(ERROR,exerr:ERROR org.exist.xquery.XPathException: err:XPST0003 expecting "else", found ']' [at line 1, column 40] )

The same goes for this one.

There is a lot of those. Fixing this bug alone would give us slightly better overall results.

Yes, as you know eXist-db's error reporting is not great!

As this is an eXist-db bug (likely many bugs), I am closing this issue, as the cause is within eXist-db itself.