risinglightdb / sqllogictest-rs

Sqllogictest parser and runner in Rust.
Apache License 2.0
168 stars 42 forks source link

enhancement: better diff for result mismatch #176

Open xxchan opened 1 year ago

xxchan commented 1 year ago

We use space as separator for actual, but expected might use tab. This can look bad.

This can be tricky since we support custom validator.

https://github.com/risinglightdb/sqllogictest-rs/blob/27eb9f50993e10b36c1f4f68ad3afe499adbbb49/sqllogictest/src/runner.rs#L749-L759


Here's an example from @BugenZhao https://github.com/risingwavelabs/risingwave/pull/9671#issuecomment-1542102712

image

xxchan commented 1 year ago

Maybe we can:

wangrunji0408 commented 1 year ago

I'm curious what's the use case for validators? 👀

xxchan commented 1 year ago

Original added by @Xuanwo https://github.com/risinglightdb/sqllogictest-rs/pull/15

As indicated by its signature, the actual result is separated by columns Vec<Vec<String>>, but the expected result is not. So we need to normalize the results. validator is the most flexible form to achieve this, but not sure if it's really needed.

See also https://github.com/risinglightdb/sqllogictest-rs/issues/109

https://github.com/risinglightdb/sqllogictest-rs/blob/27eb9f50993e10b36c1f4f68ad3afe499adbbb49/sqllogictest/src/runner.rs#L411-L426

skyzh commented 1 year ago

I think we can parse both space and tab as separator in parser?

skyzh commented 1 year ago

Okay well this is not handled by parser. Then it might be a good idea to split by both space and tab in default validator :(