BurntSushi / xsv

A fast CSV command line toolkit written in Rust.
The Unlicense
10.39k stars 324 forks source link

`NaN`s break `xsv sort -N` #271

Open vi opened 3 years ago

vi commented 3 years ago

NaNs, empty or broken values seem to silently break the entire sorting.

$ printf '3\n5\n2\n2.01\nNaN\n2\n0\n' | xsv sort -n -N
2
2.01
3
5
NaN
0
2

I expect them to aggregate somewhere before or after the proper numeric values and have subset of values which succeed to parse as numbers to be sorted properly. Somewhat like /usr/bin/sort -n.

As a minimum it should show some warning message (or maybe even have nonzero exit code) if sorting result is unreliable.

bluss commented 3 years ago

I'd suggest using a string in there to further inform the solution printf '3\n5\n2\n2.01\nNaN\nnot float\n20\n0\n' | ./target/debug/xsv sort -n -N. Then either change the default value in comparison (partial_cmp()'s unwrap_or), or just in next_num, don't classify NaN as a float but as something non-numeric. (The latter seems to be a smoother patch). Then it will sort with the non-numerics correctly, both forward and reverse.

When available, we'd use https://doc.rust-lang.org/nightly/std/primitive.f64.html#method.total_cmp