SORT/compare ignores the typespec of its function argument

[rebolbot]

Submitted by: BrianH

When you pass a function as the comparator of SORT/compare, that function should have two regular arguments, which will be values that are being compared. However, SORT/compare of a block containing error! or unset! values will still sort if you don't specify a typespec for those arguments, despite the fact that normally arguments without typespecs don't accept error! or unset! values. This is because SORT/compare doesn't do the type testing that a normal function call does - it passes in the arguments anyways. For that matter, the function doesn't need to take an argument at all, or can take more than two arguments: APPLY/only semantics are used for the function call, so none is passed to any additional arguments.

The main advantage to the lack of type checking is speed, plus we don't need to write a verbose type spec in our function! comparators. The disadvantage is that you can easily crash R3 if you pass any native function that would make sense to use as a comparator.

We need to add type checking to the function call, at least for natives. You can't crash R3 with function! or closure! code because of this, so it might be acceptable to keep the current behavior for those types. We don't want to exclude natives, actions, ops and commands altogether because they are too useful in this case.

Note that this bug wouldn't apply to SORT/compare of strings or binaries (even if that worked: see #1100). It just applies to any-block! types.

>> sort/compare [1 2 3] func [a b] [a > b]
== [3 2 1]

>> sort/compare [1 2 #[unset!]] func [a b] [a > b]
** Script error: b has no value
** Where: applier sort
** Near: a > b
; Note that the unset! got into the function, but triggered an error
; during normal evaluation. This would be acceptable.

>> sort/compare [1 2 3] :>
== [3 2 1]

>> sort/compare [1 2 #[unset!]] :>
; The R3 process crashes at this point without a system exception

^{CC - Data [ Version: alpha 97 Type: Bug Platform: All Category: Native Reproduce: Always Fixed-in:none ]}

[rebolbot]

Submitted by: abolka

In the core-tests suite.

[rebolbot]

Submitted by: szeng

Fixed by:

https://github.com/zsx/r3/commit/e23ae96beecf5612bbcc0da580415498c896416a

[rebolbot]

Submitted by: szeng

An updated fix:

https://github.com/zsx/r3/commit/cbfebfeded7f32056afdb35f6a748603d0d5af57

[rebolbot]

Submitted by: abolka

Wrapped up a basically identical fix in a PR:

https://github.com/rebol/rebol/pull/239