Implement conditional operators

Submitted by: Ladislav

By "conditional expressions" I mean expressions that are used to supply CONDITION arguments to control functions like IF, EITHER, UNLESS, ...

Conditional expressions can also be found in BLOCK arguments of control functions like ANY, ALL, CASE, WHILE, UNTIL.

In programming languages the purpose of operators like NOT, AND, OR, XOR is to combine simple conditional expressions yielding complex conditional expressions.

However, in Rebol the only conditional operator I know is NOT, which is able to take a result of a conditional expression as an argument and yield a "conditional result".

That is not the case of the AND, OR and XOR operators, which cannot take conditional values as arguments. (see the example expression below)

The fact that NOT is conditional, while AND, OR and XOR are not makes the operators mutually incompatible. That confuses users not aware of such catches. My findings prove that too many users are being caught by that.

BrianH suggested that nonconditional AND, OR and XOR operators are better than conditional, since:

they can be used in conditional expressions with a help of the TRUE? function
they protect the user, since they "cut down on none propagation"

There are serious objections to both of these, though:

users don't remember the differences between TRUE?, TO-LOGIC and FOUND?, using the most inappropriate of the three functions too often
the expressivity of the language suffers, since the users don't want to replace conditional expressions like

    if value1 AND value2 OR value3 []

    if (true? value1) AND (true? value2) OR (true? value3) []

the "cut down on none propagation" property actually does not work for conditional expressions like the latter one anyway
virtually nobody knows that the counterpart of nonconditional AND, OR and XOR operators is not the conditional NOT operator, but the nonconditional COMPLEMENT operator. The usage of the COMPLEMENT operator is known to be much lower than the usage of the NOT operator. Notice however that if the users wanted to enjoy the purported benefits of the nonconditional operators, they would use the COMPLEMENT operator more frequently.

== Summary

the conditional NOT operator is preferred by the users, when compared to its nonconditional COMPLEMENT variant (because, what the users want is to have a conditional operator, not a nonconditional one)
Rebol seems to be currently the only programming language not having conditional AND and OR operators

== Possible solution

Make the AND, OR and XOR operators conditional
Similarly as for the NOT operator, the AND, OR and XOR operators already have nonconditional counterparts. Naturally, the nonconditional AND counterpart is INTERSECT (currently just duplicating some of the functionality), the natural nonconditional counterpart of OR is UNION, and the natural nonconditional counterpart of XOR is DIFFERENCE.
My user poll results suggest that the majority of users would welcome this change.

; OR cannot combine conditional subexpressions, ; the expression below triggers a script error none or none

^{Imported from: CureCode [ Version: alpha 111 Type: Wish Platform: All Category: Native Reproduce: Always Fixed-in:none ]} ^{Imported from: https://github.com/rebol/rebol-issues/issues/1879}

Comments:

Rebolbot commented on Oct 8, 2013:

Submitted by: fork

This example:

if (true? value1) AND (true? value2) OR (true? value3) []

Is a very common one, and was an annoyance to me until I started using ANY and ALL. After which, I rarely touched AND/OR.

But despite the flexibility ANY and ALL provide, there are some cases that would read better infix and without a block. Moreover, this is something every new user would struggle with prior to discovering ANY and ALL which they may not be as enthusiastic about as many experienced Rebol programmers are.

So this sounds like a good idea to me.

Rebolbot commented on Oct 17, 2013:

Submitted by: Gregg

+1

Rebolbot commented on Mar 5, 2014:

Submitted by: BrianH

Most languages which have conditional or logical operators, also have separate operators for bitwise operations. Rebol uses the same operators for both logical and bitwise operations. We would need to implement separate bitwise operators for AND, OR or XOR before we could make changes to those logical operators. Marked as problem until we agree to do that.

As for my opinion about this proposal, I find that these operators being strictly logical is only part of why they aren't used that much. The other part is that the expressions which generate these conditional values tend to be regular prefix Rebol expressions, which don't really interact well with any operators without using parens to handle the precedence (as Fork's code demonstrates). That awkwardness turns a lot of people towards the ANY and ALL functions, and would continue to do so even if the logical operators went conditional.

Ignoring the awkwardness though (as we can with AND~, OR~ and XOR~, their prefix counterparts), I have no other objections to making these operators conditional. As long as we get separate bitwise operators to resolve the conflict, this seems like a nice way to unify Rebol semantics, which will make it easier to understand and teach.

Rebolbot commented on Mar 6, 2014:

Submitted by: BrianH

"Is there any other conditional operator in Rebol?"

You are proposing in this ticket that AND, OR and XOR be changed to conditional. So, for the existing Rebol behavior before this change would be implemented, substitute "logical" for "conditional" in that comment (actually, I'll edit the comment to clarify that is what I meant).

And yes, the "not" operator has a conditional (or logical if you prefer) variant (NOT) and a bitwise variant (COMPLEMENT). That is similar to what would have to be done for AND, OR and XOR before we could go forward with this proposal. Make a separate proposal for those new bitwise operators and we will be able to move forward with both.

Rebolbot commented on Mar 7, 2014:

Submitted by: Ladislav

Having refreshed my memory, the current behaviour is as follows:

AND is a logical and bitwise operator

the change proposed above is to:

make AND a conditional operator
make INTERSECT a bitwise operator

similarly for the OR (UNION) and XOR (DIFFERENCE) operators

Rebolbot commented on Mar 7, 2014:

Submitted by: Ladislav

I adjusted the severity to major since it would require changes to existing code.

Rebolbot commented on Mar 7, 2014:

Submitted by: BrianH

Sounds good to me - the set functions are in need of a revamp anyways (epic to come, I have a long list of existing tickets).

Treating set functions as binary operations is fine, but we need to keep them as prefix functions. The non-binary set-like behavior for non-block data needs function options, existing and new. So, the binary operations would be for binaries and integers, and any others that we think should be, probably shortcut evaluated for speed.

The binary-operations change will still need another ticket (not enough info in "proposed above", and it's a different topic). I'll start it if it doesn't magically appear before I get back to Rebol stuff later today or tomorrow.

OK about the severity, but "require changes to existing code" is not a blocking criteria until after 3.0, and this is intended as a pre-3.0 change.

Rebolbot commented on Mar 7, 2014:

Submitted by: fork

I proposed that breaking PRs must include a link to a script to find suspected breakages and offer adaptation code.

Whether anyone cares to use the script or not isn't the issue. It being perfect is also not the issue. But think about even the funct change; anywhere that had FUNCTION not followed by exactly three blocks will break non obviously.

Not asking for the world here, just saying that people will waste time tracking stuff down if there's not some sort of due diligence. Cyphre's experience with TO LOGIC! reminded me of what a hassle the SHIFT sign interpretation between R2 and R3 was. Mapping out the impact of a change may question the basis of the change also.

Rebolbot commented on Mar 7, 2014:

Submitted by: Ladislav

"I proposed that breaking PRs must include a link to a script to find suspected breakages and offer adaptation code. "

This may be a problem in this specific case. The "logical operator" usage of AND, OR and XOR will not be influenced by this proposal.

If we added the logical capability to INTERSECT, UNION and DIFFERENCE, then it would be possible to just replace existing occurrences of AND, OR and XOR by INTERSECT, UNION and DIFFERENCE, but that is also problematic, since as BrianH mentioned, INTERSECT, UNION and DIFFERENCE are prefix.

Rebolbot commented on Mar 7, 2014:

Submitted by: fork

@Ladislav I said it did not have to be perfect. Start with just searching to find the words (like a text editor would) then filter. If you see a subtree where it's an AND under a PARSE but not in parens, put it in a "lesser" review queue for the person having to adapt to the change

I have done something similar with Draem as I keep changing the rules, so the rewriting is done with source-to-source transformations.

Rebolbot commented on Mar 7, 2014:

Submitted by: BrianH

It's not problematic, it just requires shifting to prefix expressions as well, which is not hard. It doesn't get to "problematic" until we try to do something impossible (which we can still do, but it takes more work). No worries here :)

Rebolbot commented on Mar 11, 2014:

Submitted by: Ladislav

To discuss it further, assuming that a consensus to change AND, OR and XOR to conditional operators is more or less present, I assume that:

AND~, OR~ and XOR~ prefix versions should also be transformed to conditional operators
INTERSECT, UNION and DIFFERENCE should be transformed to work also as logic operators and bitwise operators for binaries and integers

Rebolbot commented on Mar 21, 2014:

Submitted by: Ladislav

An alternative to the above is a variant proposed by Cyphre with AND~, OR~ and XOR~ transformed to bitwise operators, renaming them to AND#, OR# and XOR#. The renaming is needed for users of international keyboards (the "~" character requires 4! keystrokes here).

Rebolbot commented on Apr 28, 2015:

Submitted by: fork

On naming, I believe the prefix versions of the conditional forms could simply be named AND?, OR?, and XOR?...compliant with the idea that functions ending with ? return LOGIC! results.

This raises the question of if the non-question-mark bearing infix forms should take advantage of that to return more interesting values than TRUE/FALSE. e.g. a AND b returning what ALL [a b] would, while a OR b could return what ANY [a b] would. Thus because:

>> all [10 20]
== 20

That would suggest that 10 and 20 be 20, while and? 10 20 would be TRUE.

This may or may not offer a potential shade of meaning for NOT? vs NOT, which being single arity would have a hard time being meaningful otherwise. Perhaps NOT could be more focused toward going to NONE the way it would work with if false []? There is of course no specific value besides true to generate from a FALSE or NONE...unless NOT NONE was chosen to be equivalent to a random value generator that promised not to return FALSE or NONE. :-)

Food for thought. But the key here was in trying to name the prefix forms I realized AND?, OR?, and XOR? were perfectly fitting names.

Rebolbot commented on Sep 20, 2015:

Submitted by: abolka

Noting that I haven't expressly registered my agreement here: +1

+1 for making infix AND, OR, XOR conditional
+1 for making prefix AND~, OR~, XOR~ conditional as well
+1 for introducing a separate set of infix bitwise operators for binaries and integers (AND*, OR+, XOR-; or AND#, OR#, XOR#)

Rebolbot mentioned this issue on Jan 12, 2016: Eliminate AND~, OR~, XOR~ prefix variants in favor of ALL, ANY, and new ONE

Rebolbot added Type.wish and Status.important on Jan 12, 2016

Hostilefork commented on Jan 13, 2016:

This is mostly implemented in Ren-C, with outstanding issues below.

Similarly as for the NOT operator, the AND, OR and XOR operators already have nonconditional counterparts. Naturally, the nonconditional AND counterpart is INTERSECT (currently just duplicating some of the functionality), the natural nonconditional counterpart of OR is UNION, and the natural nonconditional counterpart of XOR is DIFFERENCE.

Sounds good on digital paper...and I was going with this in Ren-C. Until I found it's not quite true, e.g. there are different meanings to the set operations and bitwise operations in an input they accept in common: BINARY!

>> intersect #{FFFF} #{FF00}
== #{FF}

>> and~ #{FFFF} #{FF00}      
== #{FF00}

It depends on whether you consider a BINARY! as a "set of bytes" (the way strings are treated as sets of codepoints) or if you consider it a series of bits to be processed with bitwise boolean logic. I do not as yet have a solution for it, though interpretation of a BINARY! as a collection of bytes seems like something that would not be frequently used...compared to the bitwise function.

Choice for new namings of the bitwise infix operators is AND*, OR+, and XOR- ... which hints at their "mathy" function and also is consistent with the definitions. e.g. bitwise AND is considered boolean's "multiply" while OR is considered its ADD because of the similarity of interaction, such as with the identity operations. If X and Y are LOGIC!, and N and M are INTEGER then:

 (TRUE AND* X AND* Y) = (X AND* Y)  ;-- like (1 * N * M) = (N * M)
 (FALSE OR+ X OR+ Y)  = (X OR+ Y)   ;-- like (0 + N + M) = (N + M)

(Relating XOR to subtraction has some precedent too, I think...though more abstractly...and I'm not sure what one might put on the end of a NOT to make it mathy or if there just won't be one and you call COMPLEMENT)

As for prefix conditional, AND?, OR? and XOR? as proposed above are also in Ren-C. A NOT? might be added for good measure though it would be a synonym for NOT. Leading to holding off on that decision was the question of if AND [X Y] would be treated as ALL [X Y] and fit into the NONE vs. conditionally-true outcome, or if it would return TRUE or FALSE. But prudence, predictability, and realism dictate that e.g. true and false be FALSE, so it is better to force to LOGIC! even if that is "lossy".

For now the AND~, OR~, and XOR~ are still around... though ~ is likely best reserved for something else and it would be best to sort out the behavior of the INTERSECT, UNION, DIFFERENCE, and COMPLEMENT operations. I'd suggest they treat BINARY! as "sets of bits" vs. "sets of bytes" due to questioning the true usefulness of the latter in the scheme of things, though ideally all capabilities would be available somewhere.

Rebolbot mentioned this issue on Jan 22, 2016: [Epic] Backwards-incompatible API changes, for the greater good

Hostilefork commented on Jan 2, 2018:

Note: While this contains the gist of the change, several improvements have been made since. Please see the Trello card. Users can choose to put the right hand side in a block or a group, and if they use a group then they get non-short-circuit evaluation...but with the same value result of the overall operation.

Taking this popular user suggestion and implementing it--brought something to light over time that hadn't been discussed yet. @Ladislav said:

"Rebol seems to be currently the only programming language not having conditional AND and OR operators"

While that is true, we should notice as well that "most" languages have short-circuit AND and OR operators. Consider something like:

 b: 1020
 either (block? b) and (304 = length? b) [
     print "b is a block of length 304"
 ][
     print "b is either not a block, or does not have length 304"
 ]

If the motivation of the change is to give users something that feels more natural...I'd imagine the "natural" expectation would be to get the second message. But with this proposal unmodified, it would error...since it's not short circuit. So both sides of the AND would be evaluated, and you can't run length? 1024.

This points to one advantage that defining boolean operators as bitwise vs. conditional has. It avoids people getting bitten by their non-short-circuiting nature.

Yet we don't have to declare the issue a death-sentence for this popular-feeling proposal. While languages like C# need things like "lambda lifting" to get short circuit operator overloads, when other parts of Rebol face this problem...such as the branches of conditionals, they just put the code into a BLOCK!:

 b: 1020
 either (block? b) and [304 = length? b] [
     print "b is a block of length 304"
 ][
     print "b is either not a block, or does not have length 304"
 ]

The lack of symmetry is a bit off-putting, however. It can be explained, but it "looks funny". Nearly every expression on the left would need to be parenthesized in R3-Alpha due to the rules of left-hand infix. And every expression on the right would need to be in a block.

But Rebol's point is being bendable enough to dress up in the illusion you would like. If that weren't the point, there'd be no reason for infix at all...it would all be prefix. The "user-friendly" infix form of AND doesn't have to follow any particular interface rules...there's already AND? if you want non-short circuit behavior, and ALL if you need something more generic.

Using Ren-C's more generalized "left-enfix normality", we can have a short-circuiting operator that merely asks you to put your right hand side in a GROUP!:

and: enfix func [
    left [any-value!]
        {Expression which will always be evaluated}
    :right [group!]
        {Quoted expression, that will be evaluated only if LEFT is TRUTHY?}
][
    either left [to-logic do right] [false]
]

This allows you to write fairly normal looking expressions, so long as the right side is in a GROUP!, and get short-circuiting:

 b: 1020
 if block? b and (length of b = 304) and (...) [
     print "b is a block of length 304"
 ] else [
     print "b is either not a block, or does not have length 304"
 ]

Due to the workings of left normal enfix, the GROUP! also means it can chain. Function calls on the left must be completed before the right can run, so left conditions will short circuit arbitrarily long chains on the right.

This gives the user the requirement to put the right hand side in a GROUP!, but the option to put the left hand side in a GROUP!. Expressively it can look very natural, blending with some of the other illusions...and having a fairly easy-to-explain logic behind it. Not requiring a GROUP! on the left fits with the principle of minimum boilerplate...which drives the idea of why the condition of an IF doesn't need to be in a BLOCK!, while the condition of a WHILE does.

Just to cover all the ground: it's worth pointing out that not all languages--especially extensible ones--have means of doing short-circuiting in cases like these. C++ for example, does not short-circuit overloaded && or ||. And since as a language Rebol already has ANY and ALL for doing short circuiting, there could be a distinct value to having operators that do not short circuit.

So it would simply be a case of thinking about whether you want both sides of the expression to be evaluated or not. Certainly that's a useful distinction for code golf. But this may feel like a level of awareness that only a "power user" is going to be able to fathom...which undermines the argument of the change aiming at "user friendliness" in reading/writing.

Point is, there are options available here.

Abandon the idea of conditional AND/OR and go back to bitwise math. I feel that's lame.
Keep the idea but live with the non-short-circuit behavior...always evaluating both sides. I could get used to it, and it would have its applications, but it's not exactly a slam dunk over leaving the bitwise behavior...part of the argument for this was that it would help new users, not confuse them more.
Do it short circuit and have right-hand-side expressions in quoted GROUP!s...it looks good enough, while being a little sneaky. But sneakiness to achieve an illusion is part of what makes Rebol interesting. And people might have wanted to put the expression in a group in the first place anyway.
Do it short circuit and have right-hand-side expressions in BLOCK!s. I think this makes the code look a little "chunky" and less "balanced" than I'd like...when the operator is there for visual/layout reasons. Also it would be strange to hard-quote a BLOCK!...you'd have to hard quote it, because allowing foo: [...] bar: [...] if foo and bar [...] would treat the first foo conditionally, and run the second as code. So GROUP!s might not just be better looking, but also be less questioned over the hard quote, since it's obviously required for deferred execution.

Ren-C is going with option #3.

UPDATE: The actual mechanics wound up distinguishing quoted GROUP! on the right from quoted BLOCK! on the right, in a finesse that is really rather remarkably cool. GROUP! on the right implies forcing the result to logic, while BLOCK! implies value-preservation...more like an ANY or ALL would do. This makes OR [...] a very nice tool as a logic-driven analogue to ELSE. See:

https://trello.com/c/oweGGIgf

Hostilefork added a commit to hostilefork/rebol that referenced this issue on Jan 2, 2018: Require GROUP! for right hand side of short-circuit AND/OR/etc.

Hostilefork mentioned this pullrequest on Jan 2, 2018: Require GROUP! for right hand side of short-circuit AND/OR/etc. Unexpected behaviour of AND

Hostilefork added a commit to metaeducation/ren-c that referenced this issue on Jan 3, 2018: Require GROUP! for right hand side of short-circuit AND/OR/etc.

Hostilefork added a commit to hostilefork/rebol that referenced this issue on Jan 3, 2018: Require GROUP! for right hand side of short-circuit AND/OR/etc.

Rebolbot mentioned this issue on Jan 23, 2018: Remove 'FOUND?

Hostilefork mentioned this pullrequest on Jan 23, 2018: Fold AND~/OR~/XOR~ in INTERSECT/UNION/DIFFERENCE, AND*=>AND+

Hostilefork mentioned this issue on Jan 24, 2018: AND, OR and XOR permit any-string! types for value1 argument, spec doesn't match, value2 doesn't permit

Hostilefork mentioned this issue on Jan 25, 2018: Getting time delta on DATE!s should not use DIFFERENCE

Hostilefork added the Ren.resolved on Jan 23, 2018

Hostilefork commented on Jan 25, 2018:

This is largely resolved in Ren-C, with two outstanding issues:

Cannot treat BINARY! as set of bytes via AND+, OR+, XOR+

Getting time delta on DATE!s should not use DIFFERENCE

Note also this test from Rebol2 does not attempt to bitwise intersect individual characters in a string, but intersects the strings as sets:

  "^(03)^(00)" and+ "^(02)^(00)" = "^(00)" ;-- Rebol2 expected "^(02)^(00)"

Rebolbot mentioned this issue on May 2, 2018: ANY and ALL treat unset! as a value, not as none! or false!

Oldes / Rebol-issues

Implement conditional operators #1879

Comments: