Add bitwise operators?

[eloff]

I ported some lua scripts recently, and because I do a lot of low-level stuff with the ffi I found that while overall the code become much more compact and readable, there was still the glaring ugliness of code like this:

import rshift, lshift, band, bor from bit d0[j] = lshift(i, 2) d1[j] = bor(rshift(band(i, 0x30), 4), lshift(band(i, 0x0f), 12)) d2[j] = bor(lshift(band(i, 0x03), 22), lshift(band(i, 0x3c), 6)) d3[j] = lshift(i, 16)

Which did not improve at all.

What do you think about adding bitwise operators? So that this code translates into the above?

d0[j] = i >> 2 d1[j] = ((i & 0x30) >>> 4) | ((i & 0x0f) << 12) d2[j] = ((i & 0x03) << 22) | ((i & 0x3c) << 6) d3[j] = i << 16

Where the bit operators would be defined as for javascript (>> arithmetic right shift, >>> logical right shift) and would not be overloadable (if you're using them performance usually matters a lot!) Precedence would be as is universal in other languages.

The syntax is widely used in various languages and is quite concise. What do you think?

Clearly you'd have to have the bitops library installed for Lua or use LuaJIT for the compiled code to run.

[stevedonovan]

This also strikes me as a good idea. If using LuaJIT, then the resulting code will run almost as fast as C.

A new operator is always a fun opportunity for overloading possibilities, but Dan is right about this one needing to be fast

[nilnor]

While having bitwise operators in Lua proper would be nice, it seems strange to me to add a language feature in Moonscript that's dependent on a library that might not even be present. Add to that 'bit' vs 'bit32' and it gets even more complicated.

[eloff]

The difference between bit32 and bit is easily handled a number of ways. Some as simple as bit32 or= bit. Or importing the actual bitwise functions into local variables first (which is definitely faster in vanilla Lua and recommended anyway.) Lua 5.2 and LuaJIT both have this library builtin. For the others it doesn't seem strange to me at all to say if you want to use the bitwise operators, download the bitops library. It could also be bundled with moonscript if need be, but as all Lua's in the future will have it anyway, it's probably not too much to ask Lua 5.1 users to install the extra dependency if they want to use bitwise operators. Either way works in my opinion.

[stevedonovan]

One area of concern is that although modern Lua has bitops, it has two different libraries (5.2 vs LuaJIT 2). Probably best to settle with Mike's bitops library (even for 5.2) since it's compatible with LuaJIT where it is handled in a very optimized way. Either way, not a big extra language dependency.

[eloff]

You're right Steve, it's not just a naming thing, for whatever misguided reason the Lua 5.2 folks decided to have their own bit library with subtly different semantics. In that case Mike's library is by far more portable and more sanely designed and specified (compare http://bitop.luajit.org/semantics.html to the Lua 5.2 reference manual.) So considering that I would be in favor of including the bitops library as a dependency (it would be unused if you are running on LuaJIT.)

Another alternative is to use the bit library available in the Lua VM with the understanding that the behavior of the moonscript will be different on different VMs. While that seems to be the decision made by the Lua 5.2 folks, I think it's ghastly, and would rather not repeat their mistake (I see no good reason for them to have broken compatibility with LuaJIT for arguably inferior and poorly documented semantics.)

[etandel]

I think the question of which bit library to use can be solved by a compile time flag:

moon --bit="bit32" script.moon

(when moonscript is excecuted via the require "moonscript" API, MS can check what library is available, if any)

I also think MS should not care about whether the user does or does not have the bit library available the same way it doesn't care about setfenv vs _ENV and other idiosyncrasies of different Lua interpreters.

[eloff]

What you say makes a lot sense @etandel. If the default behavior is to use Mike's bit library, if available, otherwise fallback to bit32, otherwise raise an error - then you almost don't need a flag. The only case you'd want the flag is if there's some third choice with a compatible interface or if you're on Lua 5.2 and have installed Mike's bit library as well, but you don't want to use it for some reason - that's pretty edge case - I'm not sure it warrants a flag. I agree completely that MS should not care about the availability of the library or idiosyncrasies of the implementation.

[eloff]

Forgot to add that while the xor operator is universally ^ in other languages, that's the power operator in Lua. So it would probably be best to make xor ^^ or ^|

[etandel]

@Eloff Though I think it would be quite confusing, one could change ^ to ** (python, fortran, ruby...) and make ^ mean xor. Just don't use ^^ because I'll always read it like this. =P

[eloff]

Haha, well I thought of doing that, but sadly Lua wanted to make ^ the power operator. It might come as a surprise to Lua programmers. I'd be in favor of doing things as you say as well, as ^ for xor and \ for power is nearly universal among languages (well those that have a power operator.) I'm not sure leafo would go for it though.

[etandel]

@Ellof Ah well, make ^ mean power and \ mean xor, then... Why not? XD

[eloff]

I'm going to try making a PR for this soon, if there's objections or alternative proposals please let me know. We can play with the syntax for the xor operator, I rather like ^| or ~|, but \ could work too.

[hashalon]

I am digging out this old issue. LUA 5.3 implements bitwise operators like ~ : NOT & : AND | : OR ~ : XOR it seems that moonscript support them except for the xor (a ~ b) get converted into (a (~b)) in LUA either moonscript should have a xor operator like \~\~ or (a ~ b) should be interpreted as (a ~ b) and the use of parenthesis should be mandatory for expression like func(~b)

[refi64]

FWIW, this is going to get messy under LuaJIT...

[RyanSquared]

@kirbyfan64 - that's correct, but MoonScript shouldn't hinder the abilities of 5.3 because it might not be available in a different runtime. This is a bug across all Lua versions, not MoonScript. It's not up to MoonScript to deal with this incompatibility.

[Nymphium]

MoonScript already has the operators (https://github.com/leafo/moonscript/pull/209).

MoonScript is just a transcompiler and the runtime is on Lua (5.1, 5.2, ... and LuaJIT), so the language itself doesn't have to think about the compatibility.

[buckle2000]

@kirbyfan64 LuaJIT linked with Lua 5.2 support bitwise operators.

[nilnor]

@buckle2000 that's not true as far as I know (though I would love to be wrong), I'm guessing you thinking of something else. There's a open request for this here fwiw.

[buckle2000]

@nilnor Yeah, you are right. I was misled by some non-official interpreter.

[RyanSquared]

@leafo mind closing this issue?

MoonScript already has the operators (#209).

[hashalon]

Really ? Even with the last moonscript compiler v0.5.0, from this code: --moonscript a = 5 b = 3 c = a ~ b print(c) I get this output: --lua local a = 5 local b = 3 local c = a(~b) return print(c) To me, the issue is not fixed yet.

[RyanSquared]

That's an alternative issue then; the unary ~ takes precedence over the binary ~.