range() function strange behaviour with arguments >= 2^24

[masmartyg]

Godot version: Godot Engine v3.2.3.stable.official

OS/device including version: Windows 10 Home 64bit ver. 2004 (build 19041.630) Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59GHz 16GB OpenGL ES 2.0 Renderer: GeForce GTX 1650/PCIe/SSE2 OpenGL ES 2.0 Batching: ON

Issue description: from arguments >= 167772156 (2^24) the range fails

range(16777215, 16777217)
output:
16777215

range(16777217, 16777219)
output:
16777216
16777217
16777218
16777219

Steps to reproduce:

func _ready():
    var a = 16777215
    var b = a + 2
    for n in range(a, b):
        print(n)

[CaptainProton42]

I did some digging, maybe it's helpful:

for n in range(a, b) uses an optimised implementation which replaces range with Vector2 (or Vector3, depending on the signature) which store their components as single-precisions floats which in turn can only represent integers between -2^24 and 2^24 exactly.

This does not apply to the "standalone" range function which uses plain integers internally. So something like

var r = range(16777217, 16777219)
for i in r:
  print(i)

produces the expected output:

16777217
16777218

The simplest fix I can think of is to check whether either of the arguments "fit" inside a Vector* and skipping the optimisation (thus using plain old range) if not.

Possibilities I see for keeping the optimisation:

• Adding Vector2i and Vector3i as variant types and using that for iteration
• Changing the implementation of for ... in range entirely

Other solutions I can think of:

• adding Vector2i and Vector3i as variant types (I tested it and this seems to work but is probably overkill)
• changing how for is implemented
• implementing a new class that implements an integer iterator (I'm not sure how this works yet)

This is also completely fixed on current master.

[omicron321]

it's even worse, scene silently crashes and closes (win10 version 10.0.19041.685) when using a pre declared/allocated range:

print("test with inlined range()")
    for i in range(100000000):
        pass

    print("test with my_range var")
    var my_range = range(100000000)
    for i in my_range:
        pass

    print("end of test (not) reached")

[Hicsy]

lulwat? can't tell if troll-post. This is by-design... of computers...

In-case not troll, here goes my bad attempt to explain: (me dumdum... need someone to big-brain explain: @oliverdunk)

16777217 is the first magic-number which doesn't exist in some binary representations. 32-bit Floating-Point number systems literally can't access it: https://introcs.cs.princeton.edu/java/91float/

That's why if you make a game where you run forever in a straight-line, everything starts going whacky after a certain pre-determinable distance. Best recent example in popular media would be Minecraft.

This particular number affects 32-bit floating-point number systems from memory... but there is unlimited examples - the actual issue is referred to as Floating Point Precision.

[oliverdunk]

@Hicsy I think you may have tagged the wrong person - I know nothing about Godot! Since I feel invested now though... what you say makes sense. Definitely wouldn't be the first time floating point numbers caused behaviour like this 😅

[Hicsy]

@oliverdunk my bad, sorry. I thought Matt/Standupmaths did something on the topic with code. I guess I forgot who it was...

Oh and @masmartyg - put it this way: Regarding the number 01111111 11111111 11111111 11111111

• A signed int32 maximum size is a TWO with 9 zeroes ~2 Billion
• A similar float32 might be a THREE with THIRTY-EIGHT zeroes The reason it can pack a much larger number range is by reserving the first portion of the bits for an exponent... so if only three (of the 4) portions are left-over, then we know there will be holes due to rounding.

ps I think due to multiplication, the holes get exponentially bigger as you get further from 1 (or -1).

simplistic eg: 2-times-2 is 4 2-times-2-times-2 is 8 OMG NUMBERS MISSING

[Listwon]

Still present in Godot Engine v3.3.2.stable.official

[Calinou]

For reference, this is fixed in master as the optimized range() now uses Vector2i and Vector3i. Those types can't be backported to 3.x without breaking compatibility with existing projects, so a different solution has to be designed for 3.x.