Speed not better than Python's int/float

[henrybzhang]

• [x] I have read the fastnumbers documentation and the README, and my question is still not answered

Hi,

I don't quite see the speed improvement in my own tests.

In [28]: len(numbs)
Out[28]: 1000000

In [29]: numbs[:5]
Out[29]: 
['0.7900414395464864',
 '0.9789330973579582',
 '0.8321431542764068',
 '0.3254459594374661',
 '0.9098654575745327']

In [33]: start_time = time.time()
    ...: for i in numbs:
    ...:     a = float(i)
    ...: print(time.time() - start_time)
    ...: 
0.44566822052001953

In [34]: start_time = time.time()
    ...: for i in numbs:
    ...:     a = fastnumbers.float(i)
    ...: print(time.time() - start_time)
    ...: 
0.5358688831329346

In [36]: %timeit float("3.1939130238")
The slowest run took 25.31 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 5: 140 ns per loop

In [37]: %timeit fastnumbers.float("3.1939130238")
The slowest run took 32.50 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 5: 213 ns per loop

Am I using the package incorrectly?

[SethMMorton]

I would expect them to perform equivalently. I suspect the reason you are measuring fastnumbers.float to be slower is that you are incurring the python lookup overhead with the dot operator. If you import as from fastnumbers import float as fnfloat then use fnfloat everywhere I suspect you should see the timings be about the same.

Why do I only expect them to be the same and not fastnumbers.float be faster?

For floats with as many digits as you have the fast algorithm does not kick in - it only works when there is less precision. This is hinted at in the How is fastnumbers so fast? section of the readme (emphasis below added to draw the eye).

CPython goes to great lengths to ensure that your string input is converted to a number correctly (you can prove this to yourself by examining the source code for integer conversions and for float conversions), but this extra effort is only needed for very large integers or for floats with many digits or large exponents. For integers, if the result could fit into a C long then a naive algorithm of < 10 lines of C code is sufficient. For floats, if the number does not require high precision or does not have a large exponent (such as "-123.45e6") then a short naive algorithm is also possible.

These naive algorithms are quite fast, but the performance improvement comes at the expense of being unsafe (no protection against overflow or round-off errors). fastnumbers uses a heuristic to determine if the input can be safely converted with the much faster naive algorithm. These heuristics are extremely conservative - if there is any chance that the naive result would not give exactly the same result as the built-in functions then it will fall back on CPython's conversion function. For this reason, fastnumbers is aways at least as fast as CPython's built-in float and int functions, and oftentimes is significantly faster because most real-world numbers pass the heuristic.

In your case, the heuristic is not passed.

I tried to highlight this in the timing tests where I showed the timing of "small" floats (e.g. low precision and low exponent) and "large" floats, and that the "large" floats are no faster than standard python.

[SethMMorton]

@c0ver Did changing the import methodology give better results?

[henrybzhang]

With the same numbers:

In [1]: from fastnumbers import float as fnfloat

In [8]: start_time = time.time()
   ...: for numb in numbs:
   ...:     a = float(numb)
   ...: print(time.time() - start_time)
   ...: 
0.36107563972473145

In [9]: start_time = time.time()
   ...: for numb in numbs:
   ...:     a = fnfloat(numb)
   ...: print(time.time() - start_time)
   ...: 
0.4131138324737549

Also, are you saying fnfloat('3.1') should be faster than float('3.1')?

In [17]: %timeit fnfloat('3.1')
The slowest run took 43.07 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 5: 159 ns per loop

In [18]: %timeit float('3.1')
The slowest run took 23.25 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 5: 86.8 ns per loop```

[SethMMorton]

On my OS (arch linux) and compiler (gcc) and hardware (Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz), I observe the following:

• fastnumbers is between 1.5x to 2.0x as fast as the builtin float for Python 3.6
• fastnumbers is only marginally faster (close to within the noise) than the builtin float on Python 3.7
• The speedup for integers is about the same for Python 3.6 and Python 3.7

So, I conclude that the Python devs have made quite a bit of improvements in their conversion methodology for floating point numbers so that fastnumbers.float is less meaningful :sob:. On your machine, obviously things differ enough from mine so that Python wins.

Thank you for pointing this out to me. I had not yet done timing tests on Python 3.7. I will update the documentation to make clear that users should evaluate for themselves on their own machines if using fastnumbers buys them anything (assuming they only want to use the builtin replacement functions... there is still a huge win if you want to use the functions with error-handling incorporated).

[henrybzhang]

Alright. Thanks for double checking my numbers.

[SethMMorton]

I'm going to keep this open till I update the documentation.