help combining itertools' takewhile and count

[copperfield42]

testing typing some of my stuff with numerary I came across this example

from typing import Union, Any, Iterable
from numerary import RealLike, IntegralLike
from itertools import takewhile, count

NumberType = Union[RealLike,IntegralLike]

def irange(start:NumberType, stop:NumberType, step:NumberType=1) -> Iterable[NumberType]:   
    """Simple iterador para producir valores en [start,stop)"""
    return takewhile(lambda x: x<stop, count(start,step))                 

mypy said this

C:\Users\ThecnoMacVZLA\Desktop\mypy_tests>mypy test3.py
test3.py:9: error: Argument 2 to "takewhile" has incompatible type "count[SupportsFloat]"; expected "Iterable[RealLike[Any]]"
Found 1 error in 1 file (checked 1 source file)

C:\Users\ThecnoMacVZLA\Desktop\mypy_tests>

I can make mypy happy by changing the result type to Iterable[Any].

What would be a better way to type hint this?

[posita]

This is an unfortunate reality of Python's own typing with respect to count. It's defined here. Note the arguments: start: _N, step: _Step = ....

_N is defined as:

_N = TypeVar("_N", int, float, SupportsFloat, SupportsInt, SupportsIndex, SupportsComplex)

_Step is defined as:

_Step: TypeAlias = SupportsFloat | SupportsInt | SupportsIndex | SupportsComplex

Note that both are restricted to (among other things) SupportsFloat. So the type of the iterator returned by count(RealLike, RealLike) is Iterator[SupportsFloat], which isn't quite what you want. There are a couple things you can do as a workaround. Since the error is in your implementation, you can silence it in a variety of ways. One way is to use cast:

from itertools import count, takewhile
from typing import Iterable, Iterator, cast
from numerary import RealLike

@overload
def irange(start: IntegralLike, stop: IntegralLike, step: IntegralLike = 1) -> Iterable[IntegralLike]:
    ...

@overload
def irange(start: RealLike, stop: RealLike, step: RealLike = 1) -> Iterable[RealLike]:
    ...

def irange(start: RealLike, stop: RealLike, step: RealLike = 1) -> Iterable[RealLike]:
    count_iter = cast(Iterator[RealLike], count(start, step))
    return takewhile(lambda x: x < stop, count_iter)

Another option is to wrap count with a runtime isinstance check, but this comes at a (slight) performance penalty (slight because the results are cached, but it's still a check performed at every iteration):

# …

@overload
def _my_count(start: IntegralLike, step: IntegralLike) -> Iterable[IntegralLike]:
    ...

@overload
def _my_count(start: RealLike, step: RealLike) -> Iterable[RealLike]:
    ...

def _my_count(start: RealLike, step: RealLike) -> Iterable[RealLike]:
    for val in count(start, step):
        assert isinstance(val, RealLike)
        yield val

# …

def irange(start: RealLike, stop: RealLike, step: RealLike = 1) -> Iterable[RealLike]:
    count_iter = _my_count(start, step)
    return takewhile(lambda x: x < stop, count_iter)

There are probably other approaches, but those are what come to mind first.

[copperfield42]

I might want to please mypy, but runtime check is overkill, so cast is it, that also solve the other that was given me problems, Decimal, in like for example

from decimal import Decimal

def fun(a:NumberType) -> Decimal
    return Decimal(a)

now

def fun(a:NumberType) -> Decimal
    a = cast(Decimal,a)
    return Decimal(a)

why mypy is so hard to please ლ(¯ロ¯"ლ)

anyway, thank you for your help and this module :)

[posita]

FYI, in most cases, where isinstance(foo, IntegralLike) is True, isinstance(foo, RealLike) should also be True, so you can probably use RealLike wherever you are currently using your NumberType union. (If you run into a counterexample, I would be really curious to know about it.) IntegralLike is still useful in cases like the above, where you want to refine an overloaded return type depending on narrower versions of the arguments.

[posita]

Regarding your Decimal issue, that's because its constructor is pretty limited. (You're probably getting an error like Argument 1 to "Decimal" has incompatible type "RealLike[Any]"; expected "Union[Decimal, float, str, Tuple[int, Sequence[int], int]]" [arg-type].) Because RealLike includes SupportsFloat, you can also do something like the following (which risks being lossy because of the float conversion):

from decimal import Decimal
from numerary import RealLike

def fun(a: RealLike) -> Decimal:
    return Decimal(float(a))

That's assuming you don't care about converting integrals to floats (i.e., the following oddity doesn't bother you):

>>> from decimal import Decimal
>>> Decimal(1)
Decimal('1')
>>> Decimal(1.0)
Decimal('1.0')
>>> Decimal(1) == Decimal(1.0)
True

If you do care about that, you can leverage the fact that IntegralLike includes SupportsInt and augment the above as follows:

from decimal import Decimal
from numerary import IntegralLike, RealLike

def fun(a: RealLike) -> Decimal:
    return Decimal(int(a)) if isinstance(a, IntegralLike) else Decimal(float(a))

>>> fun(1)
Decimal('1')
>>> fun(1.5)
Decimal('1.5')
>>> fun(1.2)
Decimal('1.1999999999999999555910790149937383830547332763671875')
>>> from fractions import Fraction
>>> fun(Fraction(1))
Decimal('1')
>>> fun(Fraction(3, 2))
Decimal('1.5')
>>> fun(Fraction(6, 5))
Decimal('1.1999999999999999555910790149937383830547332763671875')

[copperfield42]

I don't like the overload thing, is fine for mypy I suppose, but it lose the typing info if I call help on the thing and that defeat the purpose to me, I write code for people not mypy, and needing to either go to the source code somewhere in your file system or to the documentation somewhere in the internet that might not even exist to get the full signature, when a simple call to help should be more than enough...

And about the union, I think that might be like the best replacement for numbers.Number, the one I rather use, for being the more broad conceptually and also pleasing mypy... I rather have it display like NumberType or similar rather than the union thing

>>> help(fun)
Help on function fun in module __main__:

fun(a: Union[numerary.types.RealLike, numerary.types.IntegralLike])

but good enough I suppose

and in work with isintance too, which is nice

>>> isinstance(1,NumberType)
True
>>> isinstance(1.1,NumberType)
True
>>> isinstance(decimal.Decimal(1),NumberType)
True
>>> isinstance(fractions.Fraction(1),NumberType)
True
>>> 

---

and about float conversion to give it to decimal, I rather avoid that like the plague XD in my case I don't want to be that defensive, if Decimal fail at run time let it fail...

[copperfield42]

now that you mention counter examples, I made a number subclass but it does not detect it

>>> import poly
>>> r5=poly.constante.RadicalConstante(5)
>>> print(r5)
((5)**1/2)
>>> r5**2
5
>>> type(_) #anything to avoid floating point shit XD
<class 'int'>
>>> 
>>> float(r5)
2.23606797749979
>>> _**2
5.000000000000001
>>> 
>>> isinstance(r5,NumberType)
False
>>> import numbers
>>> isinstance(r5,numbers.Number)
True
>>> dir(r5)
['__abs__', '__abstractmethods__', '__add__', '__annotations__', '__bool__', '__call__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__floordiv__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__mod__', '__module__', '__mul__', '__ne__', '__neg__', '__new__', '__pos__', '__pow__', '__radd__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__', '__rmul__', '__rsub__', '__rtruediv__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__sub__', '__subclasshook__', '__truediv__', '__weakref__', '_a', '_abc_impl', '_division', '_e', '_m', '_name', '_partes', '_rdivision', '_value', 'a', 'config', 'e', 'm', 'make_new', 'name', 'unity', 'value']
>>> type(r5).mro()
[<class 'poly.constante.RadicalConstante'>, <class 'poly.constanteclass.ConstanteBase'>, <class 'poly.constanteclass.ConstanteABC'>, <class 'numbers.Number'>, <class 'abc_recipes.PowClass'>, <class 'abc_recipes.ConfigClass'>, <class 'abc.ABC'>, <class 'object'>]

I wonder why... but anyway, I will worry about that later when I get to mypy that particular module after I finish the one I'm currently working on...

[posita]

Thanks for sharing your use case! I appreciate it. And good on you for caring about human-friendly interfaces! Let me know if I can be of any further assistance!

FYI, I think my only point above was a very minor one, namely that RealLike is likely sufficient for your tests.

>>> from numerary import RealLike
>>> isinstance(1, RealLike)
True
>>> isinstance(1.1, RealLike)
True
>>> import decimal
>>> isinstance(decimal.Decimal(1), RealLike)
True
>>> import fractions
>>> isinstance(fractions.Fraction(1), RealLike)
True

Your documentation concern is well received. I wonder if this might help?

# <your_module>
from numerary import RealLike

class NumberType(RealLike):
    ...

def fun(a: NumberType) -> Decimal:
    return Decimal(int(a)) if isinstance(a, IntegralLike) else Decimal(float(a))

Then you could do:

>>> from <your_module> import NumberType, fun
>>> help(fun)
Help on function fun in module <your_module>:

fun(a: <your_module>.NumberType) -> decimal.Decimal

>>> isinstance(1, NumberType)
True
>>> isinstance(1.1, NumberType)
True
>>> import decimal, fractions
>>> isinstance(decimal.Decimal(1), NumberType)
True
>>> isinstance(fractions.Fraction(1), NumberType)
True

And you'd still get the benefit of caching (warning, implementation details exposed):

>>> NumberType._abc_inst_check_cache
{<class 'int'>: True, <class 'float'>: True, <class 'decimal.Decimal'>: True, <class 'fractions.Fraction'>: True}

That might be overkill, though. In any event, thanks very much for the kind bug report!

[posita]

… I wonder why... but anyway, I will worry about that later when I get to mypy that particular module after I finish the one I'm currently working on...

If you have the ability to point me to some sources, I can certainly help debug this. Without looking into it more deeply, I suspect it's because you're either missing a(n) __float__ method, a(n) __rmod__ method, or a(n) __rpow__ method? Not sure.

[copperfield42]

I tested your example with the class

from typing import Union, Any, Iterable, cast, Iterator, overload
from numerary import RealLike, IntegralLike
from itertools import takewhile, count
from decimal import Decimal

class NumberType(RealLike):
    pass

def irange(start:NumberType, stop:NumberType, step:NumberType=1) -> Iterable[NumberType]:   
    count_iter = cast(Iterable[NumberType], count(start,step))
    return takewhile(lambda x: x<stop, count_iter)                 

def toDecimal(a:NumberType) -> Decimal:
    a = cast(Decimal,a)   
    return Decimal(a)

but mypy does not like it all

C:\Users\ThecnoMacVZLA\Desktop\mypy_tests>mypy test3.py
test3.py:9: error: Incompatible default for argument "step" (default has type "int", argument has type "NumberType")
test3.py:14: error: Incompatible types in assignment (expression has type "Decimal", variable has type "NumberType")
test3.py:15: error: Argument 1 to "Decimal" has incompatible type "NumberType"; expected "Union[Decimal, float, str, Tuple[int, Sequence[int], int]]"
Found 3 errors in 1 file (checked 1 source file)

C:\Users\ThecnoMacVZLA\Desktop\mypy_tests>

[copperfield42]

If you have the ability to point me to some sources, I can certainly help debug this.

sure, I can put it my account, give me a minute...

[posita]

Oops! I forgot one important part:

from numerary import RealLike
from typing import Protocol

class NumberType(RealLike, Protocol):
    ...

That should clear up those errors.

[copperfield42]

here is my mess of code https://github.com/copperfield42/poly if you want to look at it XD

[posita]

here is my mess of code https://github.com/copperfield42/poly if you want to look at it XD

You'll need to make sure all requisite methods are implemented. In your current case, RadicalConstante appears to be missing the following methods:

• __hash__ from RealLike
• __rmod__ from SupportsRealOps
• __rpow__ from SupportsComplexPow

Adding those allows for the following:

>>> from numerary import RealLike
>>> import poly
>>> r5 = poly.constante.RadicalConstante(5)
>>> isinstance(r5, RealLike)
True