Open ncoghlan opened 1 month ago
I/O types are a mess, which is why we're moving more into tight protocols for arguments as used in hashlib
.
I think your analysis is correct. In fact, file_digest
can't handle None
return values:
On the other hand it seems that FileIO.readinto
can't return None
:
At the moment, FileIO
inherits readinto
from RawIOBase
. I think we just need to override it in FileIO
with a more precise return type.
On the other hand it seems that
FileIO.readinto
can't returnNone
:
Doesn't it return None
in this code path here? https://github.com/python/cpython/blob/2a5d1eb7073179a13159bce937afdbe240432e7d/Modules/_io/fileio.c#L678
I blame my the heat and my tiredness for overlooking this quite obvious code path ... That said, I'd be interested in which situation FileIO
works with non-blocking I/O. One case I can see is here:
Another is probably if an opened file descriptor has the non-blocking flag set.
I see two possible solutions: Make FileIO.readinto()
return int | MaybeNone
as the situations where this happens are a bit esoteric. That said, I actually think that this uncovered a real bug in file_digest()
as that doesn't handle the None
case correctly when it should.
In fact, file_digest
is explicitly documented as taking a SocketIO
object. And that can very much return None
:
https://github.com/python/cpython/blob/2a5d1eb7073179a13159bce937afdbe240432e7d/Lib/socket.py#L713
Using
hashlib.file_digest
with a file object obtained viapathlib.Path.open
failed to typecheck with a protocol compatibility error:I'm reasonably sure this is a
typeshed
issue rather than amypy
issue.My first thought was that this looks like an internal inconsistency in
typeshed
, as_FileDigestFileObj
is declared specifically withbytearray
rather than the more generalWriteableBuffer
(hence filing the issue here).However, my second thought was that this signature declaration is presumably as it is because
file_digest
will specifically pass abytearray
instance toreadinto
, andbytearray
is compatible withBuffer
, somypy
shouldn't be complaining about that. That means the conflict is presumably on the return type rather than on parameter types.The declared type signature nominally indicates that
file_digest
can't cope withreadinto
returningNone
, andmypy
is correctly flagging that as inconsistent with the waytypeshed
declares the file and socket IO types (allowing them to returnNone
here).Searching the typeshed code suggests this particular inconsistency exists across several different IO consumer type declarations: https://github.com/search?q=repo%3Apython%2Ftypeshed%20readinto&type=code