`pylint` is confused about optional `prog`?

[pfactum]

Hello.

Not sure if it is a pylint issue, or there's an issue with drgn type annotations, so please feel free to bounce me off in case it's the former.

Given this code:

#!/usr/bin/env python

"""Test pylint drgn woe."""

from drgn import Program

from drgn.helpers.linux.fs import for_each_mount

prog = Program()
for _ in for_each_mount():
    pass

running pylint on it results in:

drgn_pylint_woe.py:10:9: E1120: No value for argument 'prog' in function call (no-value-for-parameter)

I thought it was optional? This is what documentation says:

But lets provide it anyway:

--- drgn_pylint_woe.py~ 2024-10-10 10:41:33.476885212 +0200
+++ drgn_pylint_woe.py  2024-10-10 10:41:40.120320857 +0200
@@ -7,5 +7,5 @@
 from drgn.helpers.linux.fs import for_each_mount

 prog = Program()
-for _ in for_each_mount():
+for _ in for_each_mount(prog):
     pass

Now, this results in:

drgn_pylint_woe.py:10:9: E1120: No value for argument 'ns' in function call (no-value-for-parameter)

OK, now checking how for_each_mount() is declared:

@takes_object_or_program_or_default
def for_each_mount(
    prog: Program,
    ns: Optional[Object],
    *,
    src: Optional[Path] = None,
    dst: Optional[Path] = None,
    fstype: Optional[Union[str, bytes]] = None,
) -> Iterator[Object]:

Lets provide ns too:

--- drgn_pylint_woe.py~ 2024-10-10 10:45:03.466749338 +0200
+++ drgn_pylint_woe.py  2024-10-10 10:45:18.076969365 +0200
@@ -7,5 +7,5 @@
 from drgn.helpers.linux.fs import for_each_mount

 prog = Program()
-for _ in for_each_mount(prog):
+for _ in for_each_mount(prog, prog["init_task"].nsproxy.mnt_ns):
     pass

Now pylint is silent.

Am I missing something, or there's a problem with how for_each_mount() is declared? Or this is just pylint being unable to parse this complexity?

Thank you.

[pfactum]

And there's a problem with the latest variant of calling for_each_mount(), but for mypy:

drgn_pylint_woe.py:10: error: No overload variant of "__call__" of "TakesObjectOrProgramOrDefault" matches argument types "Any", "Any"  [call-overload]
drgn_pylint_woe.py:10: note: Possible overload variants:
drgn_pylint_woe.py:10: note:     def __call__(prog: Any, *, src: Any | None = ..., dst: Any | None = ..., fstype: str | bytes | None = ...) -> Iterator[Any]
drgn_pylint_woe.py:10: note:     def __call__(Any, /, *, src: Any | None = ..., dst: Any | None = ..., fstype: str | bytes | None = ...) -> Iterator[Any]
drgn_pylint_woe.py:10: note:     def __call__(*, src: Any | None = ..., dst: Any | None = ..., fstype: str | bytes | None = ...) -> Iterator[Any]
Found 1 error in 1 file (checked 1 source file)

On contrary, mypy doesn't complain about missing prog.

[osandov]

The optional prog thing is pretty complicated. The helper implementations always take prog, then the @takes_program_or_default and @takes_object_or_program_or_default decorators wrap the implementation with a different signature that makes prog optional. Mypy understands that, but it seems like pylint doesn't.

for_each_mount(prog) and for_each_mount(prog["init_task"].nsproxy.mnt_ns) are valid, but for_each_mount(prog, prog["init_task"].nsproxy.mnt_ns) is not. Again, I'm pretty sure Mypy gets this right, so it seems like a pylint limitation.

P.S. If I had decided to make prog optional from the beginning, I would've made it a keyword-only argument and avoided all of this. But we're stuck with it for the sake of backwards-compatibility.

[pfactum]

Thank you for the answer. I'll go for sprinkling the ignore hints for pylint then or just pass prog. Please close this report.

But we're stuck with it for the sake of backwards-compatibility.

drgn is still at 0.0.x versions, should this matter?

[osandov]

drgn is still at 0.0.x versions, should this matter?

There's enough code out there depending on drgn helpers that I try not to change the helper APIs. I've hesitated on bumping to 1.0.0 since I'm still planning on breaking/changing some of the more advanced program APIs (e.g. for #332).