search

openzim / python-scraperlib

Collection of Python code to re-use across Python-based scrapers
GNU General Public License v3.0
19 stars 16 forks source link

code is not precommit-clean #161

Closed richterdavid closed 4 months ago

richterdavid commented 4 months ago

Perhaps it's an update in pyright, but I'm seeing errors during precommit in the two files below, on main branch, that are interfering with my work on issue 155. More generally, it seems like this is likely to be an issue throughout this repo, and in all the other repos.

(.venv) david@Heron:~/github/python-scraperlib$ pyright src/zimscraperlib/zim/creator.py
/home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:127:17 - error: Expected 1 more positional argument (reportCallIssue)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:150:20 - error: Expected 1 more positional argument (reportCallIssue)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:175:66 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:197:68 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:225:15 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:273:19 - error: Variable not allowed in type expression (reportInvalidTypeForm)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:332:19 - error: Variable not allowed in type expression (reportInvalidTypeForm)
7 errors, 0 warnings, 0 informations
(.venv) david@Heron:~/github/python-scraperlib$ pyright tests/zim/test_zim_creator.py
/home/david/github/python-scraperlib/tests/zim/test_zim_creator.py
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:13:8 - error: Import "time" could not be resolved (reportMissingImports)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:107:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:108:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:109:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:110:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:114:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:252:80 - error: Argument of type "TemporaryDirectory" cannot be assigned to parameter "ref" of type "Any" in function "__init__"
    "TemporaryDirectory" is incompatible with "Any" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:262:26 - error: Argument of type "Literal[0]" cannot be assigned to parameter "start" of type "Generic" in function "__new__"
    "Literal[0]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:262:29 - error: Argument of type "Literal[4]" cannot be assigned to parameter "stop" of type "Generic" in function "__new__"
    "Literal[4]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:265:40 - error: Argument of type "Literal[-1]" cannot be assigned to parameter "s" of type "slice" in function "__getitem__"
    "Literal[-1]" is incompatible with "slice" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:268:30 - error: Argument of type "Literal[-1]" cannot be assigned to parameter "s" of type "slice" in function "__getitem__"
    "Literal[-1]" is incompatible with "slice" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:269:26 - error: Argument of type "Literal[-1]" cannot be assigned to parameter "s" of type "slice" in function "__getitem__"
    "Literal[-1]" is incompatible with "slice" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:272:44 - error: Argument of type "Literal[-1]" cannot be assigned to parameter "s" of type "slice" in function "__getitem__"
    "Literal[-1]" is incompatible with "slice" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:307:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:324:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:339:15 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:350:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:354:15 - error: Cannot instantiate abstract class "BytesIO"
    "IO.mode" is not implemented (reportAbstractUsage)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:364:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:380:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:430:29 - error: Argument of type "Path" cannot be assigned to parameter "tmp_dir" of type "Any" in function "__init__"
    "Path" is incompatible with "Any" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:435:74 - error: Argument of type "Path" cannot be assigned to parameter "tmp_dir" of type "Any" in function "__init__"
    "Path" is incompatible with "Any" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:448:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:449:18 - error: Expected 0 positional arguments (reportCallIssue)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:537:63 - error: Argument of type "int" cannot be assigned to parameter "Description" of type "str" in function "config_dev_metadata"
    "int" is incompatible with "str" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:540:67 - error: Argument of type "int" cannot be assigned to parameter "LongDescription" of type "str" in function "config_dev_metadata"
    "int" is incompatible with "str" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:545:21 - error: Argument of type "int" cannot be assigned to parameter "Description" of type "str" in function "config_dev_metadata"
    "int" is incompatible with "str" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:645:36 - error: Argument of type "Literal[1969]" cannot be assigned to parameter "year" of type "Generic" in function "__new__"
    "Literal[1969]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:645:42 - error: Argument of type "Literal[12]" cannot be assigned to parameter "month" of type "Generic" in function "__new__"
    "Literal[12]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:645:46 - error: Argument of type "Literal[31]" cannot be assigned to parameter "day" of type "Generic" in function "__new__"
    "Literal[31]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:645:50 - error: Argument of type "Literal[23]" cannot be assigned to parameter "hour" of type "Generic" in function "__new__"
    "Literal[23]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:645:54 - error: Argument of type "Literal[59]" cannot be assigned to parameter "minute" of type "Generic" in function "__new__"
    "Literal[59]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:646:32 - error: Argument of type "Literal[1969]" cannot be assigned to parameter "year" of type "Generic" in function "__new__"
    "Literal[1969]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:646:38 - error: Argument of type "Literal[12]" cannot be assigned to parameter "month" of type "Generic" in function "__new__"
    "Literal[12]" is incompatible with "Generic" (reportArgumentType)
  /home/david/github/python-scraperlib/tests/zim/test_zim_creator.py:646:42 - error: Argument of type "Literal[31]" cannot be assigned to parameter "day" of type "Generic" in function "__new__"
    "Literal[31]" is incompatible with "Generic" (reportArgumentType)
35 errors, 0 warnings, 0 informations
richterdavid commented 4 months ago

Here's the precommit output for a dummy change on my branch for issue 155.

> git -c user.useConfigOnly=true commit --quiet --allow-empty-message --file -
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
black....................................................................Failed
- hook id: black
- files were modified by this hook

reformatted src/zimscraperlib/zim/creator.py

All done! ✨ 🍰 ✨
1 file reformatted.

ruff.....................................................................Passed
pyright (system).........................................................Failed
- hook id: pyright
- exit code: 1

/home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:131:17 - error: Expected 1 more positional argument (reportCallIssue)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:157:21 - error: Cannot instantiate abstract class "BytesIO"
    "IO.mode" is not implemented (reportAbstractUsage)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:166:32 - error: Argument missing for parameter "key" (reportCallIssue)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:180:20 - error: Expected 1 more positional argument (reportCallIssue)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:205:66 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:227:68 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:255:15 - error: Expected type expression but received "_alias" (reportGeneralTypeIssues)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:303:19 - error: Variable not allowed in type expression (reportInvalidTypeForm)
  /home/david/github/python-scraperlib/src/zimscraperlib/zim/creator.py:362:19 - error: Variable not allowed in type expression (reportInvalidTypeForm)
9 errors, 0 warnings, 0 informations
benoit74 commented 4 months ago

These issues are linked to your local setup.

Pyright version is pinned (see https://github.com/openzim/python-scraperlib/blob/7d498319baadba715316c15cf9857ff2f6974a00/pyproject.toml#L43) so rules are fixed, and pyright is invoked in the CI (see https://github.com/openzim/python-scraperlib/blob/7d498319baadba715316c15cf9857ff2f6974a00/.github/workflows/QA.yaml#L34) which is green.