Problem: mypy found 84 errors in py-utils

[knekrasov]

mypy is a well-known tool for Python source code static validation. Primarily it helps to find the issues with the types but since it tries to analyze the semantics of the sources it can also highlight some other problems (like impossible imports, non-existent keys de-referencing etc).

In py-utils it found 84 errors. At least some of them deserve attention.

Full output:

mypy cortx
cortx/utils/data/access/base_model.py:16: error: Skipping analyzing 'schematics.models': found module but no type hints or library stubs
cortx/utils/validator/commands.py:55: error: Cannot find implementation or library stub for module named 'v_network'
cortx/utils/validator/commands.py:73: error: Cannot find implementation or library stub for module named 'v_consul'
cortx/utils/validator/commands.py:91: error: Cannot find implementation or library stub for module named 'v_storage'
cortx/utils/validator/commands.py:109: error: Cannot find implementation or library stub for module named 'v_salt'
cortx/utils/validator/commands.py:127: error: Cannot find implementation or library stub for module named 'v_bmc'
cortx/utils/validator/commands.py:145: error: Cannot find implementation or library stub for module named 'v_elasticsearch'
cortx/utils/product_features/model.py:19: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/product_features/model.py:22: error: Incompatible types in assignment (expression has type "str", base class "BaseModel" defined the type as "None")
cortx/utils/ha/hac/validation.py:22: error: Skipping analyzing 'networkx': found module but no type hints or library stubs
cortx/utils/ha/hac/compile.py:21: error: Skipping analyzing 'networkx': found module but no type hints or library stubs
cortx/utils/ha/hac/compile.py:22: error: Skipping analyzing 'matplotlib.pyplot': found module but no type hints or library stubs
cortx/utils/ha/hac/compile.py:22: error: Skipping analyzing 'matplotlib': found module but no type hints or library stubs
cortx/utils/log.py:72: error: Argument "address" to "SysLogHandler" has incompatible type "Tuple[str, str]"; expected "Union[Tuple[str, int], str]"
cortx/utils/data/access/queries.py:21: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/data/access/queries.py:148: error: "Data" has no attribute "group_by"
cortx/utils/amqp/rabbitmq/amqp.py:20: error: Cannot find implementation or library stub for module named 'pika'
cortx/utils/amqp/rabbitmq/amqp.py:20: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
cortx/utils/amqp/rabbitmq/amqp.py:24: error: Cannot find implementation or library stub for module named 'pika.exceptions'
cortx/utils/cron.py:19: error: Cannot find implementation or library stub for module named 'crontab'
cortx/utils/security/cipher.py:21: error: Module 'cryptography.fernet' has no attribute 'InvalidSignature'
cortx/utils/schema/conf.py:23: error: Need type annotation for '_payloads' (hint: "_payloads: Dict[<type>, <type>] = ...")
cortx/utils/security/secure_storage.py:16: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/security/secure_storage.py:30: error: Incompatible types in assignment (expression has type "str", base class "BaseModel" defined the type as "None")
cortx/utils/security/secure_storage.py:65: error: Incompatible return value type (got "Optional[Any]", expected "NamedEncryptedBytes")
cortx/utils/message_bus/tcp/kafka/kafka.py:19: error: Skipping analyzing 'confluent_kafka': found module but no type hints or library stubs
cortx/utils/data/db/generic_storage.py:20: error: Skipping analyzing 'schematics.exceptions': found module but no type hints or library stubs
cortx/utils/data/db/generic_storage.py:50: error: "None" has no attribute "keys"
cortx/utils/data/db/generic_storage.py:51: error: "None" has no attribute "keys"
cortx/utils/data/db/generic_storage.py:54: error: "None" has no attribute "keys"
cortx/utils/data/db/generic_storage.py:55: error: "None" has no attribute "keys"
cortx/utils/data/db/generic_storage.py:85: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:91: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:119: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:125: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:130: error: Missing return statement
cortx/utils/data/db/generic_storage.py:139: error: "None" has no attribute "keys"
cortx/utils/data/db/generic_storage.py:146: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:159: error: "GenericDataBase" has no attribute "_model"
cortx/utils/data/db/generic_storage.py:165: error: "GenericDataBase" has no attribute "_model"
cortx/utils/ha/dm/models/node_status.py:18: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/ha/dm/models/node_status.py:22: error: Incompatible types in assignment (expression has type "str", base class "BaseModel" defined the type as "None")
cortx/utils/ha/dm/models/decisiondb.py:18: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/ha/dm/models/decisiondb.py:23: error: Incompatible types in assignment (expression has type "str", base class "BaseModel" defined the type as "None")
cortx/utils/data/db/elasticsearch_db/storage.py:25: error: Skipping analyzing 'elasticsearch_dsl': found module but no type hints or library stubs
cortx/utils/data/db/elasticsearch_db/storage.py:26: error: Skipping analyzing 'elasticsearch_dsl.response': found module but no type hints or library stubs
cortx/utils/data/db/elasticsearch_db/storage.py:27: error: Skipping analyzing 'elasticsearch': found module but no type hints or library stubs
cortx/utils/data/db/elasticsearch_db/storage.py:29: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/data/db/elasticsearch_db/storage.py:32: error: Skipping analyzing 'schematics.exceptions': found module but no type hints or library stubs
cortx/utils/data/db/elasticsearch_db/storage.py:209: error: Need type annotation for '_mapping'
cortx/utils/data/db/elasticsearch_db/storage.py:248: error: Dict entry 0 has incompatible type "str": "str"; expected "str": "Dict[str, Dict[Any, Any]]"
cortx/utils/data/db/elasticsearch_db/storage.py:249: error: Dict entry 1 has incompatible type "str": "str"; expected "str": "Dict[str, Dict[Any, Any]]"
cortx/utils/data/db/elasticsearch_db/storage.py:363: error: "Type[ElasticSearchDB]" has no attribute "pool"
cortx/utils/data/db/elasticsearch_db/storage.py:366: error: Argument 4 to "ElasticSearchDB" has incompatible type "None"; expected "ThreadPoolExecutor"
cortx/utils/data/db/elasticsearch_db/storage.py:412: error: Value of type "None" is not indexable
cortx/utils/data/db/elasticsearch_db/storage.py:412: error: Argument 2 to "next" has incompatible type "None"; expected "str"
cortx/utils/data/db/elasticsearch_db/storage.py:415: error: Value of type "None" is not indexable
cortx/utils/data/db/elasticsearch_db/storage.py:416: error: Incompatible types in assignment (expression has type "Dict[Any, Any]", variable has type "None")
cortx/utils/data/db/elasticsearch_db/storage.py:416: error: "None" has no attribute "items"
cortx/utils/data/db/elasticsearch_db/storage.py:440: error: "None" has no attribute "__iter__" (not iterable)
cortx/utils/data/db/consul_db/storage.py:28: error: Skipping analyzing 'consul.aio': found module but no type hints or library stubs
cortx/utils/data/db/consul_db/storage.py:29: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/data/db/consul_db/storage.py:30: error: Skipping analyzing 'schematics.exceptions': found module but no type hints or library stubs
cortx/utils/data/db/consul_db/storage.py:170: error: Item "str" of "Union[Template, str]" has no attribute "substitute"
cortx/utils/data/db/consul_db/storage.py:172: error: Item "str" of "Union[Template, str]" has no attribute "substitute"
cortx/utils/data/db/consul_db/storage.py:229: error: Incompatible types in assignment (expression has type "Dict[<nothing>, <nothing>]", variable has type "None")
cortx/utils/data/db/consul_db/storage.py:255: error: Argument 4 to "ConsulDB" has incompatible type "Optional[ThreadPoolExecutor]"; expected "ThreadPoolExecutor"
cortx/utils/data/db/consul_db/storage.py:354: error: Incompatible types in assignment (expression has type "Data", variable has type "Query")
cortx/utils/data/db/consul_db/storage.py:363: error: Item "None" of "Optional[AbstractEventLoop]" has no attribute "run_in_executor"
cortx/utils/data/db/consul_db/storage.py:374: error: "Callable[[Any, SortOrder], Any]" has no attribute "field"
cortx/utils/data/db/consul_db/storage.py:381: error: "Callable[[Any, SortOrder], Any]" has no attribute "order"
cortx/utils/data/db/consul_db/storage.py:386: error: Unsupported left operand type for + ("Callable[[int], Any]")
cortx/utils/data/db/consul_db/storage.py:390: error: Unsupported operand types for > ("int" and "Callable[[int], Any]")
cortx/utils/data/db/consul_db/storage.py:413: error: Item "None" of "Optional[AbstractEventLoop]" has no attribute "run_in_executor"
cortx/utils/data/db/consul_db/storage.py:443: error: Item "None" of "Optional[AbstractEventLoop]" has no attribute "run_in_executor"
cortx/utils/data/db/consul_db/storage.py:465: error: Return type "Coroutine[Any, Any, None]" of "delete_by_id" incompatible with return type "Coroutine[Any, Any, bool]" in supertype "GenericDataBase"
cortx/utils/data/db/consul_db/storage.py:465: error: Return type "Coroutine[Any, Any, None]" of "delete_by_id" incompatible with return type "Coroutine[Any, Any, bool]" in supertype "IDataBase"
cortx/utils/data/db/consul_db/storage.py:489: error: Item "None" of "Optional[AbstractEventLoop]" has no attribute "run_in_executor"
cortx/utils/data/db/db_provider.py:23: error: Skipping analyzing 'schematics': found module but no type hints or library stubs
cortx/utils/data/db/db_provider.py:24: error: Skipping analyzing 'schematics.types': found module but no type hints or library stubs
cortx/utils/data/db/db_provider.py:101: error: Function "asyncio.coroutines.coroutine" is not valid as a type
cortx/utils/data/db/db_provider.py:101: note: Perhaps you need "Callable[...]" or a callback protocol?
cortx/utils/data/db/db_provider.py:153: error: Function "asyncio.coroutines.coroutine" is not valid as a type
cortx/utils/data/db/db_provider.py:153: note: Perhaps you need "Callable[...]" or a callback protocol?
cortx/utils/data/db/db_provider.py:186: error: Need type annotation for '_cached_async_decorators' (hint: "_cached_async_decorators: Dict[<type>, <type>] = ...")
cortx/utils/data/db/db_provider.py:199: error: Argument 1 to "issubclass" has incompatible type "object"; expected "type"
cortx/utils/ha/dm/decision_monitor.py:75: error: On Python 3 '{}'.format(b'abc') produces "b'abc'"; use !r if this is a desired behavior
Found 84 errors in 20 files (checked 77 source files)

[stale[bot]]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @s-arya @sachinpunadikar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[cortx-admin]

Donald R Bloyer commented in Jira Server:

[~932497] Have we done any maintenance of pyutils since August that may have resolved some of these.  Do we have a sense of what these may impact?

 

[rahul-tripathi-git]

hi @knekrasov

In py-utils it found 84 errors. At least some of them deserve attention.

Can you please confirm if you installed cortx-py-utils rpm before trying mypy on the repo?

[stale[bot]]

This issue/pull request has been marked as needs attention as it has been left pending without new activity for 4 days. Tagging @s-arya @sachinpunadikar for appropriate assignment. Sorry for the delay & Thank you for contributing to CORTX. We will get back to you as soon as possible.

[hessio]

@knekrasov is no longer at Seagate and this issue has become stale.

[cortx-admin]

Rahul Tripathi commented in Jira Server:

NA

[cortx-admin]

Rahul Tripathi commented in Jira Server:

Github issue got fixed, closing this Bug

[cortx-admin]

Rahul Tripathi commented in Jira Server:

Closing this issue

[cortx-admin]

Rahul Tripathi commented in Jira Server:

Rejected bug