47 undefined variable errors with Pylint

[vandanavk]

Description

Running Pylint on incubator-mxnet with undefined-variable enabled, results in 52 errors. This could lead to NameError on execution.

Steps to reproduce

What to do:
1. Edit the enable section of tests/ci_build/pylintrc to include undefined-variable
2. Run
find . -name "*.py" -exec pylint --rcfile=${ROOT_DIR}/tests/ci_build/pylintrc --ignore-patterns=".*\.so$$,.*\.dll$$,.*\.dylib$$" --msg-template='{path}({line}): [{msg_id} {obj}] {msg}' -E '{}' +
from the root directory on your machine.
3. Look for Errors with code E0602 or having the text Undefined variable

Package used (Python/R/Scala/Julia): I'm using Python2

Build info (Required if built from source)

MXNet commit hash: f5b95b090815e879b57dca233604dcb3f1df967a

Error Message:

Code/module affected	Undefined variable
./docs/mxdoc.py:75:16:	root_path
./example/autoencoder/model.py:25:8:	ModuleNotFoundError
./example/deep-embedded-clustering/model.py:25:8:	ModuleNotFoundError
./example/fcn-xs/image_segmentaion.py:100:60:	ctx
./example/fcn-xs/image_segmentaion.py:103:60:	ctx
./example/fcn-xs/image_segmentaion.py:104:26:	ctx
./example/neural-style/end_to_end/model_vgg19.py:94:17:	out
./example/neural-style/end_to_end/model_vgg19.py:105:16:	out
./example/profiler/profiler_executor.py:89:15:	search_plan
./example/profiler/profiler_executor.py:89:37:	data_shapes
./example/reinforcement-learning/a3c/a3c.py:206:16:	robo_data
./example/sparse/factorization_machine/metric.py:111:22:	label_zero_num
./example/sparse/factorization_machine/metric.py:111:39:	label_one_num
./example/ssd/dataset/pycocotools/coco.py:266:41:	m
./example/ssd/dataset/pycocotools/coco.py:266:53:	m
./example/ssd/dataset/pycocotools/coco.py:273:52:	m
./example/ssd/dataset/pycocotools/coco.py:435:16:	m
./example/ssd/symbol/common.py:209:23:	start_offset
./example/ssd/symbol/common.py:210:46:	start_offset
./python/mxnet/initializer.py:700:20:	_INITIALIZER_REGISTRY
./tools/flakiness_checker.py:69:11:	FileNotFoundError
./example/gluon/image_classification.py:143:31	get_cifar10_iterator
./example/gluon/image_classification.py:152:31	get_imagenet_iterator
./example/gluon/image_classification.py:155:31	get_caltech101_iterator
./example/gluon/image_classification.py:158:31	dummy_iterator
./example/reinforcement-learning/dqn/atari_game.py:37:10	get_numpy_rng
./example/reinforcement-learning/dqn/atari_game.py:72:19	get_numpy_rng
./example/reinforcement-learning/dqn/base.py:98:35	get_bucket_key
./example/reinforcement-learning/dqn/base.py:161:28	save_params
./example/reinforcement-learning/dqn/base.py:164:27	save_misc
./example/reinforcement-learning/dqn/base.py:170:49	load_params
./example/reinforcement-learning/dqn/dqn_demo.py:33:27	sys
./example/reinforcement-learning/dqn/dqn_demo.py:39:10	get_numpy_rng
./example/reinforcement-learning/dqn/dqn_demo.py:90:10	parse_ctx
./example/reinforcement-learning/dqn/dqn_demo.py:138:16	time
./example/reinforcement-learning/dqn/dqn_demo.py:147:33	time
./example/reinforcement-learning/dqn/dqn_demo.py:221:31	time
./example/reinforcement-learning/dqn/dqn_demo.py:235:14	time
./example/reinforcement-learning/dqn/dqn_run_test.py	get_numpy_rng
./example/reinforcement-learning/dqn/replay_memory.py:31:19	get_numpy_rng
./tests/python/mkl/test_mkldnn.py:264:4	test_mkldnn_install
./example/rnn/word_lm/train.py:91:32	rnn
./example/rnn/word_lm/train.py:93:11	softmax_ce_loss
./example/sparse/factorization_machine/train.py:102:12	factorization_machine_model
./example/sparse/wide_deep/train.py:75:40	get_uci_adult
./example/sparse/wide_deep/train.py:76:34	get_uci_adult
./example/sparse/wide_deep/train.py:78:12	wide_deep_model

What have you tried to solve it?

1. Solving the errors one by one

[cclauss]

Thanks. I am learning more about pylint from your approach here.

l’m using Python2

Have you tried doing python3 -m pylint ... ?

basestring, long, raw_input, unicode, xrange, etc. are defined names in Python 2 but not in Python 3.

[vandanavk]

@cclauss I tried Python3 too. The only difference I saw was that the undefined errors related to exceptions didn't occur.

[vandanavk]

Fixes for 30 errors have been merged.

The ones remaining are:

Code/module affected	Undefined variable
./example/deep-embedded-clustering/model.py:25:8:	ModuleNotFoundError
./example/neural-style/end_to_end/model_vgg19.py:94:17:	out
./example/neural-style/end_to_end/model_vgg19.py:105:16:	out
./example/profiler/profiler_executor.py:89:15:	search_plan
./example/profiler/profiler_executor.py:89:37:	data_shapes
./example/reinforcement-learning/a3c/a3c.py:206:16:	robo_data
./example/sparse/factorization_machine/metric.py:111:22:	label_zero_num
./example/sparse/factorization_machine/metric.py:111:39:	label_one_num

./docs/mxdoc.py:75:16: | root_path | PR https://github.com/apache/incubator-mxnet/pull/11990 ./tools/flakiness_checker.py:69:11: | FileNotFoundError | PR https://github.com/apache/incubator-mxnet/pull/11991 Plus PR https://github.com/apache/incubator-mxnet/pull/11982

[cclauss]

flake8 (on Python 3) moved from finding 22 undefined names down to 19. https://github.com/apache/incubator-mxnet/issues/8270#issuecomment-411455720

[cclauss]

ModuleNotFoundError was added in Python 3.6 so this will be raised if pylint (or flake8) is running on Python <3.6. Change ModuleNotFoundError to ImportError to be compatible with all currently supported versions of Python.

[vandanavk]

All the errors in this list have been fixed. Closing this bug.

[cclauss]

Pylint finds no undefined names when running on Python 2 and when running on Python 3?

[vandanavk]

The more recent lint scans that I've been doing are with Python3 - didn't find any new errors.

[cclauss]

See #12200 !!

[cclauss]

PyLint is not currently configured to find issues with Python files in ./amalgamation, ./tests, etc...