Getting Error "ModuleNotFoundError: No module named 'official.resnet'" in PerfZero benchmark

[deepeshguru]

FIrst I create docker image using this: python3 benchmarks/perfzero/lib/setup.py --dockerfile_path=docker/Dockerfile_ubuntu_1804_tf_v2 --tensorflow_pip_spec=tensorflow==2.2.0

export ROOT_DATA_DIR=/data

docker run -it --rm -v $(pwd):/workspace -v $ROOT_DATA_DIR:$ROOT_DATA_DIR perfzero/tensorflow \ python3 /workspace/benchmarks/perfzero/lib/benchmark.py \ --gcloud_key_file_url="" \ --git_repos="https://github.com/tensorflow/models.git" \ --python_path=models \ --benchmark_methods=official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu \ --root_data_dir=$ROOT_DATA_DIR

Output: 2020-05-29 10:58:41,000 INFO: Created directory /workspace/benchmarks/perfzero/workspace/output/2020-05-29-10-58-40-999344 2020-05-29 10:58:41,003 ERROR: Benchmark execution for official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu failed due to error: Traceback (most recent call last): File "/workspace/benchmarks/perfzero/lib/perfzero/benchmark_method_runner.py", line 100, in _run_internal constructor_args=constructor_args) File "/workspace/benchmarks/perfzero/lib/perfzero/utils.py", line 452, in instantiate_benchmark_class module = importlib.import_module(module_import_path) File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 941, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'official.resnet'

[lindong28]

Hey @deepeshguru, sorry for the inconvenience. The example benchmark_methods was outdated after the file https://github.com/tensorflow/models/blob/master/official/r1/resnet/estimator_benchmark.py is moved in the repository github.com/tensorflow/models.

I have updated README to be consistent with the latest file paths in the tensorflow/models repository. Can you try again with --benchmark_methods=official.r1.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu?

[romanzac]

Hi @lindong28, I am still having same problem like @deepeshguru...

Thanks for your time.

root@90bb61532d68:/workspace# python3 /workspace/perfzero/lib/benchmark.py --gcloud_key_file_url="" --benchmark_methods=official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu --root_data_dir=/data 2020-06-03 02:49:16,310 INFO: The following benchmark methods will be executed: ['official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu'] 2020-06-03 02:49:16,310 INFO: The following benchmark methods will be executed: ['official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu'] 2020-06-03 02:49:16,323 INFO: Created directory /workspace/perfzero/workspace/output/2020-06-03-02-49-16-322230 2020-06-03 02:49:16,323 INFO: Created directory /workspace/perfzero/workspace/output/2020-06-03-02-49-16-322230 2020-06-03 02:49:16,328 ERROR: Benchmark execution for official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu failed due to error: Traceback (most recent call last): File "/workspace/perfzero/lib/perfzero/benchmark_method_runner.py", line 100, in _run_internal constructor_args=constructor_args) File "/workspace/perfzero/lib/perfzero/utils.py", line 452, in instantiate_benchmark_class module = importlib.import_module(module_import_path) File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 941, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 953, in _find_and_load_unlocked ModuleNotFoundError: No module named 'official.resnet'

[lindong28]

Hi @romanzac, it looks like you are using benchmark_methods= official.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu when running the PerfZero. The root cause of the issue is that, official.resnet... no longer points to a valid module in https://github.com/tensorflow/models due to file movement in that repository.

So the fix is to change the path to point to the correct location. Can you try benchmark_methods= official.r1.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu?

[romanzac]

@lindong28 it works for me, many thanks.

[lindong28]

Sounds great. Thank you for confirming that it works :)

[vladfi1]

It appears that more movements have happened and official.r1 no longer exists.