search

tensorflow / benchmarks

A benchmark framework for Tensorflow
Apache License 2.0
1.14k stars 630 forks source link

perfzero resnet benchmark is outdated #525

Open vladfi1 opened 2 years ago

vladfi1 commented 2 years ago

Due to refactoring in the tf models repo, the estimator benchmarks seem to have moved or no longer exist. For example, running the following command

$ python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.r1.resnet.estimator_benchmark.Resnet50EstimatorBenchmarkSynth.benchmark_graph_1_gpu \
--gcloud_key_file_url=""

produces ModuleNotFoundError: No module named 'official.r1'. I've tried to change the path to something else -- using official.legacy.image_classification.resnet makes some progress -- but I haven't been able to figure it out.

vladfi1 commented 2 years ago

I managed to get it working with the following command:

python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.benchmark.keras_imagenet_benchmark.Resnet50KerasBenchmarkSynth.benchmark_1_gpu \
--gcloud_key_file_url=""

It would be very helpful to have a way of listing available benchmarks!

wangdalin commented 2 years ago

I managed to get it working with the following command:

python3 lib/benchmark.py \
--git_repos="https://github.com/tensorflow/models.git;benchmark" \
--python_path=models \
--benchmark_methods=official.benchmark.keras_imagenet_benchmark.Resnet50KerasBenchmarkSynth.benchmark_1_gpu \
--gcloud_key_file_url=""

It would be very helpful to have a way of listing available benchmarks!

@vladfi1 Thanks! Your command helps me a a lot!

johnnynunez commented 1 year ago

It's not working now