undertherain / benchmarker

modular framework for [not only] deep learning performance benchmarking
http://blackbird.pw/performance
Mozilla Public License 2.0
9 stars 5 forks source link

Unneeded torch import #181

Closed vatai closed 3 years ago

vatai commented 3 years ago

cblas (and other non python) gemm and friends were importing torch. As a (very undesirable) result you needed to have pytorch installed, even if you wanted to measure some non-pytorch, C/C++ gemm benchmark. Hopefully this helps with the issue.

The original error message was:

     Traceback (most recent call last):
       File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
         "__main__", mod_spec)
       File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
         exec(code, run_globals)
       File "/home/a/a0175786/benchmarker/benchmarker/__main__.py", line 95, in <module>
         main()
       File "/home/a/a0175786/benchmarker/benchmarker/__main__.py", line 53, in main
         result = benchmarker.benchmarker.run(unknown_args)
       File "/home/a/a0175786/benchmarker/benchmarker/benchmarker.py", line 120, in run
         benchmark = framework_mod.Benchmark(params, unknown_args)
       File "/home/a/a0175786/benchmarker/benchmarker/frameworks/do_cblas.py", line 10, in __init__
         super().__init__(params, remaining_args)
       File "/home/a/a0175786/benchmarker/benchmarker/frameworks/i_gemm.py", line 26, in __init__
         self.data = self.load_data()
       File "/home/a/a0175786/benchmarker/benchmarker/frameworks/i_benchmark.py", line 16, in load_data
         "benchmarker.kernels." + params["problem"]["name"] + ".data"
       File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
         return _bootstrap._gcd_import(name[level:], package, level)
       File "<frozen importlib._bootstrap>", line 994, in _gcd_import
       File "<frozen importlib._bootstrap>", line 971, in _find_and_load
       File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
       File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
       File "<frozen importlib._bootstrap_external>", line 678, in exec_module
       File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
       File "/home/a/a0175786/benchmarker/benchmarker/kernels/gemm/data.py", line 1, in <module>
         import torch
     ModuleNotFoundError: No module named 'torch'
undertherain commented 3 years ago

mmmmm now torch kernell spends a bi of time converting tensors while in cblas they were simply ignored

the current state is clearly not good, but we need some smarter soluon like converting or generating data depending on framework bun not in call method

vatai commented 3 years ago

This is why we do reviews... I might have a solution for that...