Problems with torch 1.4

vatai commented 2 years ago

Jens reported this, so didn't check myself what's exactly happening. I suspect the following:

old (e.g. 1.4) pytorch didn't have mkldnn backend, only had mkldnn tensor layout. and then newer pytorch had both.
so old torch can only do mkldnn tensor layout (with .to_mkldnn() method), while newer version can also doo backend=mkldnn separately...
what benchmarker messes up in your case with pytorch 1.4 is that it forces backend=mkldnn when you just want the .to_mkldnn() (i.e. --tensor_layout).

Some error messages:

jens@kiev0:~/fake_a64fx_cuda/dep/benchmarker|master! 
⇒  python3 -m benchmarker --mode=training --framework=pytorch --problem=resnet50 --problem_size=32 --batch_size=4 --backend=DNNL --tensor_layout=DNNL
Traceback (most recent call last):
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 114, in <module>
    main()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 70, in main
    result = benchmarker.benchmarker.run(unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/benchmarker.py", line 120, in run
    benchmark = framework_mod.Benchmark(params, unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 70, in __init__
    self.setup_data_and_model()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 96, in setup_data_and_model
    torch.backends.mkldnn.enabled = True
AttributeError: module 'torch.backends' has no attribute 'mkldnn'

jens@kiev0:~/fake_a64fx_cuda/dep/benchmarker|master!
⇒  python3 -m benchmarker --mode=training --framework=pytorch --problem=resnet50 --problem_size=32 --batch_size=4 --tensor_layout=DNNL
Traceback (most recent call last):
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jens/spack/opt/spack/linux-centos7-broadwell/gcc-7.4.0/python-3.7.6-l6ym2ggx7veduyeembjgckv5r4a4fuui/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 114, in <module>
    main()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/__main__.py", line 70, in main
    result = benchmarker.benchmarker.run(unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/benchmarker.py", line 120, in run
    benchmark = framework_mod.Benchmark(params, unknown_args)
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 70, in __init__
    self.setup_data_and_model()
  File "/home/jens/fake_a64fx_cuda/dep/benchmarker/benchmarker/frameworks/do_pytorch.py", line 104, in setup_data_and_model
    torch.backends.mkldnn.enabled = False
AttributeError: module 'torch.backends' has no attribute 'mkldnn'

undertherain commented 2 years ago

this really should be the other way around backend first, layout came alter I'd assume at 1.4. there were none, we can surely do some try catch there

vatai commented 2 years ago

Thanks, I really wanted your input on this. It felt it should be the other way around... can it be that in old torch, the way to set mkldnn was .to_mkldnn instead of setting backend? But maybe I overlooked something

The two commits Jens compared: e20f59d9c2326367eb7ce329b5a251e2ce61d1eb works 5e67fe4a7111d2b40a6b9314290ca1ba809543e7 broken

vatai commented 2 years ago

Ok. It is something funky indeed. Both commits have backend.mkldnn.enable = True etc. but for some reason the version that worked for Jens was not calling the backend.mkldnn.enable = True code, despite having --backend=DNNL set :cry:

undertherain commented 2 years ago

why torch 1.4 btw? it's more ancient than dinosaurs

vatai commented 2 years ago

Yepp!!! :grin:

undertherain / benchmarker

Problems with torch 1.4 #190