search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
27.81k stars 11.45k forks source link

`AttributeError: type object 'Missed' has no attribute 'demangler_lock'` running opt-viewer #62403

Open nico opened 1 year ago

nico commented 1 year ago

On an M1 Max.

Compiling this file https://github.com/ibogosavljevic/johnysswlab/blob/f677ea00bde2fcc69f48e36d2028619e428c05ef/2022-12-optimizationreport/compiler-optimizations.cpp

Installed deps with pip3 install pygments and pip3 install PyYAML.

% clang -O3 comp-opt.cc -fsave-optimization-record -c -std=c++17

% python3 ~/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py comp-opt.opt.yaml     
Reading YAML files...
Rendering index page...
Rendering HTML files...
    2 of 2multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/optpmap.py", line 25, in _wrapped_func
    return func(argument, filter_)
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 243, in _render_file
    SourceFileRenderer(source_dir, output_dir, filename, no_highlight).render(remarks)
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 184, in render
    self.render_source_lines(self.source_stream, line_remarks)
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 105, in render_source_lines
    self.render_inline_remarks(remark, html_line)
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 108, in render_inline_remarks
    inlining_context = r.DemangledFunctionName
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/optrecord.py", line 146, in DemangledFunctionName
    return self.demangle(self.Function)
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/optrecord.py", line 70, in demangle
    with cls.demangler_lock:
AttributeError: type object 'Missed' has no attribute 'demangler_lock'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 379, in <module>
    main()
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 368, in main
    generate_report(all_remarks,
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 292, in generate_report
    optpmap.pmap(_render_file_bound,
  File "/Users/thakis/src/llvm-project/llvm/tools/opt-viewer/optpmap.py", line 50, in pmap
    result = pool.map(_wrapped_func, func_and_args, *args, **kwargs)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 771, in get
    raise self._value
AttributeError: type object 'Missed' has no attribute 'demangler_lock'

Works fine if I pass -j 1 too.

Also mentioned here https://github.com/androm3da/optviewer-demo/issues/4#issuecomment-718117214 and here https://reviews.llvm.org/D41784.

This is at LLVM e23891a3823e6bf (yesterday's HEAD).

@anemet

neboat commented 9 months ago

I ran into this same issue when running an existing opt-viewer test on macOS 14 (running on an M2 Max) using Python 3.12. In addition, I see a lot of SyntaxWarning messages before the crash.

Here's the crash I'm seeing, with some repeated messages removed:

$ python3 --version
Python 3.12.0
$ python3 llvm/tools/opt-viewer/opt-viewer.py -s llvm/test/tools/opt-viewer/Inputs -o tmp llvm/test/tools/opt-viewer/Inputs/basic/or.yaml --no-highlight --demangler=./build/bin/llvm-cxxfilt 
/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py:138: SyntaxWarning: invalid escape sequence '\S'
  indent = re.sub("\S", " ", indent)
Reading YAML files...
/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py:138: SyntaxWarning: invalid escape sequence '\S'
  indent = re.sub("\S", " ", indent)
(message repeats about a dozen times)
Rendering index page...
Rendering HTML files...
llvm/tools/opt-viewer/opt-viewer.py:138: SyntaxWarning: invalid escape sequence '\S'
  indent = re.sub("\S", " ", indent)
(message repeats about a dozen times)
    2 of 2multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/neboat/.pyenv/versions/3.12.0/lib/python3.12/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/.pyenv/versions/3.12.0/lib/python3.12/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
           ^^^^^^^^^^^^^^^^
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/optpmap.py", line 25, in _wrapped_func
    return func(argument, filter_)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 293, in _render_file
    SourceFileRenderer(source_dir, output_dir, filename, no_highlight).render(remarks)
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 216, in render
    self.render_source_lines(self.source_stream, line_remarks)
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 123, in render_source_lines
    self.render_inline_remarks(remark, html_line)
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 126, in render_inline_remarks
    inlining_context = r.DemangledFunctionName
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/optrecord.py", line 153, in DemangledFunctionName
    return self.demangle(self.Function)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/optrecord.py", line 77, in demangle
    with cls.demangler_lock:
         ^^^^^^^^^^^^^^^^^^
AttributeError: type object 'Missed' has no attribute 'demangler_lock'. Did you mean: 'demangler_proc'?
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 471, in <module>
    main()
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 457, in main
    generate_report(
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/opt-viewer.py", line 376, in generate_report
    optpmap.pmap(
  File "/Users/neboat/Software/llvm-project/llvm/tools/opt-viewer/optpmap.py", line 57, in pmap
    result = pool.map(_wrapped_func, func_and_args, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/.pyenv/versions/3.12.0/lib/python3.12/multiprocessing/pool.py", line 367, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/neboat/.pyenv/versions/3.12.0/lib/python3.12/multiprocessing/pool.py", line 774, in get
    raise self._value
AttributeError: type object 'Missed' has no attribute 'demangler_lock'

I'm not seeing these issues when using Python 3.9 on the same system. In addition, if I run opt-viewer with -j 1, everything works as expected, and I get neither the crash nor the SyntaxWarning messages.

I'm using LLVM 5fdb70be7b0a (HEAD of main this morning).