[clang-format] Crash with Whitesmiths and labels in top-level block

[Quuxplusone]

Bugzilla Link	PR50663
Status	RESOLVED FIXED
Importance	P normal
Reported by	Oliver Old (oliver.old@outlook.com)
Reported on	2021-06-10 14:34:29 -0700
Last modified on	2021-06-16 00:04:17 -0700
Version	12.0
Hardware	PC All
CC	djasper@google.com, klimek@google.com, llvm-bugs@lists.llvm.org, mydeveloperday@gmail.com, oliver.old@outlook.com, tstellar@redhat.com
Fixed by commit(s)	rGf7f9f94b2e2b, rGc7d7ace46258
Attachments	clang-format-repro.zip (355 bytes, application/x-zip-compressed)
Blocks	PR49317
Blocked by
See also

Created attachment 24941
Zipped goto.cpp and switch.cpp files

Formatting a file with a label within a top-level block causes clang-format to
hang for an extended period of time and use a lot of memory. It then proceeds
to crash:

PS C:\Users\olive\clang-format-repro> clang-format.exe --
style='{BreakBeforeBraces: Whitesmiths}' goto.cpp
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash
backtrace.
Stack dump:
0.      Program arguments: "C:\\Program Files\\LLVM\\bin\\clang-format.exe" "--
style={BreakBeforeBraces: Whitesmiths}" goto.cpp
PS C:\Users\olive\clang-format-repro> cat goto.cpp
int main() {
test:
    return 0;
}

No actual backtrace is printed.

I have prepared test cases with both a simple function and a switch statement
and both lead to a crash.

My suspicion is that this was caused by the patch that tried to fix case labels
with Whitesmiths [1]. This bug does not occur with version 11.0 of clang-format.

[1] https://reviews.llvm.org/D94500

[Quuxplusone]

Attached clang-format-repro.zip (355 bytes, application/x-zip-compressed): Zipped goto.cpp and switch.cpp files

[Quuxplusone]

I tested this with 11 and 12 and can confirm it does indeed hang,

I tested this with my latest build and it doesn't, but I build debug so its possible it doesn't manifest itself.

[Quuxplusone]

I tested Debug and Release builds of the current tip of the main (Currently 13) and I cannot reproduce.

I checked out the 12.0.1 branches and tested both debug and release

12.0.1 does indeed still crash (Debug and Release), my guess its more a crash caused by out of memory as the process rapid grows to many GB

Just to clarify my previous comment 11 GA was fine 12 GA was not.

I'm not sure if this was somehow a known issue, I don't remember it between 12 and 13 especially for clang-format.

[Quuxplusone]

Did eventually get a stack from 12.0.1

/buildareas/clang/build_ninja/bin/clang-format.exe  test1.cpp
Assertion failed: Index < Length && "Invalid index!", file
C:\cygwin64\buildareas\clang\llvm-project\llvm\include\llvm/ADT/ArrayRef.h,
line 248
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash
backtrace.
Stack dump:
0.      Program arguments:
C:\\cygwin64\\buildareas\\clang\\build_ninja\\bin\\clang-format.exe test1.cpp
0x00007FF7B16DC89C (0x000000C600000016 0x0000000000000000 0x00007FF7B15A0C85
0x0000000000000000), HandleAbort() + 0xC bytes(s),
C:\cygwin64\buildareas\clang\llvm-project\
llvm\lib\Support\Windows\Signals.inc, line 408
0x00007FFF473CBC31 (0x00007FFF00000016 0x000000C617389A70 0x0000000801000006
0xFFFFFF0100000120), raise() + 0x441 bytes(s)
0x00007FFF473CD889 (0x000000C617389A70 0x0000000000000240 0x00007FFF474CDA70
0x00007FF7B1FD8020), abort() + 0x39 bytes(s)
0x00007FFF473D34B5 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x00007FFF000000F8
0x0000000000000000), _get_wide_winmain_command_line() + 0x2515 bytes(s)
0x00007FFF473D3027 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x00000248000000F8
0x0000024C9D4B0080), _get_wide_winmain_command_line() + 0x2087 bytes(s)
0x00007FFF473D1091 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x0000024C000000F8
0x00007FF7B18F7E99), _get_wide_winmain_command_line() + 0xF1 bytes(s)
0x00007FFF473D3A1F (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0xCCCCCCCC000000F8
0xCCCCCCCCCCCCCCCC), _wassert() + 0x2F bytes(s)
0x00007FF7B18F7E99 (0x000000C61738A0A0 0x00000000FFFFFFFF 0xCCCCCCCCCCCCCCCC
0xCCCCCCCCCCCCCCCC), llvm::ArrayRef<int>::operator[]() + 0x59 bytes(s),
C:\cygwin64\buildarea
s\clang\llvm-project\llvm\include\llvm\ADT\ArrayRef.h, line 248 + 0x36 byte(s)
0x00007FF7B18FE3E9 (0x000000C61738A210 0x000000C61738A0A0 0xCCCCCCCCFFFFFFFF
0xCCCCCCCCCCCCCCCC), clang::format::`anonymous
namespace'::LevelIndentTracker::getIndent() +
0x39 bytes(s), C:\cygwin64\buildareas\clang\llvm-
project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 115 + 0x10 byte(s)
0x00007FF7B18FF01A (0x000000C61738A210 0x000002455CA2FC80 0xCCCCCCCCCCCCCCCC
0xCCCCCCCCCCCCCCCC), clang::format::`anonymous
namespace'::LevelIndentTracker::nextLine() + 0
x11A bytes(s), C:\cygwin64\buildareas\clang\llvm-
project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 63 + 0x4A byte(s)
0x00007FF7B18FE642 (0x000000C61738A168 0x000002455CA2FD00 0x000000C61738A210
0x000000C61738C408), clang::format::`anonymous
namespace'::LineJoiner::getNextMergedLine() +
0x62 bytes(s), C:\cygwin64\buildareas\clang\llvm-
project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 178
0x00007FF7B18EEBB0 (0x000000C61738B300 0x000000C61738C408 0x000000C61738A400
0x000000C600000000), clang::format::UnwrappedLineFormatter::format() + 0x550
bytes(s), C:\cyg
win64\buildareas\clang\llvm-
project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 1148 + 0x1A byte(s)
0x00007FF7B1874229 (0x000000C61738C780 0x000000C61738C4E8 0x000000C61738C4B8
0x000000C61738C408), clang::format::`anonymous namespace'::Formatter::analyze()
+ 0x379 bytes
(s), C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\Format.cpp,
line 1584 + 0x105 byte(s)
0x00007FF7B18D1B04 (0x000000C61738C780 0x000000C61738D8C8 0x000000C61738D3A0
0x000000C61738E188), clang::format::TokenAnalyzer::process() + 0x4F4 bytes(s),
C:\cygwin64\bu
ildareas\clang\llvm-project\clang\lib\Format\TokenAnalyzer.cpp, line 93
0x00007FF7B186B24C (0x000000C61738D760 0x000000C61738D8C8 0x000002455CA36F00
0xCCCCCCCCCCCCCCCC), <lambda_be157eee42e494a9ed92b4bd992e61c3>::operator()() +
0x7C bytes(s),
 C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\Format.cpp, line 2729 + 0x3C byte(s)
0x00007FF7B185038C (0x000000C61738D8C8 0x000000C61738D760 0x000002455CA36F00
0xCCCCCCCCCCCCCCCC), std::invoke<<lambda_be157eee42e494a9ed92b4bd992e61c3>
&,clang::format::E
nvironment const &>() + 0x3C bytes(s), C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\type_traits, line 1596
+ 0x14 byte
(s)
0x00007FF7B183F04C (0x000000C61738D8C8 0x000000C61738D760 0x000002455CA36F00
0xCCCCCCCCCCCCCCCC),
std::_Invoker_ret<std::pair<clang::tooling::Replacements,unsigned int>,0
>::_Call<<lambda_be157eee42e494a9ed92b4bd992e61c3> &,clang::format::Environment
const &>() + 0x3C bytes(s), C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enterprise
\VC\Tools\MSVC\14.28.29910\include\functional, line 753 + 0x14 byte(s)
0x00007FF7B186EB6B (0x000000C61738D758 0x000000C61738D8C8 0x000002455CA36F00
0xCCCCCCCCCCCCCCCC),
std::_Func_impl_no_alloc<<lambda_be157eee42e494a9ed92b4bd992e61c3>,std::
pair<clang::tooling::Replacements,unsigned int>,clang::format::Environment
const &>::_Do_call() + 0x4B bytes(s), C:\Program Files (x86)\Microsoft Visual
Studio\2019\Enter
prise\VC\Tools\MSVC\14.28.29910\include\functional, line 938 + 0x23 byte(s)
0x00007FF7B186B5C9 (0x000000C61738D758 0x000000C61738D8C8 0x000002455CA36F00
0x000000C61738DA40),
std::_Func_class<std::pair<clang::tooling::Replacements,unsigned int>,cl
ang::format::Environment const &>::operator()() + 0x79 bytes(s), C:\Program
Files (x86)\Microsoft Visual
Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\function
al, line 986 + 0x2C byte(s)
0x00007FF7B18316A9 (0x000000C61738DB50 0x000000C61738DD78 0x000000C61738DB70
0x000000C61738DB80), clang::format::internal::reformat() + 0x499 bytes(s),
C:\cygwin64\builda
reas\clang\llvm-project\clang\lib\Format\Format.cpp, line 2746
0x00007FF7B182F5F0 (0x000000C61738E1A8 0x000000C61738DD78 0x000000C61738F370
0x000000C61738F380), clang::format::reformat() + 0xD0 bytes(s),
C:\cygwin64\buildareas\clang\
llvm-project\clang\lib\Format\Format.cpp, line 2769 + 0x9C byte(s)
0x00007FF7B15F96A6 (0x000000C61738F6B0 0x000002455CA2BA10 0x000000C61738F690
0x0000000000000000), clang::format::format() + 0xC16 bytes(s),
C:\cygwin64\buildareas\clang\l
lvm-project\clang\tools\clang-format\ClangFormat.cpp, line 420
0x00007FF7B15FA7C1 (0x0000000000000002 0x000002455CA21860 0x0000000000000000
0x00007FF7B1ABB8ED), main() + 0x421 bytes(s), C:\cygwin64\buildareas\clang\llvm-
project\clang
\tools\clang-format\ClangFormat.cpp, line 535 + 0x56 byte(s)
0x00007FF7B1ABC3E9 (0x00007FF7B1FD1000 0x00007FF7B1FD16F0 0x0000000000000000
0x0000000000000000), invoke_main() + 0x39 bytes(s),
D:\a01\_work\26\s\src\vctools\crt\vcstart
up\src\startup\exe_common.inl, line 79
0x00007FF7B1ABC2CE (0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000), __scrt_common_main_seh() + 0x12E bytes(s),
D:\a01\_work\26\s\src\vctools
\crt\vcstartup\src\startup\exe_common.inl, line 288 + 0x5 byte(s)
0x00007FF7B1ABC18E (0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000), __scrt_common_main() + 0xE bytes(s),
D:\a01\_work\26\s\src\vctools\crt\v
cstartup\src\startup\exe_common.inl, line 331
0x00007FF7B1ABC47E (0x000000C61697C000 0x0000000000000000 0x0000000000000000
0x0000000000000000), mainCRTStartup() + 0xE bytes(s),
D:\a01\_work\26\s\src\vctools\crt\vcsta
rtup\src\startup\exe_main.cpp, line 17
0x00007FFF90687034 (0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000), BaseThreadInitThunk() + 0x14 bytes(s)
0x00007FFF91942651 (0x0000000000000000 0x0000000000000000 0x0000000000000000
0x0000000000000000), RtlUserThreadStart() + 0x21 bytes(s)
make: *** [Makefile:24: all] Error 3

[Quuxplusone]

I applied https://reviews.llvm.org/D94500 to the 12.0.1 branch and this fixes your issue.

Can we consider this issue fixed? for LLVM 13.0

I would prefer that @timwoj fixed this in the branch if that's what needs to be done before 13 is released

[Quuxplusone]

Would be nice to see an early fix since this is obviously broken and shouldn't crash the formatter. As long as this issue remains, I will not be able to adopt clang-format for my code.

[Quuxplusone]

The fix does not apply cleanly, could someone backport this and push a branch to their local github fork?

[Quuxplusone]

Tom am I allowed to simply commit/push it directly into the branch?

do I need to go back round the Phabricator review cycle? if the changes to get into the branch are minimal?

[Quuxplusone]

I should have mentioned, I don't use a fork.

[Quuxplusone]

(In reply to MyDeveloperDay from comment #7)
> Tom am I allowed to simply commit/push it directly into the branch?
>

Yes, if you can backport the patch cleanly, go ahead and push it.  Just make
sure to use git cherry-pick -x so that the commit in main is referenced in the
commit log and make sure the lit tests pass.

[Quuxplusone]

Ok its pushed to the 12.x branch (cherry picked -x), for the life of me I hope I've done it correctly, the unit tests all pass.

https://github.com/llvm/llvm-project/commit/c7d7ace46258b04aa4b5df08952bfebc6fc4ce94

If I've messed anything up, I apologize in advance, but I guess we all have to do something scary the first time

Thanks for your help.

[Quuxplusone]

Is the a branch build bot?

[Quuxplusone]

(In reply to MyDeveloperDay from comment #11)
> Is the a branch build bot?

Yes, there are branch buildbots.  They report status to github, so if you click
on the commit in the web interface, you should be able to see the status.

[Quuxplusone]

Merged: c7d7ace46258

[Quuxplusone]

Thanks Tom for helping me out (I got my green tick in the end, took a while...)

Thank you for everything you do.