search

microsoft / clrmd

Microsoft.Diagnostics.Runtime is a set of APIs for introspecting processes and dumps.
MIT License
1.05k stars 255 forks source link

OverflowException in ElfVirtualAddressSpace.Read #849

Closed tomasdeml closed 4 years ago

tomasdeml commented 4 years ago

I have updated the ClrMD nuget package to the latest version and found out that it cannot open my recent Linux dump. It crashes with the exception when opening the dump:

Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
   at Microsoft.Diagnostics.Runtime.Linux.ElfVirtualAddressSpace.Read(Int64 address, Span`1 buffer) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfVirtualAddressSpace.cs:line 54
   at Microsoft.Diagnostics.Runtime.Linux.Reader.TryRead[T](Int64 position) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfReader.cs:line 27
   at Microsoft.Diagnostics.Runtime.Linux.ElfLoadedImage.Open() in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfLoadedImage.cs:line 35
   at Microsoft.Diagnostics.Runtime.CoredumpReader.CreateModuleInfo(ElfLoadedImage image) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataReaders\Core\CoreDumpReader.cs:line 102
   at Microsoft.Diagnostics.Runtime.CoredumpReader.EnumerateModules() in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataReaders\Core\CoreDumpReader.cs:line 94
   at Microsoft.Diagnostics.Runtime.Linux.LinuxDefaultSymbolLocator..ctor(IDataReader reader) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\LinuxDefaultSymbolLocator.cs:line 25
   at Microsoft.Diagnostics.Runtime.DataTarget..ctor(CustomDataTarget customTarget) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 70
   at Microsoft.Diagnostics.Runtime.DataTarget.LoadDump(String displayName, Stream stream, CacheOptions cacheOptions, Boolean leaveOpen) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 283
   at Microsoft.Diagnostics.Runtime.DataTarget.LoadDump(String filePath, CacheOptions cacheOptions) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 311

When I tried again with the DEBUG build I got a more interesting exception (ElfVirtualAddressSpace.Read:52):

Unhandled exception. System.OverflowException: Arithmetic operation resulted in an overflow.
   at Microsoft.Diagnostics.Runtime.Linux.ElfVirtualAddressSpace.Read(Int64 address, Span`1 buffer) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfVirtualAddressSpace.cs:line 52
   at Microsoft.Diagnostics.Runtime.Linux.Reader.TryRead[T](Int64 position) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfReader.cs:line 27
   at Microsoft.Diagnostics.Runtime.Linux.ElfLoadedImage.Open() in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\ElfLoadedImage.cs:line 35
   at Microsoft.Diagnostics.Runtime.CoredumpReader.CreateModuleInfo(ElfLoadedImage image) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataReaders\Core\CoreDumpReader.cs:line 102
   at Microsoft.Diagnostics.Runtime.CoredumpReader.EnumerateModules() in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataReaders\Core\CoreDumpReader.cs:line 94
   at Microsoft.Diagnostics.Runtime.Linux.LinuxDefaultSymbolLocator..ctor(IDataReader reader) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\Linux\LinuxDefaultSymbolLocator.cs:line 25
   at Microsoft.Diagnostics.Runtime.DataTarget..ctor(CustomDataTarget customTarget) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 70
   at Microsoft.Diagnostics.Runtime.DataTarget.LoadDump(String displayName, Stream stream, CacheOptions cacheOptions, Boolean leaveOpen) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 283
   at Microsoft.Diagnostics.Runtime.DataTarget.LoadDump(String filePath, CacheOptions cacheOptions) in C:\_Temp\_Repositories\clrmd\src\Microsoft.Diagnostics.Runtime\src\DataTargets\DataTarget.cs:line 311

Git Bisect identified this commit as the culprit:

69a9a6e06ece15fc5811be83fe24daae091cd580 is the first bad commit 
commit 69a9a6e06ece15fc5811be83fe24daae091cd580                  
Author: Next Turn <45985406+NextTurn@users.noreply.github.com>   
Date:   Fri Aug 21 23:15:55 2020 +0800                           
Optimize memory readers (#808)

The dump is quite large (46 GB). 

> sosstatus
Target platform: 8664 Context size 04d0
.NET Core (Unix) runtime at 00007FE713EBB000 size 00754000
Runtime directory: /app
DAC file path: /tmp/sos8/libmscordaccore.so
Temp path: /tmp/sos8/
Cache: /root/.dotnet/symbolcache
Server: http://msdl.microsoft.com/download/symbols/

> eeversion
4.700.20.20201 (3.x runtime)
4.700.20.20201 @Commit: 05e3582c126663120b82a2eec8fd12dcb6929064
Server mode with 16 gc heaps

Feel free to ask for more details.

tomasdeml commented 4 years ago

I have added a few print statements and the output is:

Buffer.Length = 64
Bytes read = 0
Virtual size = 69632
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 1955127296
Offset = 0

Buffer.Length = 2
Bytes read = 0
Virtual size = 1955127296
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 1955127296
Offset = 1000615936

Buffer.Length = 2
Bytes read = 0
Virtual size = 1955127296
Offset = 1000615936

Buffer.Length = 64
Bytes read = 0
Virtual size = 728600576
Offset = 0

Buffer.Length = 2
Bytes read = 0
Virtual size = 728600576
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 1073487872
Offset = 0

Buffer.Length = 2
Bytes read = 0
Virtual size = 1073487872
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 637214720
Offset = 0

Buffer.Length = 2
Bytes read = 0
Virtual size = 637214720
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 816349184
Offset = 0

Buffer.Length = 2
Bytes read = 0
Virtual size = 816349184
Offset = 0

Buffer.Length = 64
Bytes read = 0
Virtual size = 2780004352
Offset = 0

=== OverflowException ===

It seems that the overflow occurs when casting the result of expression (int)(virtualSize - offset) to int. Result 2 780 004 352 is larger than int.MaxValue.

leculver commented 4 years ago

Should be fixed here: https://www.nuget.org/packages/Microsoft.Diagnostics.Runtime/2.0.151903