ClrMD 2.0 reports less number of pinned handles than ClrMD 1.1

[crui3er]

Hello, I am investigating process dump with ClrMD and noticed that after upgrading from version 1.1.1 to 2.0.3 it reports less pinned handles. I prepared 2 similar programs (for both version) that collect and report to file information about pinned handles. Here is extract from reports that show difference:

v.2.0.3 Total pinned: 599 Total overall: 78725 System.Byte[] -> 4 System.Object[] -> 28 System.Threading.OverlappedData -> 445

v.1.1.1 Total pinned: 1042 Total overall: 79168 System.Byte[] -> 326 System.Object[] -> 149 System.Threading.OverlappedData -> 445

Can you explain where this difference come from?

Also what is interesting is that when I analyze my dump file with JetBrains dotMemory tool it reports 1283 pinned objects , according to it there are 566 pinned byte arrays on a heap. Which is even more than I get with ClrMD 1.1.1.

Here is an example of program that I uses to report pinned handles information with ClrMD 2.0.3

private static string HandleToString(ClrHandle handle)
{
    return string.Format("{0} @{1:x12} -> {2}", (object)handle.HandleKind, handle.Address, HandleObjectToString(handle.Object));
}

private static string HandleObjectToString(ClrObject obj)
{
    ClrType type = obj.Type;
    return string.Format("{0} {1:x} {2}", type?.Name, obj.Address, obj.Size);
}

private static void ReportPinnedHandles(DataTarget dataTarget)
{
    var runtime = dataTarget.ClrVersions[0].CreateRuntime();

    var fileName = @"C:\Logs\PinnedHandlesReport_new_2_0_3.txt";
    if (File.Exists(fileName))
        File.Delete(fileName);

    using(var stream = new FileStream(fileName, FileMode.CreateNew))
    using (var writer = new StreamWriter(stream)) {
        var totalPinned = 0;
        var totalOverall = 0;
        var types = new Dictionary<ClrType, ulong>();
        foreach (var handle in runtime.EnumerateHandles()) {
            totalOverall++;
            if (!handle.IsPinned)
                continue;
            totalPinned++;
            var type = handle.Object.Type;
            if (!types.TryGetValue(type, out var currentNumber))
                types.Add(type, 1);
            else
                types[type] = currentNumber + 1;

            writer.WriteLine(HandleToString(handle));
        }
        writer.WriteLine();
        writer.WriteLine("Stat: ");
        foreach (var pair in types.OrderBy(c => c.Key.Name)) {
            writer.WriteLine("  " + pair.Key + " -> " + pair.Value);
        }
        writer.WriteLine();
        writer.WriteLine("Total pinned: " + totalPinned);
        writer.WriteLine("Total overall: " + totalOverall);
    }
}

I also attached full handles reports generated for both versions. PinnedHandlesReport_old_1_1_1.txt PinnedHandlesReport_new_2_0_3.txt

[leculver]

The person who wrote the original code for pinned handles in 1.1 made a design decision to report "fake" roots for the sub-objects that async pinned objects point to. You can see that code in 1.1 here: https://github.com/microsoft/clrmd/blob/b4c865dcfff7f96f5a0b58813afc35669b0891ee/src/Microsoft.Diagnostics.Runtime/src/Desktop/DesktopGCHeap.cs#L515-L576

In 2.0, we report only the real, actual GC handles as the runtime itself sees them.

This doesn't affect live vs dead object reporting, but if these sub-objects are indeed pinned transitively (I'm not sure whether they are or not) that might mean we should add something to the API surface to help discover these extra pinned objects.

You can duplicate the algorithm here if you need to replicate the 1.1 behavior for now.

[crui3er]

It seems there is a difference between pinned handles and pinned objects (which i am looking for). AsyncPinned handles are used for System.Threading.OverlappedData class that has special meaning for .net runtime (including GC). Objects stored in m_userObject are considered as logically pinned by runtime and can not be moved.

OverlappedData objects are special in that the objects that they refer to are logically pinned, even though there are no pinning handles in the handle table. This is achieved by treating the m_userObject field specially when marking through the heap:

• If m_userObject is an ordinary object, it is reported to the GC as pinned
• If m_userObject is an array, all pointers within the array are reported to the GC as pinned, but not the array itself.

https://stackoverflow.com/questions/7554927/what-is-an-async-pinned-handle https://github.com/dotnet/coreclr/pull/14982 https://docs.microsoft.com/en-us/dotnet/api/system.threading.overlapped.pack?view=netframework-4.7.1 (see Note section)

So I think it makes sense to extend API to help discover these all (or only extra) pinned objects.

[leculver]

I'll be shipping a fix to this in the next ClrMD release.

[leculver]

This was fixed a while back, but I forgot to close the issue.