Identifying Memory Leaks with dotnet-dump and dotnet-gcdump

[Samsung Newsroom]

In this tutorial, you will learn how to monitor memory usage of a Tizen .NET application and identify possible memory leaks using dotnet-dump and dotnet-gcdump.

Note: The total memory usage of an application process is affected by various factors (for example, shared size, swap size, or memory allocated by the runtime for its internal use). Only the managed memory region (GC heap) can be analyzed with dotnet-dump and dotnet-gcdump.

Prerequisites

You need the following things installed on your device:

• .NET diagnostic tools (see Installing .NET diagnostic tools on Tizen devices)
• Target application

Monitoring memory usage

First, obtain the process ID of the currently running application process. Use dotnet dump ps to list all .NET processes running on the device.

sh-3.2$ dotnet counters ps
     17876 dotnet-loader /usr/bin/dotnet-loader
     17464 LeakMemory.dll /usr/bin/dotnet-loader

To examine the current memory usage of the target process, use dotnet counters monitor.

sh-3.2$ dotnet counters monitor -p 17464
Press p to pause, r to resume, q to quit.
    Status: Running

[System.Runtime]
    % Time in GC since last GC (%)                         0
    Allocation Rate / 1 sec (B)                       56,992
    CPU Usage (%)                                         29
    Exception Count / 1 sec                                0
    GC Heap Size (MB)                                     34
    Gen 0 GC Count / 60 sec                                0
    Gen 0 Size (B)                                        12
    Gen 1 GC Count / 60 sec                                0
    Gen 1 Size (B)                                    13,820
    Gen 2 GC Count / 60 sec                                0
    Gen 2 Size (B)                                   493,068
    LOH Size (B)                                  33,587,792
    Monitor Lock Contention Count / 1 sec                  0
    Number of Active Timers                                3
    Number of Assemblies Loaded                           52
    ThreadPool Completed Work Item Count / 1 sec          61
    ThreadPool Queue Length                                0
    ThreadPool Thread Count                                2
    Working Set (MB)                                      89

In this tutorial, we are only interested with the managed heap size (GC Heap Size) of the process. (cf. Working Set indicates the total bytes of memory used by the process.) If the heap size unexpectedly grows over time, it is likely that your application has a memory leak.

Analyzing managed heap dump with dotnet-dump

To capture a memory dump (coredump) from the target process, use dotnet dump collect.

sh-3.2$ dotnet dump collect -p 17464 -o /home/owner/share/coredump.17464
Writing full to /home/owner/share/coredump.17464
Complete

Make sure the target process has write access to the specified path. Also ensure that there's enough space available on the disk (df -h /opt/usr) because the size of the dump is often very large (200+ MB). Otherwise, you will see a file write error (HRESULT: 0x80004005).

To analyze the generated coredump, use dotnet dump analyze.

sh-3.2$ dotnet dump analyze /home/owner/share/coredump.17464
Loading core dump: /home/owner/share/coredump.17464 ...
Ready to process analysis commands. Type 'help' to list available commands or 'help [command]' to get detailed help on a command.
Type 'quit' or 'exit' to exit the session.
>

You can now type any SOS command in the command prompt. For example, clrstack -all provides stack traces of all managed threads.

If we want to find managed objects within the GC heap that caused a memory leak, the dumpheap -stat command provides the overall statistics about the managed heap.

> dumpheap -stat
Statistics:
      MT    Count    TotalSize Class Name
...
f026e634       60        35556 System.Object[]
f026f3b8       30        46038 System.Char[]
f026fc3c      245        51356 System.Int32[]
f0290744     1531       133922 System.String
e3706968        2     16777240 Xamarin.Forms.ContentPage[]

The last line reveals that the memory leak is caused by Xamarin.Forms.ContentPage[] type objects. Use the gcroot command to print the chain of references that are keeping those objects alive.

> dumpheap -type Xamarin.Forms.ContentPage[]
 Address       MT     Size
f126391c e38cad40       12
d9c82010 e38cad40 33554444

Statistics:
      MT    Count    TotalSize Class Name
e38cad40        2     33554456 Xamarin.Forms.ContentPage[]
Total 2 objects

> gcroot d9c82010
Thread 4438:
    FF8BF398 F7023CD0 <unknown>
            ...
            ->  F12637B0 LeakMemory.Views.CircularButtonPage
            ->  F1263908 System.Collections.Generic.List`1[[Xamarin.Forms.ContentPage, Xamarin.Forms.Core]]
            ->  D9C82010 Xamarin.Forms.ContentPage[]

Found 1 unique roots (run 'gcroot -all' to see all roots).

As shown, the leaky objects are held by a LeakMemory.Views.CircularButtonPage type object.

This is the general procedure of identifying major sources of memory leaks in your application. You can continue investigating the coredump using various SOS commands and options to find other underlying issues in your code.

Analyzing managed heap dump with dotnet-gcdump

An alternative to dotnet-dump is dotnet-gcdump which also allows you to track object allocation and memory leaks from a running application process. GC dumps are often easier to use than coredumps because they are portable (cross-platform format) and lightweight (very small size). Find more information about GC dumps at Collecting and analyzing memory dumps.

To capture a GC dump (snapshot) from the target process, use dotnet gcdump collect. You can take multiple snapshots from the same process while it's running.

sh-3.2$ dotnet gcdump collect -p 17464 -o /home/owner/share/17464.gcdump
Writing gcdump to '/home/owner/share/17464.gcdump'...
        Finished writing 11101371 bytes.

Using Visual Studio

You can use Visual Studio to open and analyze the generated GC snapshot. Copy the .gcdump file from the device to the host by typing sdb pull /home/owner/share/17464.gcdump and open it in Visual Studio.

By sorting the objects by their types and sizes, List<Xamarin.Forms.ContentPage> appears at the top of the table, which is the main source of the memory leak. The chain of references is also shown in the bottom panel (Paths to Root).

If you have more than one snapshot, compare two different snapshots using diff to see which objects grow fast over time. Choose a snapshot that you want to serve as the baseline in the Compare to: section.

Using PerfView

If the information obtained above is not enough for you, you can switch to another tool called PerfView. At first glance, this tool doesn't look very intuitive, but is extremely helpful for performing complex analysis of managed heap data.

Tip: PerfView also supports the trace file format (.nettrace) used by the dotnet-trace tool (see Finding Performance Bottlenecks with dotnet-trace). You can specify detailed provider keywords to capture certain event information (for example, JIT or GC) when recording trace data.

By switching between a set of different views and filtering necessary data, you can get detailed information about managed objects in the GC heap. Pressing F1 anywhere opens a pop-up window with the help page for the current view. Also read the following pages for further details of the PerfView tool. Note that some parts of the documents are outdated or not generally applicable for Tizen (PerfView had been Windows-only for a long time before .NET Core was released in 2016).

• Profiling workflow for dotnet/runtime repository
• Performance Profiling of .NET Core 3 applications on Linux with dotnet-trace and PerfView

View the full blog at its source