missing symbols from generated pdb on windows 11 for FFmpeg shared libraries

[neeraj9]

First of all. Thanks for this tool . It is a great start.

Operating System: Windows 11 22H2 x64 Library avfilter-9.dll (from FFmpeg release/6.0 built with debug symbols for win64) via Linux mingw.

Overview

I noticed that some of the symbols are missing, while others exist. I believe there is a case where cv2pdb is able to recognize few in DWARF and not others.

How did I verify this?

Take FFmpeg (release/6.0) avfilter-9.dll which was built with debug symbols and then run llvm-dwarfdump on it. Note that llvm-dwarfdump also has an issue, but it still generates sufficient info to validate the claim.

The associated source code in FFmpeg avfilter-9.dll is as follows:

int avfilter_graph_create_filter(AVFilterContext **filt_ctx, const AVFilter *filt,
                                 const char *name, const char *args, void *opaque,
                                 AVFilterGraph *graph_ctx)
{
    int ret;

    *filt_ctx = avfilter_graph_alloc_filter(graph_ctx, filt, name);
    if (!*filt_ctx)
        return AVERROR(ENOMEM);

    ret = avfilter_init_str(*filt_ctx, args);
    if (ret < 0)
        goto fail;

    return 0;

fail:
    avfilter_free(*filt_ctx);
    *filt_ctx = NULL;
    return ret;
}

llvm-dwarfdump -a --color avfilter-9.dll > avfilter-9.dll.dwarf.txt 

A snippet of the dwarf output is as follows (for reference). Notice that

0x002ba926:   DW_TAG_subprogram
                DW_AT_external  (true)
                DW_AT_name  ("avfilter_graph_create_filter")
                DW_AT_decl_file ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                DW_AT_decl_line (138)
                DW_AT_decl_column   (0x05)
                DW_AT_prototyped    (true)
                DW_AT_type  (0x002b264d "int")
                DW_AT_low_pc    (0x000000018008b267)
                DW_AT_high_pc   (0x000000018008b2bc)
                DW_AT_frame_base    (DW_OP_call_frame_cfa)
                DW_AT_call_all_calls    (true)
                DW_AT_sibling   (0x002baa47)

0x002ba960:     DW_TAG_formal_parameter
                  DW_AT_name    ("filt_ctx")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (138)
                  DW_AT_decl_column (0x34)
                  DW_AT_type    (0x002b67b9 "AVFilterContext **")
                  DW_AT_location    (0x00063598: 
                     [0x000000018008b267, 0x000000018008b278): DW_OP_reg2 RCX
                     [0x000000018008b278, 0x000000018008b2a3): DW_OP_reg4 RSI
                     [0x000000018008b2a3, 0x000000018008b2a4): DW_OP_entry_value(DW_OP_reg2 RCX), DW_OP_stack_value
                     [0x000000018008b2a4, 0x000000018008b2bc): DW_OP_reg4 RSI)
                  DW_AT_GNU_entry_view  (0x00063590)

0x002ba978:     DW_TAG_formal_parameter
                  DW_AT_name    ("filt")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (138)
                  DW_AT_decl_column (0x4e)
                  DW_AT_type    (0x002b64c6 "const AVFilter *")
                  DW_AT_location    (0x000635bc: 
                     [0x000000018008b267, 0x000000018008b27c): DW_OP_reg1 RDX
                     [0x000000018008b27c, 0x000000018008b2bc): DW_OP_entry_value(DW_OP_reg1 RDX), DW_OP_stack_value)
                  DW_AT_GNU_entry_view  (0x000635b8)

0x002ba98c:     DW_TAG_formal_parameter
                  DW_AT_name    ("name")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (139)
                  DW_AT_decl_column (0x2e)
                  DW_AT_type    (0x002b2a2f "const char *")
                  DW_AT_location    (0x000635d2: 
                     [0x000000018008b267, 0x000000018008b27c): DW_OP_reg8 R8
                     [0x000000018008b27c, 0x000000018008b2bc): DW_OP_entry_value(DW_OP_reg8 R8), DW_OP_stack_value)
                  DW_AT_GNU_entry_view  (0x000635ce)

0x002ba99f:     DW_TAG_formal_parameter
                  DW_AT_name    ("args")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (139)
                  DW_AT_decl_column (0x40)
                  DW_AT_type    (0x002b2a2f "const char *")
                  DW_AT_location    (0x000635ee: 
                     [0x000000018008b267, 0x000000018008b27c): DW_OP_reg9 R9
                     [0x000000018008b27c, 0x000000018008b292): DW_OP_reg3 RBX
                     [0x000000018008b292, 0x000000018008b2b5): DW_OP_entry_value(DW_OP_reg9 R9), DW_OP_stack_value
                     [0x000000018008b2b5, 0x000000018008b2ba): DW_OP_reg3 RBX
                     [0x000000018008b2ba, 0x000000018008b2bc): DW_OP_entry_value(DW_OP_reg9 R9), DW_OP_stack_value)
                  DW_AT_GNU_entry_view  (0x000635e4)

0x002ba9b3:     DW_TAG_formal_parameter
                  DW_AT_name    ("opaque")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (139)
                  DW_AT_decl_column (0x4c)
                  DW_AT_type    (0x002b2a48 "void *")
                  DW_AT_location    (DW_OP_fbreg +32)

0x002ba9c1:     DW_TAG_formal_parameter
                  DW_AT_name    ("graph_ctx")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (140)
                  DW_AT_decl_column (0x31)
                  DW_AT_type    (0x002b67d5 "AVFilterGraph *")
                  DW_AT_location    (DW_OP_fbreg +40)

0x002ba9d6:     DW_TAG_variable
                  DW_AT_name    ("ret")
                  DW_AT_decl_file   ("/ffbuild/ffmpeg\libavfilter\avfiltergraph.c")
                  DW_AT_decl_line   (142)
                  DW_AT_decl_column (0x09)
                  DW_AT_type    (0x002b264d "int")
                  DW_AT_location    (0x0006361e: 
                     [0x000000018008b292, 0x000000018008b29b): DW_OP_reg0 RAX
                     [0x000000018008b2a4, 0x000000018008b2ab): DW_OP_reg0 RAX
                     [0x000000018008b2ab, 0x000000018008b2b5): DW_OP_reg3 RBX)
                  DW_AT_GNU_entry_view  (0x00063618)

Notice that the DWARF has all the symbols within the function. Now, lets generate PDB and inspect that with Microsoft cvdump.exe application. I even checked with WinDbg while debugging the application and got same results (so they match).

cv2pdb avfilter-9.dll avfilter-9-fordebug.dll avfilter-9.pdb

cvdump avfilter-9.pdb > avfilter-9.pdb.txt

A snippet of avfilter-9.pdb.txt for reference.

(02D8B8) S_GPROC32: [0001:0008A267], Cb: 00000055, Type:     T_NOTYPE(0000), avfilter_graph_create_filter
         Parent: 00000000, End: 0002D930, Next: 00000000
         Debug start: 00000000, Debug end: 00000055

(02D8FC)  S_REGREL32: rsp+00000038, Type:             0x100B, opaque
(02D914)  S_REGREL32: rsp+00000040, Type:             0x15CF, graph_ctx
(02D92C)  S_ENDARG

(02D930) S_END

We can clearly notice that the function avfilter_graph_create_filter is missing many symbols. If I look at some of the other functions then they are missing symbols entirely. It appears that cv2pdb is missing implementation of DWARF 5 standard.

The DWARF version can be determined via the .debug_info contents from the output from llvm-dwarfdump tool.

.debug_info contents:
0x00000000: Compile Unit: length = 0x0000188c, format = DWARF32, version = 0x0005, unit_type = DW_UT_compile, abbr_offset = 0x0000, addr_size = 0x08 (next unit at 0x00001890)

Update: I used dia2dump utility provided as a sample from Visual Studio 2022 as well and got similar results as cvdump.exe.

dia2dump -all avfilter-9-fordebug.dll > avfilter-9.dll.dia2dump.txt

The sample output for the same function from dia2dump is as follows:

Function       : static, [0008B267][0001:0008A267], len = 00000055, avfilter_graph_create_filter
                 Function attribute:
                 Function info:
FuncDebugStart :   static, [0008B267][0001:0008A267]
FuncDebugEnd   :   static, [0008B2BC][0001:0008A2BC]
Data           :   rsp Relative, [00000038], Local, Type: void *, opaque
Data           :   rsp Relative, [00000040], Local, Type: struct AVFilterGraph *, graph_ctx

Function       : static, [0008B0FF][0001:0008A0FF], len = 00000065, avfilter_graph_free
                 Function attribute:
                 Function info:
FuncDebugStart :   static, [0008B0FF][0001:0008A0FF]
FuncDebugEnd   :   static, [0008B164][0001:0008A164]

[neeraj9]

Apparently, https://github.com/davea42/libdwarf-code is an actively maintained project instead of the non-existent reference in this project readme for libdwarf (a broken link).

[neeraj9]

Any next steps to implement DWARF 5 support to fix this issue is appreciated. I am new to this project and any details or reading material (which includes PRs I should look at) to get started quickly will be of great help.

cc: @rainers

[rainers]

Sorry for the delay. Thanks for the detailed report. Not sure I can help in this case (see below), but is the DLL with debug information available for download somewhere? The one from https://www.gyan.dev/ffmpeg/builds/ doesn't seem to have any.

re DWARF5: Actually, I'm not using this tool anymore myself, as the reference D compiler natively supports emitting usable CodeView debug information for quite some time now. Maybe this might change if GDC (the D compiler in the GCC compiler suite) is made easily available and becomes more popular on Windows.

Apart from the initial implementation, DWARF support is mostly added by other contributors. The basic DWARF5 implementation PRs are https://github.com/rainers/cv2pdb/pull/67 and https://github.com/rainers/cv2pdb/pull/69 (name similarity a coincidence?).