search

crosire / blink

A tool which allows you to edit source code of any MSVC C++ project live at runtime
BSD 2-Clause "Simplified" License
1.09k stars 81 forks source link

Use compilation command stored in obj file #6

Closed optimizedaway closed 5 years ago

optimizedaway commented 5 years ago

I believe full compilation command is stored in on of .debug$S sections of obj file.

dumpbin /section:.debug$S /rawdata main.obj

Excerpt from the dump:

SECTION HEADER #2
.debug$S name
       0 physical address
       0 virtual address
   31FC8 size of raw data
   18CFB file pointer to raw data (00018CFB to 0004ACC2)
   4ACC3 file pointer to relocation table
       0 file pointer to line numbers
      A6 number of relocations
       0 number of line numbers
42100040 flags
         Initialized Data
         Discardable
         1 byte align
         Read Only

RAW DATA #2
  00000000: 04 00 00 00 F1 00 00 00 FC 06 00 00 2E 00 01 11  ....n...ü.......
  00000010: 00 00 00 00 44 3A 5C 44 65 76 65 6C 5C 70 62 72  ....D:\Devel\pbr
  00000020: 5C 62 75 69 6C 64 5C 48 6F 74 2E 78 36 34 5C 70  \build\Hot.x64\p
  00000030: 62 72 5C 6D 61 69 6E 2E 6F 62 6A 00 3A 00 3C 11  br\main.obj.:.<.
  00000040: 01 41 00 00 D0 00 13 00 0F 00 69 68 00 00 13 00  .A..?.....ih....
  00000050: 0F 00 69 68 00 00 4D 69 63 72 6F 73 6F 66 74 20  ..ih..Microsoft 
  00000060: 28 52 29 20 4F 70 74 69 6D 69 7A 69 6E 67 20 43  (R) Optimizing C
  00000070: 6F 6D 70 69 6C 65 72 00 B1 05 3D 11 01 63 77 64  ompiler.±.=..cwd
  00000080: 00 44 3A 5C 44 65 76 65 6C 5C 70 62 72 5C 70 62  .D:\Devel\pbr\pb
  00000090: 72 00 63 6C 00 43 3A 5C 50 72 6F 67 72 61 6D 20  r.cl.C:\Program 
  000000A0: 46 69 6C 65 73 20 28 78 38 36 29 5C 4D 69 63 72  Files (x86)\Micr
  000000B0: 6F 73 6F 66 74 20 56 69 73 75 61 6C 20 53 74 75  osoft Visual Stu
  000000C0: 64 69 6F 5C 32 30 31 37 5C 43 6F 6D 6D 75 6E 69  dio\2017\Communi
  000000D0: 74 79 5C 56 43 5C 54 6F 6F 6C 73 5C 4D 53 56 43  ty\VC\Tools\MSVC
  000000E0: 5C 31 34 2E 31 35 2E 32 36 37 32 36 5C 62 69 6E  \14.15.26726\bin
  000000F0: 5C 48 6F 73 74 58 38 36 5C 78 36 34 5C 43 4C 2E  \HostX86\x64\CL.
  00000100: 65 78 65 00 63 6D 64 00 2D 63 20 2D 49 44 3A 5C  exe.cmd.-c -ID:\
  00000110: 44 65 76 65 6C 5C 70 62 72 5C 70 62 72 20 2D 49  Devel\pbr\pbr -I
  00000120: 44 3A 5C 44 65 76 65 6C 5C 70 62 72 5C 62 75 69  D:\Devel\pbr\bui
  00000130: 6C 64 5C 48 6F 74 2E 78 36 34 5C 70 62 72 5C 73  ld\Hot.x64\pbr\s
  00000140: 68 61 64 65 72 73 20 2D 49 44 3A 5C 44 65 76 65  haders -ID:\Deve
  00000150: 6C 5C 70 62 72 5C 69 6D 67 75 69 20 2D 49 44 3A  l\pbr\imgui -ID:
  00000160: 5C 44 65 76 65 6C 5C 70 62 72 5C 76 69 73 69 74  \Devel\pbr\visit
  00000170: 5F 73 74 72 75 63 74 5C 69 6E 63 6C 75 64 65 20  _struct\include 
  00000180: 2D 5A 49 20 2D 4A 4D 43 20 2D 6E 6F 6C 6F 67 6F  -ZI -JMC -nologo
  00000190: 20 2D 57 33 20 2D 57 58 2D 20 2D 64 69 61 67 6E   -W3 -WX- -diagn
  000001A0: 6F 73 74 69 63 73 3A 63 6C 61 73 73 69 63 20 2D  ostics:classic -
  000001B0: 73 64 6C 2D 20 2D 4D 50 20 2D 4F 64 20 2D 44 5F  sdl- -MP -Od -D_
  000001C0: 44 45 42 55 47 20 2D 44 5F 57 49 4E 44 4F 57 53  DEBUG -D_WINDOWS
  000001D0: 20 2D 44 5F 57 49 4E 33 32 5F 57 49 4E 4E 54 3D   -D_WIN32_WINNT=
  000001E0: 30 78 30 36 30 30 20 2D 44 57 49 4E 33 32 20 2D  0x0600 -DWIN32 -
  000001F0: 44 5F 55 4E 49 43 4F 44 45 20 2D 44 55 4E 49 43  D_UNICODE -DUNIC
  00000200: 4F 44 45 20 2D 47 46 20 2D 47 6D 2D 20 2D 45 48  ODE -GF -Gm- -EH
  00000210: 73 20 2D 45 48 63 20 2D 52 54 43 31 20 2D 4D 44  s -EHc -RTC1 -MD
  00000220: 64 20 2D 47 53 2D 20 2D 66 70 3A 70 72 65 63 69  d -GS- -fp:preci
  00000230: 73 65 20 2D 70 65 72 6D 69 73 73 69 76 65 2D 20  se -permissive- 
  00000240: 2D 5A 63 3A 77 63 68 61 72 5F 74 20 2D 5A 63 3A  -Zc:wchar_t -Zc:
  00000250: 66 6F 72 53 63 6F 70 65 20 2D 5A 63 3A 69 6E 6C  forScope -Zc:inl
  00000260: 69 6E 65 20 2D 73 74 64 3A 63 2B 2B 31 37 20 2D  ine -std:c++17 -
  00000270: 46 6F 44 3A 5C 44 65 76 65 6C 5C 70 62 72 5C 62  FoD:\Devel\pbr\b
  00000280: 75 69 6C 64 5C 48 6F 74 2E 78 36 34 5C 70 62 72  uild\Hot.x64\pbr
  00000290: 5C 20 2D 46 64 44 3A 5C 44 65 76 65 6C 5C 70 62  \ -FdD:\Devel\pb
  000002A0: 72 5C 62 75 69 6C 64 5C 48 6F 74 2E 78 36 34 5C  r\build\Hot.x64\
  000002B0: 70 62 72 5C 76 63 31 34 31 2E 70 64 62 20 2D 47  pbr\vc141.pdb -G
  000002C0: 64 20 2D 54 50 20 2D 46 43 20 2D 65 72 72 6F 72  d -TP -FC -error
  000002D0: 72 65 70 6F 72 74 3A 70 72 6F 6D 70 74 20 2D 49  report:prompt -I
  000002E0: 22 43 3A 5C 50 72 6F 67 72 61 6D 20 46 69 6C 65  "C:\Program File
  000002F0: 73 20 28 78 38 36 29 5C 4D 69 63 72 6F 73 6F 66  s (x86)\Microsof
  00000300: 74 20 56 69 73 75 61 6C 20 53 74 75 64 69 6F 5C  t Visual Studio\
  00000310: 32 30 31 37 5C 43 6F 6D 6D 75 6E 69 74 79 5C 56  2017\Community\V
  00000320: 43 5C 54 6F 6F 6C 73 5C 4D 53 56 43 5C 31 34 2E  C\Tools\MSVC\14.
  00000330: 31 35 2E 32 36 37 32 36 5C 69 6E 63 6C 75 64 65  15.26726\include
  00000340: 22 20 2D 49 22 43 3A 5C 50 72 6F 67 72 61 6D 20  " -I"C:\Program 
  00000350: 46 69 6C 65 73 20 28 78 38 36 29 5C 4D 69 63 72  Files (x86)\Micr
  00000360: 6F 73 6F 66 74 20 56 69 73 75 61 6C 20 53 74 75  osoft Visual Stu
  00000370: 64 69 6F 5C 32 30 31 37 5C 43 6F 6D 6D 75 6E 69  dio\2017\Communi
  00000380: 74 79 5C 56 43 5C 54 6F 6F 6C 73 5C 4D 53 56 43  ty\VC\Tools\MSVC
  00000390: 5C 31 34 2E 31 35 2E 32 36 37 32 36 5C 61 74 6C  \14.15.26726\atl
  000003A0: 6D 66 63 5C 69 6E 63 6C 75 64 65 22 20 2D 49 22  mfc\include" -I"
  000003B0: 43 3A 5C 50 72 6F 67 72 61 6D 20 46 69 6C 65 73  C:\Program Files
  000003C0: 20 28 78 38 36 29 5C 4D 69 63 72 6F 73 6F 66 74   (x86)\Microsoft
  000003D0: 20 56 69 73 75 61 6C 20 53 74 75 64 69 6F 5C 32   Visual Studio\2
  000003E0: 30 31 37 5C 43 6F 6D 6D 75 6E 69 74 79 5C 56 43  017\Community\VC
  000003F0: 5C 41 75 78 69 6C 69 61 72 79 5C 56 53 5C 69 6E  \Auxiliary\VS\in
  00000400: 63 6C 75 64 65 22 20 2D 49 22 43 3A 5C 50 72 6F  clude" -I"C:\Pro
  00000410: 67 72 61 6D 20 46 69 6C 65 73 20 28 78 38 36 29  gram Files (x86)
  00000420: 5C 57 69 6E 64 6F 77 73 20 4B 69 74 73 5C 31 30  \Windows Kits\10
  00000430: 5C 49 6E 63 6C 75 64 65 5C 31 30 2E 30 2E 31 37  \Include\10.0.17
  00000440: 37 36 33 2E 30 5C 75 63 72 74 22 20 2D 49 22 43  763.0\ucrt" -I"C
  00000450: 3A 5C 50 72 6F 67 72 61 6D 20 46 69 6C 65 73 20  :\Program Files 
  00000460: 28 78 38 36 29 5C 57 69 6E 64 6F 77 73 20 4B 69  (x86)\Windows Ki
  00000470: 74 73 5C 31 30 5C 49 6E 63 6C 75 64 65 5C 31 30  ts\10\Include\10
  00000480: 2E 30 2E 31 37 37 36 33 2E 30 5C 75 6D 22 20 2D  .0.17763.0\um" -
  00000490: 49 22 43 3A 5C 50 72 6F 67 72 61 6D 20 46 69 6C  I"C:\Program Fil
  000004A0: 65 73 20 28 78 38 36 29 5C 57 69 6E 64 6F 77 73  es (x86)\Windows
  000004B0: 20 4B 69 74 73 5C 31 30 5C 49 6E 63 6C 75 64 65   Kits\10\Include
  000004C0: 5C 31 30 2E 30 2E 31 37 37 36 33 2E 30 5C 73 68  \10.0.17763.0\sh
  000004D0: 61 72 65 64 22 20 2D 49 22 43 3A 5C 50 72 6F 67  ared" -I"C:\Prog
  000004E0: 72 61 6D 20 46 69 6C 65 73 20 28 78 38 36 29 5C  ram Files (x86)\
  000004F0: 57 69 6E 64 6F 77 73 20 4B 69 74 73 5C 31 30 5C  Windows Kits\10\
  00000500: 49 6E 63 6C 75 64 65 5C 31 30 2E 30 2E 31 37 37  Include\10.0.177
  00000510: 36 33 2E 30 5C 77 69 6E 72 74 22 20 2D 49 22 43  63.0\winrt" -I"C
  00000520: 3A 5C 50 72 6F 67 72 61 6D 20 46 69 6C 65 73 20  :\Program Files 
  00000530: 28 78 38 36 29 5C 57 69 6E 64 6F 77 73 20 4B 69  (x86)\Windows Ki
  00000540: 74 73 5C 31 30 5C 49 6E 63 6C 75 64 65 5C 31 30  ts\10\Include\10
  00000550: 2E 30 2E 31 37 37 36 33 2E 30 5C 63 70 70 77 69  .0.17763.0\cppwi
  00000560: 6E 72 74 22 20 2D 49 22 43 3A 5C 50 72 6F 67 72  nrt" -I"C:\Progr
  00000570: 61 6D 20 46 69 6C 65 73 20 28 78 38 36 29 5C 57  am Files (x86)\W
  00000580: 69 6E 64 6F 77 73 20 4B 69 74 73 5C 4E 45 54 46  indows Kits\NETF
  00000590: 58 53 44 4B 5C 34 2E 36 2E 31 5C 49 6E 63 6C 75  XSDK\4.6.1\Inclu
  000005A0: 64 65 5C 75 6D 22 20 2D 49 22 43 3A 5C 50 72 6F  de\um" -I"C:\Pro
  000005B0: 67 72 61 6D 20 46 69 6C 65 73 20 28 78 38 36 29  gram Files (x86)
  000005C0: 5C 57 69 6E 64 6F 77 73 20 4B 69 74 73 5C 4E 45  \Windows Kits\NE
  000005D0: 54 46 58 53 44 4B 5C 34 2E 36 2E 31 5C 49 6E 63  TFXSDK\4.6.1\Inc
  000005E0: 6C 75 64 65 5C 75 6D 22 20 2D 58 00 73 72 63 00  lude\um" -X.src.
  000005F0: 61 70 70 5C 6D 61 69 6E 2E 63 70 70 00 70 64 62  app\main.cpp.pdb
  00000600: 00 44 3A 5C 44 65 76 65 6C 5C 70 62 72 5C 62 75  .D:\Devel\pbr\bu
  00000610: 69 6C 64 5C 48 6F 74 2E 78 36 34 5C 70 62 72 5C  ild\Hot.x64\pbr\
  00000620: 76 63 31 34 31 2E 70 64 62 00 00 09 00 24 11 6C  vc141.pdb....$.l
  00000630: 6F 67 67 65 72 00 09 00 24 11 69 6D 70 6F 72 74  ogger...$.import
  00000640: 00 0C 00 24 11 4D 69 63 72 6F 73 6F 66 74 00 06  ...$.Microsoft..
  00000650: 00 24 11 57 52 4C 00 0A 00 24 11 44 65 74 61 69  .$.WRL...$.Detai
...

This command could be used to recompile modified source files. Right now compilation command has to be hardcoded in the blink.exe.

crosire commented 5 years ago

Ah, so it is still there after all. Yeah, I did that previously (it's still in the commit log somewhere), but removed it since that information did no longer exist in the PDBs of projects I compiled with Visual C++ 2017 and thus I had though Microsoft removed it. But it could be that you need to enable the "Edit & Continue" PDB format for that information to appear in the PDB.

optimizedaway commented 5 years ago

I'm on "Edit & Continue" PDB (/ZI) and cannot find any compilation command neither with hexviewer nor with cvdump. There is only a linker command present.

crosire commented 5 years ago

Nevermind, I misread your post. You were talking about the OBJ file, not PDB. That's less usefull though, since there is no way to obtain this information from the application? Unless paths to the OBJ files are stored in the PDB, but if I recall correctly that is not the case.

optimizedaway commented 5 years ago

OBJ files are listed in the PDB as modules.

Microsoft (R) Debugging Information Dumper  Version 14.00.23611
Copyright (C) Microsoft Corporation.  All rights reserved.

*** IDs

*** MODULES

0001 "D:\Devel\pbr\build\Hot.x64\pbr\pbr.res"
0002 "D:\Devel\pbr\build\Hot.x64\pbr\tangent.obj"
0003 "D:\Devel\pbr\build\Hot.x64\pbr\mikktspace.obj"
0004 "D:\Devel\pbr\build\Hot.x64\pbr\import_obj.obj"
0005 "D:\Devel\pbr\build\Hot.x64\pbr\import.obj"
0006 "D:\Devel\pbr\build\Hot.x64\pbr\gui.obj"
0007 "D:\Devel\pbr\build\Hot.x64\pbr\tick.obj"
0008 "D:\Devel\pbr\build\Hot.x64\pbr\input_win.obj"
0009 "D:\Devel\pbr\build\Hot.x64\pbr\input.obj"
000A "D:\Devel\pbr\build\Hot.x64\pbr\states11.obj"
000B "D:\Devel\pbr\build\Hot.x64\pbr\scene11.obj"
000C "D:\Devel\pbr\build\Hot.x64\pbr\render11.obj"
000D "D:\Devel\pbr\build\Hot.x64\pbr\import11.obj"
000E "D:\Devel\pbr\build\Hot.x64\pbr\device11.obj"
000F "D:\Devel\pbr\build\Hot.x64\pbr\transform.obj"
0010 "D:\Devel\pbr\build\Hot.x64\pbr\log.obj"
0011 "D:\Devel\pbr\build\Hot.x64\pbr\hresult.obj"
0012 "D:\Devel\pbr\build\Hot.x64\pbr\file.obj"
0013 "D:\Devel\pbr\build\Hot.x64\pbr\allocator.obj"
0014 "D:\Devel\pbr\build\Hot.x64\pbr\main.obj"
0015 "D:\Devel\pbr\build\Hot.x64\pbr\imgui_widgets.obj"
0016 "D:\Devel\pbr\build\Hot.x64\pbr\imgui_draw.obj"
0017 "D:\Devel\pbr\build\Hot.x64\pbr\imgui.obj"
0018 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\d3d11.lib" "Import:d3d11.dll"
0019 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\d3d11.lib" "d3d11.dll"
001A "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\dxgi.lib" "Import:dxgi.dll"
001B "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\dxgi.lib" "dxgi.dll"
001C "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\dxguid.lib" "d:\os\obj\amd64fre\onecoreuap\windows\directx\misc\dxguid\daytona\objfre\amd64\d3d10guid.obj"
001D "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\ws2_32.lib" "Import:WS2_32.dll"
001E "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\ws2_32.lib" "WS2_32.dll"
001F "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\kernel32.lib" "Import:KERNEL32.dll"
0020 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\kernel32.lib" "KERNEL32.dll"
0021 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\user32.lib" "Import:USER32.dll"
0022 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\user32.lib" "USER32.dll"
0023 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\imm32.lib" "Import:IMM32.dll"
0024 "C:\Program Files (x86)\Windows Kits\10\lib\10.0.17763.0\um\x64\imm32.lib" "IMM32.dll"
…
optimizedaway commented 5 years ago

List of compilands (OBJ files) can be obtained with DIA https://gist.github.com/optimizedaway/5a2c2b5a9284e6a2e55ff0dd1badf083

crosire commented 5 years ago

Nice! Well in that case, this is doable. I want to avoid dependencies and DIA, but can extend the PDB parser to read that information easily (it's in the DBI stream, right after the DBI header). https://github.com/crosire/blink/blob/2ced27abd536dcae884413a321f9ced072c27ffb/source/pdb_reader.cpp#L217

optimizedaway commented 5 years ago

Great! avoid DIA if you can, although with NoRegCoCreate at least you avoid COM hell. I'm available should you need any help.

crosire commented 5 years ago

This works now =)

eduardgonzalez commented 5 years ago

First of all, Thank you fo blink. It is amazing :)

As you already know, blink is not able to extract the compilation command when compiling with /Z7. (in that case compilation command is not stored in .debug$S section). I use fastbuild (https://github.com/fastbuild/fastbuild ) and distributed compilation is problematic without Z7. After investigating the contents of an OBJ file compiled with Z7 I noticed that compilation command seems to be stored in .debug$T instead of .debug$S...