leecher1337 / ntvdmx64

Run Microsoft Windows NTVDM (DOS) on 64bit Editions
805 stars 81 forks source link

About source code #119

Closed peter8777555 closed 2 years ago

peter8777555 commented 3 years ago

I have build 13/Feb/2021 version and it works well.

I try dir ntdos/a/s/p

ntdos.map ntdos.sym ntdos404.map ntdos404.sym ntdos411.map ntdos411.sym ntdos412.map ntdos412.sym ntdos804.map ntdos804.sym

Where can i find the source code ? I need to debug something.

Thank you.

leecher1337 commented 3 years ago

.sym/.map files can help you with debugging, as they contain offsets for the respective functions. DOS Kernel is in mvdm\dos\v86\doskrnl\dos

For debugging, you can use your favourite DOS-debugger, or debug on hypervisor-side with yoda. ntsd/windbg support coming shortly. Be aware that these are debuggers without DOS symbol support, so you need to resolve symbols by hand. The CPU tracing/logging function of yoda often helped me debugging problems.

Good luck!

peter8777555 commented 3 years ago

Thank you for useful information.

By the way,i have a suggestion.

NTVDMX64 needs C:\Temp\SymbolCache

for some reasons:

  1. Maybe MS stop download the symbol files in the future.
  2. I hope download one time ONLY for NTVDMX64 support Win2003/2008/7/8/10.
  3. Backup ALL OS symbol files.

Add a new set var,for example ntvdmsym

set ntvdmsym=c:\MyTools\NTVDMX64\SymbolCache (User define the c:\MyTools\NTVDMX64\SymbolCache)

You can make a bat to get Win2003/2008/7/8/10 symbol files.

c:\MyTools\NTVDMX64\SymbolCache\usa\SymbolCache\Win2003 c:\MyTools\NTVDMX64\SymbolCache\usa\SymbolCache\Win2008 c:\MyTools\NTVDMX64\SymbolCache\usa\SymbolCache\Win7 c:\MyTools\NTVDMX64\SymbolCache\usa\SymbolCache\Win8 c:\MyTools\NTVDMX64\SymbolCache\usa\SymbolCache\Win10

sometimes i need CHT version for testing

c:\MyTools\NTVDMX64\SymbolCache\CHT\SymbolCache\Win2003 c:\MyTools\NTVDMX64\SymbolCache\CHT\SymbolCache\Win2008 c:\MyTools\NTVDMX64\SymbolCache\CHT\SymbolCache\Win7 c:\MyTools\NTVDMX64\SymbolCache\CHT\SymbolCache\Win8 c:\MyTools\NTVDMX64\SymbolCache\CHT\SymbolCache\Win10

NTVDMX64 check NTVDMX64 language version(usa/cht/jp/...) and automatic reading %ntvdmsym% for symbol files.

leecher1337 commented 3 years ago

At least, I have a full backup of old MS Symbol packs, even these hard-to-find Windows 2000 symbols:

Windows_2000\RTM Windows_2000\SP1 Windows_2000\SP2 Windows_2000\SP3 Windows_2000\SP4 Windows_2003\RTM Windows_2003\SP1 Windows_2003\SP2 Windows_XP\RTM Windows_XP\SP1 Windows_XP\SP1-CHK Windows_XP\SP2 Windows_XP\SP2-CHK Windows_XP\SP3 Windows_XP\SP3-CHK Windows_Vista\RTM Windows_Vista\SP1 Windows_Vista\SP2 Windows_7\RTM Windows_7\SP1 Windows_8\RTM Windows_10\10240 Windows_10\10586 Windows_10\14295 Windows_10\14393 Windows_10\14915

I also have an archive of Service packs (i.e also hard-to-find Windows 2000) for various languages and versions. If you need a particular symbol pack from my huge symbol pack archive and can't find it online, feel free to ask.

But I don't know about symbols and these various language versions. Also with these rolling Windows 10 Releases, MS doesn't provide symbol packs anymore as these are so many versions and changes that it's virtually impossible to keep track. The NTVDMx64 installer contains https://github.com/leecher1337/ntvdmx64/blob/master/ntvdmpatch/release/util/symfetch.exe which also gets used with the most revent installer to fetch and precache the desired symbols. If you have a huge MSDN library with all these various Windows versions in various languages, you can pull them from your CDs and create a big symbol cache yourself and maybe make a seperate repository with it, maybe we need it some day, who knows.. :) I would suggest to keep the directory structure that the symbol cache uses, because it's already sorted by checksum of the various files. That has the advantage that you can take it 1:1 and serve it on your own symbol server.

peter8777555 commented 3 years ago

I love NTVDMX64,it is a great project. Thank leecher1337 for creating NTVDMX64.

I hope NTVDMX64 with long long life forever. I hope NTVDMX64 can run after 50/100/1000 years :-D

NTVDMX64 can run:

  1. Build NTVDMX64 and install.bat
  2. MS Symbol packs --> (This is my worry issue)

NTVDMX64 supports Win2003/2008/7/8/10. There are too many variables(various OS version/various language/various OS service pack) How to ensure NTVDMX64 can run after 50/100/1000 years ? Why does NTVDMX64 MUST with MS Symbol pack to run ?

We must have a good way to ensure NTVDMX64 can run. So it must have a backup of MS Symbol packs.

In my PC(Windows 7 CHT X64) SymbolCache : 5.75 MB Compress SymbolCache.7z : 0.98 MB It is a SMALL file.

I STRONG suggestion make a bat download MS Symbol packs for running NTVDMX64. For me,i must have CHT ver,if CHS/usa versions are more better. NTVDMX64 build and create 17 language versions.

For example: GetSymbolPack.bat Full --> Download 17 language versions for Win2003/2008/7/8/10. GetSymbolPack.bat CHT CHS usa --> Download 3 language versions ONLY for Win2003/2008/7/8/10.

Thank you leecher1337.

leecher1337 commented 3 years ago

The reason why we need symbols is to find the internal function's entry points in the system DLLs. Finding the functions by signatures would be very unrealiable, therefore Symbols get used to resolve them by their original name. Especially as there are so many updates to Windows 10, it's the easiest method do catch up with the hundreds of versions of the DLLs out there. I'm also a bit worried about debug Symbols for older Windows Versions (mainly because Win 2k Symbol packs also disapperaed suddenly and because M$ also removed lots of old KB articles for no good reason, so we cannot really trust them in this regard). I don't think that they will retire Symbol Server anytime soon, developers and M$ customers would get very, very mad at them, if they did. I'd worry more about the fact that some functions NTVDMx64 relies on may disappear from the OS, as M$ decided to freeze 32bit Windows 10 development and only continue development on the x64 branch which basically means that the 32bit Windows code we heavily rely on may be thrown out some day, as it is basically unused on x64 (except by NTVDMx64, we just have luck that it is still compiled in x64 Windows Versions too). Ironically, Windows 10 is much better in this regard than Windows Server 2003/ XP64. In these older Windows Versions, the 64bit CreateProcess function was missing crucial parts which forced ldntvdm.dll loader to basically do a complete CreateProcess reimplementation with VDM support compiled in, which gets used to launch NTVDM. Windows 7 still had some symbols internal to Kernel32, whereas starting with Windows 8 we have some exported public DLL functions we can hook to detect if 16bit application gets started.

As said, I do have a collection of various symbol packs. IIRC starting with Windows 7, they use the same DLL as in EN/US on all systems and just customize their language with the .mui files, so you don't have troubles with the various language versions of various DLLs anymore. Now still there are lots of DLL versions out there, it got far worse with the release of Windows 10 and all these frequent updates, which is neraly impossible to keep track of. I fear that your idea of creating hundreds of Symbol packs for NTVDMx64 is not so easy to accomplish, as you cannot just tell Symbol server "Give me all Symbols for DLL X for OS Version Y". Basically, you need to have the original file and get a hash from it so that the Symbol downloader knows what to download from Symbol Server. So I guess you'd need to have a MSDN subscription to have ALL Windows versions and updates that were ever released, extract all their DLLs and then ask Symbol Server to give you the symbols for it. Quite a lot of work!

In case you really don't have any Symbols, you can still resort to disassemble all the various libraries by hand and find the entry points of the appropriate functions and tell ldntvdm.dll about it by creating registry keys for the entry point addresses (in fact that's how the ldntvdm Symbol caching mechanism works). But of course, no user would be happy about having to do this by hand.

peter8777555 commented 3 years ago

IIRC starting with Windows 7, they use the same DLL as in EN/US on all systems and just customize their language with the .mui files, so you don't have troubles with the various language versions of various DLLs anymore.

It sounds like a good news. If i use CHT OS(Win2003/2008/7/8/10) and download the symbol, NTVDMX64 can run on ALL languages OS.

you cannot just tell Symbol server "Give me all Symbols for DLL X for OS Version Y". Basically, you need to have the original file and get a hash from it so that the Symbol downloader knows what to download from Symbol Server.

I have a idea for get symbol. Can i use Windows PE(Not FULL OS) to get symbol ? Just for symbol ONLY for getting backup symbol files. If Yes,can you make a bat to do this ?

By the way,

Windows 7 CHT X64 OS:

C:\TEMP\SYMBOLCACHE ├─appinfo.pdb │ └─9DAB25FEC28B4C5DBB47D89984B443092 ├─conhost.pdb │ └─ED5644D3B6B249648791AC3136673FC52 ├─kernel32.pdb │ └─C4312728BA1F4691955E99B2E026FAFC2 └─wkernel32.pdb └─1C690A8592304467BB15A09CEA7180FA2

9DAB25FEC28B4C5DBB47D89984B443092 ED5644D3B6B249648791AC3136673FC52 C4312728BA1F4691955E99B2E026FAFC2 1C690A8592304467BB15A09CEA7180FA2

Question : What is this ? Is this a fix or random vaule ? if i copy these files to ANOTHER Windows 7 OS,NTVDMX64 can work ?

leecher1337 commented 3 years ago

IIRC starting with Windows 7, they use the same DLL as in EN/US on all systems and just customize their language with the .mui files, so you don't have troubles with the various language versions of various DLLs anymore. It sounds like a good news. If i use CHT OS(Win2003/2008/7/8/10) and download the symbol, NTVDMX64 can run on ALL languages OS.

Theoretically yes, it seems like I have the same appinfo.pdb as you. I have German Windows 7 X64, SP1, Build 7601:

appinfo.dll - 6.1.7601.17514 conhost.exe - 6.1.7601.17514 kernel32.dll - 6.1.7601.17514 kernelbase.dll - 6.1.7601.17514

C:. +---appinfo.pdb ¦ +---9DAB25FEC28B4C5DBB47D89984B443092 +---conhost.pdb ¦ +---1A6E4869F31941B3BDB517A77A7331F72 +---kernel32.pdb ¦ +---7FBE5C47FC104C35925D71D76A8D46602 +---wkernel32.pdb +---820EEB5D68EF443ABBE61E837F814BE12

So the other hashes seem to differ, maybe you have other versions of the DLLs (that's why I listed them above), or there are still diffrences in the version for multiple languages. I checked also the downloaded symbol packs from MS for Win 7 RTM and SP1, seems that these are also different hashes again, i.e. conhost.pdb on SP1 is hash 60EE9F444C274ADCB1CD242A5F0BE2072 there. AFAIK, these hashes to uniquely identify a file are generated by the function SymSrvGetFileIndexes.

I have a idea for get symbol. Can i use Windows PE(Not FULL OS) to get symbol ? Just for symbol ONLY for getting backup symbol files. If Yes,can you make a bat to do this ? I don't understand, what does Win PE have to do with it?

You need to get all available DLLs from all available Windows versions, i.e. by downloading and extracting them from all possible updates and servicepacks that changed the file and by extracting them from all OS ISOs and then you can tell symbol server to hash them.

You can also scrape symbols with symchk utility: https://stackoverflow.com/questions/50179282/how-to-load-all-windows-symbols-from-server-starting-with-w10-version-1803-bu But as said, that requires you to get hold of all possible versions of the respective DLL files.

peter8777555 commented 3 years ago

MyPC_Win7X64_CHT.zip MyPC_Win7X64_CHT.zip

Microsoft Windows [Ver 6.1.7601] Windows 7 Ultimate Build 7601 Service Pack 1

appinfo.dll 6.1.7601.17514 (win7sp1_rtm.101119-1850) conhost.exe 6.1.7601.18015 (win7sp1_gdr.121129-1432) kernel32.dll 6.1.7601.18015 (win7sp1_gdr.121129-1432) KernelBase.dll 6.1.7601.18015 (win7sp1_gdr.121129-1432)

Thank you.

peter8777555 commented 3 years ago

there are still diffrences in the version for multiple languages. I checked also the downloaded symbol packs from MS for Win 7 RTM and SP1, seems that these are also different hashes again, i.e. conhost.pdb on SP1 is hash 60EE9F444C274ADCB1CD242A5F0BE2072 there. AFAIK, these hashes to uniquely identify a file are generated by the function SymSrvGetFileIndexes.

If i get many symbol packs with different hashes(various OS version/various language/various OS service pack). NTVDMX64 can work ?

For Example for appinfo.pdb : put them in the same as C:\TEMP\SYMBOLCACHE\appinfo.pdb

C:\TEMP\SYMBOLCACHE ├─appinfo.pdb │ └─111115FEC28B4C5DBB47D89984B443092 ├─appinfo.pdb │ └─222225FEC28B4C5DBB47D89984B443092 ├─appinfo.pdb │ └─333335FEC28B4C5DBB47D89984B443092 ├─appinfo.pdb │ └─444445FEC28B4C5DBB47D89984B443092

leecher1337 commented 3 years ago

Yes. You can i.e. download WinFuture update Pack for Windows 7 (which contains a collection of all updates released for Win7 after SP1) and maybe write a script to extract all the needed .dll files from there, if an update contains a version of the file, to get all the diffrent file versions and then run the symbol dumper over them and you should get a directory structure like you mentioned above that you can backup

peter8777555 commented 3 years ago

Yes.It is a good news. Thank you.