DumpRegisteredManifest fails on 248 providers

[OfekShilon]

I tested manifest dumping on all ~1100 registered providers on my machine, below is the full list of failures:

ACPI Driver Trace Provider Active Directory Domain Services: SAM Active Directory: Kerberos Client Active Directory: NetLogon ADODB.1 ADOMD.1 ASP.NET Events ATA Port Driver Tracing Provider AuthFw NetShell Plugin BCP.1 BFE Trace Provider BITS Service Trace Certificate Services Client CredentialRoaming Trace Certificate Services Client Trace Circular Kernel Session Provider Classpnp Driver Tracing Provider Critical Section Trace Provider DBNETLIB.1 Deduplication Tracing Provider Disk Class Driver Tracing Provider Downlevel IPsec API Downlevel IPsec NetShell Plugin Downlevel IPsec Policy Store Downlevel IPsec Service EA IME API FD Core Trace FD Publication Trace FD SSDP Trace FD WNet Trace FD WSDAPI Trace FDPHost Service Trace File Kernel Trace; Operation Set 1 File Kernel Trace; Operation Set 2 File Kernel Trace; Optional Data File Kernel Trace; Volume To Log FWPKCLNT Trace Provider FWPUCLNT Trace Provider Heap Trace Provider IKEEXT Trace Provider IMAPI1 Shim IMAPI2 Concatenate Stream IMAPI2 Disc Master IMAPI2 Disc Recorder IMAPI2 Disc Recorder Enumerator IMAPI2 dll IMAPI2 Interleave Stream IMAPI2 Media Eraser IMAPI2 MSF IMAPI2 Multisession Sequential IMAPI2 Pseudo-Random Stream IMAPI2 Raw CD Writer IMAPI2 Raw Image Writer IMAPI2 Standard Data Writer IMAPI2 Track-at-Once CD Writer IMAPI2 Utilities IMAPI2 Write Engine IMAPI2 Zero Stream IMAPI2FS Tracing Infrared Monitor Service IntelCPEventProvider IPMI Driver Trace IPMI Provider Trace IrDA Protocol Driver Junos Pulse KMDFv1 Trace Provider Layer2 Security HC Diagnostics Trace Local Security Authority (LSA) Microsoft-IE-ReadingView Microsoft-Office-Business Connectivity Services Microsoft-Windows-AppModel-MessagingDataModel Microsoft-Windows-Bluetooth-BthLEEnum Microsoft-Windows-CertificateServicesClient-CertEnroll Microsoft-Windows-Connected-Search Microsoft-Windows-CredentialProviders Microsoft-Windows-CredProvHost Microsoft-Windows-DataCollectionService Microsoft-Windows-DirectWrite Microsoft-Windows-DirectWrite-FontCache Microsoft-Windows-EmbeddedAppLauncher Microsoft-Windows-EtwCollector Microsoft-Windows-EventLog-WMIProvider Microsoft-Windows-Fax Microsoft-Windows-Host-Network-Management Microsoft-Windows-InstallUX Microsoft-Windows-LanguageProfile Microsoft-Windows-LLTD-Mapper Microsoft-Windows-LLTD-MapperIO Microsoft-Windows-LLTD-Responder Microsoft-Windows-MFH265Enc Microsoft-Windows-Narrator-Inproc Microsoft-Windows-NetAdapterCim-Diag Microsoft-Windows-NetworkConnectivityStatus Microsoft-Windows-Ntfs-SQM Microsoft-Windows-NvdimmN Microsoft-Windows-OfflineFiles-CscApi Microsoft-Windows-OfflineFiles-CscDclUser Microsoft-Windows-OfflineFiles-CscFastSync Microsoft-Windows-OfflineFiles-CscNetApi Microsoft-Windows-OfflineFiles-CscService Microsoft-Windows-OfflineFiles-CscUM Microsoft-Windows-P2P-Mesh Microsoft-Windows-P2P-PNRP Microsoft-Windows-PerceptionRuntime Microsoft-Windows-PerceptionSensorDataService Microsoft-Windows-PersistentMemory-INvdimm Microsoft-Windows-PersistentMemory-NvdimmN Microsoft-Windows-PersistentMemory-VirtualNvdimm Microsoft-Windows-PmemDisk Microsoft-Windows-PnPMgrTriggerProvider Microsoft-Windows-PrintDialogs Microsoft-Windows-PrintDialogs3D Microsoft-Windows-PriResources-Deployment Microsoft-Windows-QoS-WMI-Diag Microsoft-Windows-Remote-FileSystem-Log Microsoft-Windows-Remote-FileSystem-Monitor Microsoft-Windows-RemoteDesktopServices-RemoteFX-SessionLicensing Microsoft-Windows-ResourceManager Microsoft-Windows-ScmBus Microsoft-Windows-SEC-Mitigation Microsoft-Windows-Sens Microsoft-Windows-StickyNotes Microsoft-Windows-SystemSettingsThreshold Microsoft-Windows-Tcpip-SQM-Provider Microsoft-Windows-Thermal-Polling Microsoft-Windows-UI-Shell Microsoft-Windows-USB-CCID Microsoft-Windows-WDAG-Service Microsoft-Windows-WiFiConfigSP Microsoft-Windows-WinML Microsoft-Windows-WLAN-BMRHandler Microsoft-WS-Licensing Microsoft.Windows.ResourceManager MMC Mobility Center Performance Trace Mobility Center Trace Mount Manager Trace MSADCE.1 MSADCF.1 MSADCO.1 MSADDS.1 MSADOX.1 MSDADIAG.ETW MSDAPRST.1 MSDAREM.1 MSDART.1 MSDASQL_1 MSDATL3.1 msiscsi_iScsi MUI Resource Trace Multimedia-HEVCDECODER Native WIFI Filter Driver Trace Native WIFI MSM Trace Network Location Awareness Trace Network Profile Manager NisDrvWFP Provider NSC IrDA Driver Ntfs_NtfsLog NTLM Security Protocol ODBC.1 ODBCBCP.1 OLEDB.1 PNPX AssocDB Trace Portable Device Connectivity API Trace PrintFilterPipelineSvc_ObjectsGuid RDP4VSTrace RdpCore Api Trace RDPEncComTrace Refsv1WppTrace RefsWppTrace RowsetHelper.1 RSS Platform Backgroundsync Perf Trace RSS Platform Backgroundsync Trace RSS Platform Perf Trace RSS Platform Trace SBP2 Port Driver Tracing Provider SD Bus Trace Security: Kerberos Authentication Security: NTLM Authentication Security: SChannel Security: TSPkg Security: WDigest Sensor ClassExtension Trace Service Control Manager Trace SetupAPI Trace SQLOLEDB_1 SQLSRV32.1 TCPIP Service Trace TerminalServer-MediaFoundationPlugin Thread Pool TS Client ActiveX Control Trace TS Client Trace TS Rdp Init Trace TS RDP Shell Trace TS Rdp Sound End Point Trace UMB Trace UmBus Driver Trace UMDF - Driver Manager Trace UMDF - Framework Trace UMDF - Host Process Trace UMDF - Lpc Driver Trace UMDF - Lpc Trace UMDF - Platform Library Trace UMDF - Reflector Trace UMDF - Test Trace UMDF - WDF Core UMPass Driver Trace USB Storage Driver Tracing Provider Volsnap VSS tracing provider Windows Connect Now Windows Defender Firewall API Windows Defender Firewall API - GP Windows Defender Firewall Driver Windows Defender Firewall NetShell Plugin Windows Defender Firewall Service Windows Kernel Trace Windows Media Player Trace Windows NetworkItemFactory Trace Windows Notification Facility Provider Windows Remote Management Trace Windows Wininit Trace Windows Winlogon Trace Wireless Client Trace WLAN AutoConfig Trace WLAN Diagnostics Trace WLAN Dialog Trace WLAN Extensibility Trace WLAN HC Diagnostics Trace WMI_Tracing WMI_Tracing_Client_Operations WMP Network Sharing API WMP Network Sharing Service WMP Network Sharing Taskbar WPD API Trace WPD Bluetooth MTP Emumerator Driver Trace WPD BusEnumService Trace WPD ClassExtension Trace WPD ClassInstaller Trace WPD Composite Driver Trace WPD FSDriver Trace WPD ShellExtension Trace WPD ShellServiceObject Trace WPD Types Trace WPD WiaCompat Trace WPD WMDMCompat Trace WSAT_TraceProvider Wudfx02000_KmdfTraceGuid XWizard Framework

Note that IntelCPEventProvider hangs - I suggested a fix in PR #835. (it still fails after the fix, but doesn't hang)

[OfekShilon]

the failures are because the (undocumented) TdhGetAllEventsInformation returns ERROR_NOT_FOUND for all the 256 event versions tried.
I suspect the right thing to do would be to P/Invoke the official TdhEnumerateManifestProviderEvents

[vancem]

The TdhEnumerateManifestProviderEvents API did not exist (or at least I missed it) when TraceEvents RegisteredTraceEventParser.GetManifestForRegisteredProvider was created. It looks like it would be good to move to that API now.

I should note that generally speaking you only need to use TraceParserGen on providers that are used heavily or you have non-trivial logic associated with their interpretation (where the strong typing of the events maybe helpful). Otherwise simply using RegisteredTraceEventParser to parse the events on the fly is fine.

[christian-clausen]

I noticed a similar number of manifest dump issues (using RegisteredTraceEventParser.GetManifestForRegisteredProvider) on my machine.

Our use-case is not related to TraceParserGen. We want to enable our users to browse and explore registered manifests, so they can build queries without/before enabling providers.

[caljnj]

hi i'd like to be able to do the same thing as the original poster.

I'm just starting out with C#.. Is there a bit of an idiot's guide to what I should be requesting when I P/Invoke TdhEnumerateManifestProviderEvents?

I guess it starts with replacing the function in this line.. then working out the change in input variables..

https://github.com/microsoft/perfview/blob/ef1b2562ed07b85a0e5386a711d91988ef395208/src/TraceEvent/RegisteredTraceEventParser.cs#L140

[brianrob]

@caljnj, I am not aware of one. I did a bit of searching and only found this MSDN documentation: https://docs.microsoft.com/en-us/windows/win32/etw/what-s-new-in-event-tracing and https://docs.microsoft.com/en-us/windows/win32/api/tdh/nf-tdh-tdhenumeratemanifestproviderevents.

[caljnj]

many thanks @brianrob! i'll dig around a bit in these articles and see what i come up with

[OfekShilon]

Recently a different repository added a fix for a similar issue: https://github.com/zodiacon/EtwExplorer/pull/3

They write -

If an instrumentation manifest is not available, then search for a matching WMI EventTrace class and enumerate the metadata there. The mapping of fields isn't perfect - but it was quicker than an alternate UI. "Windows Kernel Trace" and "Active Directory Domain Services: SAM" are two good examples. Some providers seem to have neither a manifest or a mof definition. e.g. Microsoft-Antimalware-ShieldProvider

[jdu2600]

☝️ That was me. 😃 That workaround was for dumping (MOF) metadata for legacy (non-manifest) providers. For these providers, returning "not found" when querying for a manifest is likely valid.

Though I also logged some of the missing/invalid XML errors that I encountered while trying to dump some manifest providers with GetManifestForRegisteredProvider() in #1067, #1068 and #1069.