Profiler does not log the release version on startup when the platform is Linux

[JcolemanNR]

Feature Description

During a recent GTSE investigation, @nr-ahemsath noted that the profiler did not log the agent version number on startup as expected. The customer's environment was an Azure App Service with Linux hosting. Based on a quick reading of the profiler code, it looks like the code that logs the agent version number is completely bypassed on Linux: https://github.com/newrelic/newrelic-dotnet-agent/blob/c3545e8cb2962f1916222c0e024c343aac48cd66/src/Agent/NewRelic/Profiler/Profiler/ICorProfilerCallbackBase.h#L1019

(the #ifdef PAL_STDCPP_COMPAT block is what is built for the Linux version of the profiler, instead of the #else block).

This is likely because the API being used to read the agent core DLL's file version info is Windows-specific: https://stackoverflow.com/questions/38094316/getfileversioninfo-equivalent-in-linux

Depending on the effort to implement the equivalent functionality in Linux, I think it would be worth it, since the GTSE issue in question caused a couple days of effort to be spent troubleshooting something which would have been completely obvious if the version of the profiler code running had been apparent from reading the log file.

This will help Customers, GTS, and Engineering while using the agent.

[angelatan2]

Move to Q2 Technical Debt Milestone.

[workato-integration[bot]]

https://issues.newrelic.com/browse/NEWRELIC-3641

[workato-integration[bot]]

Jira CommentId: 173421 Commented by ahemsath:

There are a number of command-line tools and libraries for reading metadata from .DLL files that can work on Linux (without access to the Windows-only GetFileVersionInfo API).  My approach will be to try and reverse-engineer how one of them works.  I don't think we should create a dependency on an external tool to solve this problem.

Exiftool (Perl): [https://github.com/exiftool/exiftool]

pe-toolkit (Javascript): [https://github.com/ryanc16/pe-toolkit]

pe-parse (C++ library): [https://github.com/trailofbits/pe-parse]

That last one would seem to be closest to what we want since it's C++ code.  It's MIT-licensed so it could be used wholesale in the profiler, theoretically (haven't tried yet).  Since we only need a very specific subset of its functionality, we could also just forklift the relevant portions of the code into the profiler, with attribution of course.  For now I'm going to see if I can build the library standalone and debug through it to understand how it reads the file version info.

 

[workato-integration[bot]]

Jira CommentId: 173421 Commented by ahemsath:

There are a number of command-line tools and libraries for reading metadata from .DLL files that can work on Linux (without access to the Windows-only GetFileVersionInfo API).  My approach will be to try and reverse-engineer how one of them works.  I don't think we should create a dependency on an external tool to solve this problem.

Exiftool (Perl): [https://github.com/exiftool/exiftool]

pe-toolkit (Javascript): [https://github.com/ryanc16/pe-toolkit]

pe-parse (C++ library): [https://github.com/trailofbits/pe-parse]

That last one would seem to be closest to what we want since it's C++ code.  It's MIT-licensed so it could be used wholesale in the profiler, theoretically (haven't tried yet).  Since we only need a very specific subset of its functionality, we could also just forklift the relevant portions of the code into the profiler, with attribution of course.  For now I'm going to see if I can build the library standalone and debug through it to understand how it reads the file version info.

 

[workato-integration[bot]]

Jira CommentId: 177257 Commented by ahemsath:

Here's some code that might work but hasn't been tested:

include

include

include

include

include

include

bool read_bytes(std::ifstream& file, std::vector& buffer, size_t count) { buffer.resize(count); file.read(buffer.data(), count); return file.gcount() == count; }

template T read_struct(std::ifstream& file) { T result; file.read(reinterpret_cast<char*>(&result), sizeof(T)); return result; }

std::string GetDotNetAssemblyVersion(const std::string& filePath) { std::ifstream assemblyFile(filePath, std::ios::binary); if (!assemblyFile) { return ""; }

// Read the MS-DOS header
auto dos_header = read_struct<IMAGE_DOS_HEADER>(assemblyFile);
if (dos_header.e_magic != IMAGE_DOS_SIGNATURE) \{
    return "";
}

// Go to the PE header location
assemblyFile.seekg(dos_header.e_lfanew, std::ios::beg);

// Read the PE header
auto pe_header = read_struct<IMAGE_NT_HEADERS>(assemblyFile);
if (pe_header.Signature != IMAGE_NT_SIGNATURE) \{
    return "";
}

// Find the CLR header in the DataDirectory
auto clr_header_data_dir = pe_header.OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR];
if (clr_header_data_dir.VirtualAddress == 0 || clr_header_data_dir.Size == 0) \{
    return "";
}

// Read the CLR header
assemblyFile.seekg(clr_header_data_dir.VirtualAddress, std::ios::beg);
auto clr_header = read_struct<IMAGE_COR20_HEADER>(assemblyFile);

// Find the metadata location
auto metadata_rva = clr_header.MetaData.VirtualAddress;
auto metadata_size = clr_header.MetaData.Size;
if (metadata_rva == 0 || metadata_size == 0) \{
    return "";
}

// Read the metadata header
assemblyFile.seekg(metadata_rva, std::ios::beg);
auto metadata_header = read_struct<STORAGESIGNATURE>(assemblyFile);

// Verify the metadata signature
if (metadata_header.lSignature != 0x424A5342) \{
    return "";
}

// Get the version string from the metadata header
std::vector<char> versionBuffer;
if (!read_bytes(assemblyFile, versionBuffer, metadata_header.iVersionString)) \{
    return "";
}

std::string version(versionBuffer.begin(), versionBuffer.end());
return version;

}

int main() { std::string assemblyPath = "/path/to/your/assembly.dll"; std::string version = GetDotNetAssemblyVersion(assemblyPath); if (version.empty()) { std::cout << "Unable to get the assembly version." << std::endl; } else { std::cout << "Assembly version: " << version << std::endl; } return 0; }

some struct definitions:

include

// MS-DOS Header (also called DOS MZ header)

pragma pack(push, 1)

struct IMAGE_DOS_HEADER { std::uint16_t e_magic; // Magic number std::uint16_t e_cblp; // Bytes on the last page of the file std::uint16_t e_cp; // Pages in the file std::uint16_t e_crlc; // Relocations std::uint16_t e_cparhdr; // Size of the header in paragraphs std::uint16_t e_minalloc; // Minimum extra paragraphs needed std::uint16_t e_maxalloc; // Maximum extra paragraphs needed std::uint16_t e_ss; // Initial (relative) SS value std::uint16_t e_sp; // Initial SP value std::uint16_t e_csum; // Checksum std::uint16_t e_ip; // Initial IP value std::uint16_t e_cs; // Initial (relative) CS value std::uint16_t e_lfarlc; // Address of the relocation table std::uint16_t e_ovno; // Overlay number std::uint16_t e_res[4]; // Reserved words std::uint16_t e_oemid; // OEM identifier (for e_oeminfo) std::uint16_t e_oeminfo; // OEM information; e_oemid specific std::uint16_t e_res2[10]; // Reserved words std::int32_t e_lfanew; // File address of the new exe header };

pragma pack(pop)

define IMAGE_DOS_SIGNATURE 0x5A4D // MZ

// PE File Header

pragma pack(push, 1)

struct IMAGE_FILE_HEADER { std::uint16_t Machine; std::uint16_t NumberOfSections; std::uint32_t TimeDateStamp; std::uint32_t PointerToSymbolTable; std::uint32_t NumberOfSymbols; std::uint16_t SizeOfOptionalHeader; std::uint16_t Characteristics; };

pragma pack(pop)

// Data Directory

pragma pack(push, 1)

struct IMAGE_DATA_DIRECTORY { std::uint32_t VirtualAddress; std::uint32_t Size; };

pragma pack(pop)

define IMAGE_NUMBEROF_DIRECTORY_ENTRIES 16

// Optional Header

pragma pack(push, 1)

struct IMAGE_OPTIONAL_HEADER { std::uint16_t Magic; std::uint8_t MajorLinkerVersion; std::uint8_t MinorLinkerVersion; std::uint32_t SizeOfCode; std::uint32_t SizeOfInitializedData; std::uint32_t SizeOfUninitializedData; std::uint32_t AddressOfEntryPoint; std::uint32_t BaseOfCode; std::uint32_t BaseOfData; std::uint32_t ImageBase; std::uint32_t SectionAlignment; std::uint32_t FileAlignment; std::uint16_t MajorOperatingSystemVersion; std::uint16_t MinorOperatingSystemVersion; std::uint16_t MajorImageVersion; std::uint16_t MinorImageVersion; std::uint16_t MajorSubsystemVersion; std::uint16_t MinorSubsystemVersion; std::uint32_t Win32VersionValue; std::uint

[workato-integration[bot]]

Jira CommentId: 179234 Commented by cventura:

Multiple approaches have been experimented with to add the desired behavior (printing out a version number in the linux profiler logs).

Reading and parsing the version info from the managed agent dll at runtime.

Setting and printing a profiler version number on linux.

For approach 1 I looked at the libraries that Alex found as well as the code snippet that he provided. Both of those approaches require cloning some built in windows structs and constants. That either happens in the library or within the code that we write. This approach also requires extensive testing across multiple linux versions and arm64 to ensure that we do not cause any unexpected problems. During our standup meeting as a team we decided that the amount of effort required to implement this approach is not worth doing at this point in time.

For approach 2, we are already generating version information in the profiler for the windows builds. This approach utilizes a .net45 application that gets version info added to it via msbuild and then that application generates a VersionInfo.h file that contains version constants that can be used in the native code. Multiple approaches were attempted to replicate this behavior on Linux.

Update the console application to run on .net6 and use that application to generate the version file

Use a shell script to generate the version file by replicating the git commands and version logic defined in the msbuild definitions

For the first approach, there are a couple of problems. First, we build the profiler using an older version of ubuntu that is no longer supported by .net. Second, we do not have access to the local git repository in the docker container that's used to build the profiler. We run into a similar problem with the second approach. In order to get things working we will need a solution for local linux builds, CI linux x64 builds, CI linux arm64 builds, and windows builds. There's a lot more room for introducing undetected errors or some slight drift between all of these profiler builds.

My recommendation is to close this issue until we can improve our profiler build system.

[workato-integration[bot]]

This issue won't be actioned.