OctopusDeploy / Halibut

| Public | A secure communication stack for .NET using JSON-RPC over SSL.
Other
12 stars 44 forks source link

Keep trace logs for failed tests #508

Closed acodrington closed 11 months ago

acodrington commented 11 months ago

Background

Trace logs can provide us with invaluable information about Halibut when debugging problems, but unfortunately they are too verbose to be practical on the build server, as the extra logging causes the build logs to be prohibitively large.

This PR attempts to have the best of both worlds (info-level logs when everything is fine, trace-level logs when there are problems) by logging a test's trace logs to a separate log file which can be kept if the test fails.

Results

Fixes [sc-56164]

During the setup for each individual test (BaseTest.SetUp), a TraceLogFileLogger is created and passed to the SerilogLoggerBuilder when initializing the logger. This new class is responsible for writing logs to a temporary file, and copying that file to the build artifacts directory if needed.

Once the SerilogLoggerBuilder.Build method is called, the instance of TraceLogFileLogger is stored in memory for the currently running test and is utilised by a custom Serilog sink which will find and write to the appropriate trace logger for the current test each time a log message is sent.

After the test is completed, the teardown method (BaseTest.TearDown) will check the status of the test and have the TraceLogFileLogger copy the trace log file to the build artifacts if the test failed.

Trace log files are named {TestHash}.tracelog, where TestHash is a short Base64 hash based on the test name. This TestHash is also printed in the log output, allowing a developer to easily find the corresponding trace log file for any failed test.

Before

No trace logs for failed tests 😞

After

20231013-111756_firefox_pNkiCzwGe3

20231013-111549_firefox_Fje2qcr1nq

20231013-140855_jq5jSdEWB4

How to review this PR

Are there are any other tests or test base class which need to modified to log trace-level logs? I've modified the default for the IClientAndServiceBuilder implementations, but are there any tests that don't use these builders that need updating?

Pre-requisites

shortcut-integration[bot] commented 11 months ago

This pull request has been linked to Shortcut Story #56164: Add trace level logging to Halibut which logs to console or file only if the test fails..