[QUESTION] How to update log file path based on test result (hashtag617)

[vikramsubramanian]

I've been playing around with the logging and so far it's great! But now I am wanting to do a bit more customization to suite our needs.

So basically we have a lot of API tests to get through and we're logging both PASSED and FAILED assertions to our logs and the HTML report.

This part is fine, except for the small hiccup that any one particular log file could be thousands of lines long.

So I have created a couple of functions to help split these logs on a per-test basis.

This is to get the current test details:

def get_current_test():
    path_to_module_and_test_name = os.environ['PYTEST_CURRENT_TEST'].split(' ')[0]
    list_details = path_to_module_and_test_name.split("::")
    path_to_module = list_details[0]
    test_name_with_data = list_details[1]
    path_details_list = path_to_module.split('/')
    test_module = path_details_list[-1]

    service_name = 'general'

    for x in ast.literal_eval(os.environ['API_SERVICES']):
        if x in path_details_list:
            service_name = x
            break

    if '[' in test_name_with_data:
        test_name = test_name_with_data.split('[')[0]
        test_data = test_name_with_data.replace(test_name, '')
    else:
        test_name = test_name_with_data
        test_data = test_name_with_data

    os.environ['test_module'] = test_module
    os.environ['test_service'] = service_name
    os.environ['test_name'] = test_name
    os.environ['test_data'] = test_data

Then I have a fixture to 'remove and create' the logger:

def update_logger():
    hashtag get test details
    get_current_test()

    hashtag set default status
    test_status = 'UNKNOWN'

    hashtag prevent double logs
    logger.remove()

    hashtag set logging
    logger.add(Path(os.environ.get('reports_dir'),
                    test_status,
                    os.environ['test_service'],
                    f'{os.environ["test_name"]}.log'),
               format=formatter)
    logger.add(sys.stderr, format=formatter)

And so far this works good for us. We end up with a break-down like this:

Reports
└── 18.03.2022
    └── 15.38.54
        ├── Execution_Report.html
        └── general
            └── test_again.log

And so basically, the idea is that we're able to split log files on a per test basis (or whatever we want really) to keep the logs as short and precise as possible.

BUT, this approach introduces a lot of overhead for having to look through MANY directories and log files to find the logs for failed tests.

So I searched a bit and there's a way to grab the result of a test, like so:

 hookwrapper=True)
def pytest_runtest_makereport(item, call):
    hashtag execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()

    hashtag we only look at actual [PASS/FAIL/SKIP] test calls, not setup/teardown
    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

And from this, I'd like to update the path that the logs would be generated in.

So I'd end up with something like this:

Reports
└── 18.03.2022
    └── 15.38.54
        ├── Execution_Report.html
        ├── FAIL
        │   └── general
        │       └── test_again.log
        └── PASS
            └── general
                └── Execution_Log.log

But the problem is that the path is already set for the log file in the update_logger fixture and I'm wondering if there's a way of changing that path location in the pytest_runtest_makereport fixture?

[vikramsubramanian]

Hi

I'm not sure this issue is totally related to Loguru. You probably need to find a way to store logged filepath somewhere "globally" and re-use from fixture or hook to move the file once test is finished (make sure the file is closed beforehand). Loguru won't move the file by itself, handlers aren't mutable once added to the logger. You could implement a custom comrpession function but problem is still the same: you need a way to access the test result "globally".

[vikramsubramanian]

I'm not sure this issue is totally related to Loguru

Yeah, I totally agree :) I was wondering if Loguru perhaps had a feature built in that would allow to perform this action.

make sure the file is closed beforehand

How would I do this with Loguru?

It seems default logger has this:

logging.shutdown() logging.close()

And I've tried these options with Loguru:

logger.remove()
logger.stop()
logger.complete()

I thought perhaps logger.remove() would 'close the file' (and generate the log file for me) but it doesn't do that.. I can only see the log file when everything is finished running.

Following my original post, this is what I have tried:

import shutil

 hookwrapper=True)
def pytest_runtest_makereport(item, call):
    hashtag execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()

    hashtag we only look at actual [PASS/FAIL/SKIP] test calls, not setup/teardown
    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

        existing_location = str(Path(os.environ.get('reports_dir'),
                                     f'{os.environ["test_name"]}.log'))

        new_location = str(Path(os.environ.get('reports_dir'),
                                test_status,
                                os.environ['test_service'],
                                f'{os.environ["test_name"]}.log'))

        logger.remove()

        shutil.move(existing_location, new_location)

But unfortunately, the existing_location file is not yet in that location.

[vikramsubramanian]

I apologize, I was a little over-eager on the response!

I have solved the issue and I'm just posting here for if anyone else finds this useful :)

I use a config.py (in root directory) file to set my report path:

from datetime import datetime
from pathlib import Path
import os
import pytest

def pytest_load_initial_conftests(args, early_config, parser):
    hashtag get date values
    current_date = datetime.now()
    date_folder = current_date.strftime('%d.%m.%Y')
    time_folder = current_date.strftime('%H.%M.%S')

    hashtag set directories
    os.environ['current_dir'] = str(Path(__file__))
    os.environ['project_dir'] = str(str([p for p in Path(os.environ['current_dir']).parents if p.parts[-1] == 'src'][0]))
    os.environ['reports_dir'] = str(Path(os.environ['project_dir'], 'Reports', f'{date_folder}', f'{time_folder}'))

    hashtag set services
    os.environ['API_SERVICES'] = str(['svc_1', 'svc_2, 'svc_3'])

Then I have my fixtures in the conftest.py (also in root directory):

import os
import sys
from pathlib import Path

import pytest
from loguru import logger

import import move_file
import get_current_test
import formatter

def update_logger():
    hashtag get test details
    get_current_test()

    hashtag prevent double logs
    logger.remove()

    hashtag set logging
    logger.add(Path(os.environ.get('reports_dir'),
                    f'{os.environ["test_name"]}.log'),
               format=formatter)
    logger.add(sys.stderr, format=formatter)

 hookwrapper=True)
def pytest_runtest_makereport(item, call):
    hashtag execute all other hooks to obtain the report object
    outcome = yield
    rep = outcome.get_result()

    hashtag we only look at actual [PASS/FAIL/SKIP] test calls, not setup/teardown
    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

        existing_location = str(Path(os.environ.get('reports_dir'),
                                     f'{os.environ["test_name"]}.log'))

        new_location = str(Path(os.environ.get('reports_dir'),
                                test_status,
                                os.environ['test_service'],
                                f'{os.environ["test_name"]}.log'))

        hashtag remove logger (generate log file)
        logger.remove()

        hashtag create NEW dir (if not already created)
        os.makedirs(Path(os.environ.get('reports_dir'),
                         test_status,
                         os.environ['test_service']),
                    exist_ok=True)

        hashtag move the file
        move_file(existing_location=existing_location,
                  new_location=new_location)

The custom functions I have used (move_file, get_current_test, formatter):

import shutil
from pathlib import Path

import pytest

def move_file(existing_location: str, new_location: str):
    try:
        shutil.move(Path(existing_location), new_location)
    except FileNotFoundError as e:
        pytest.fail(f'Could not find desired file. Please check file locations\n{e}')
    except Exception as e:
        pytest.fail(f'Unable to move file\n{e}')

import ast
import os

def get_current_test():
    path_to_module_and_test_name = os.environ['PYTEST_CURRENT_TEST'].split(' ')[0]
    list_details = path_to_module_and_test_name.split("::")
    path_to_module = list_details[0]
    test_name_with_data = list_details[1]
    path_details_list = path_to_module.split('/')
    test_module = path_details_list[-1]

    service_name = 'general'

    for x in ast.literal_eval(os.environ['API_SERVICES']):
        if x in path_details_list:
            service_name = x
            break

    if '[' in test_name_with_data:
        test_name = test_name_with_data.split('[')[0]
        test_data = test_name_with_data.replace(test_name, '')
    else:
        test_name = test_name_with_data
        test_data = test_name_with_data

    os.environ['test_module'] = test_module
    os.environ['test_service'] = service_name
    os.environ['test_name'] = test_name
    os.environ['test_data'] = test_data

def formatter(record):
    hashtag set the default logging info
    base_format = f'\n{{time:DD/MMM/YYYY H:M:S}} - {{level}}: {{message}}' + " " * 3

    base = base_format.format_map(record)
    lines = str(record["extra"].get("data", "")).splitlines()

    hashtag create indent format based on length of 'BASE'
    indent = "\n" + " " * len(base)

    hashtag create 'NEW' formatted message based on indentation format
    reformatted = base + indent.join(lines)

    hashtag update the original 'RECORD' with the updated formatted data
    record["extra"]["reformatted"] = reformatted

    hashtag return
    return "{extra[reformatted]}\n{exception}"

And then this will:

1. Start logging stderr when the test execution begins
2. Generate actual log file AFTER test has run using this hook pytest_runtest_makereport
3. Move the log file after it's been generated

I'm sure I can make this cleaner, but for now it's working as I need it.

Once again, thank you for the help and this awesome logger ! :)

[vikramsubramanian]

That looks fine. ;)

Here is a slightly simplified version to explain the principle in broad terms:

from loguru import logger
import pytest
import os
import shutil

 hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    rep = outcome.get_result()

    if rep.when == "call":
        if rep.passed:
            test_status = 'PASS'
        elif rep.failed:
            test_status = 'FAIL'
        elif rep.skipped:
            test_status = 'SKIP'
        else:
            test_status = 'ERROR'

        os.makedirs(test_status, exist_ok=True)
        shutil.move(item.session.config._logfile_path, test_status)

def update_logger(request):
    logfile_path = "logs/" + request.node.name + ".log"
    request.config._logfile_path = logfile_path
    logger.remove()
    handler_id = logger.add(logfile_path, mode="w")
    yield
    logger.remove(handler_id)

Note I'm using [pytest.Config]( to store the log filepath and move it ad the end of the test (once it has been removed by fixture).

[vikramsubramanian]

Sorry for late reply..

Thank you so much - you see, knew it could be done cleaner!

Really appreciate your time and help with the MULTIPLE back and forth questions and code.

I hope I don't need to bother you too soon again! :D

[vikramsubramanian]

Great, glad I could help you. :+1:

I'm closing this issue then. ;)