Kuadrant / testsuite

4 stars 14 forks source link

Add Kuadrant version to pytest report #511

Closed azgabur closed 1 month ago

azgabur commented 2 months ago

This change will add image strings of deployed Kuadrants on clusters (including second cluster if defined) in pytest report header when running testsuite with make targets. This string is taken from CatalogSource object and in case the object is not found, no string will be printed. In the future this can be expanded for Kind setup, or doing smarter job in finding the right CatalogSource.

This can make debugging easier and is mostly helpful for figuring out nightly build version as the version string in Kuadrant subscription object is "0.0.0".

azgabur commented 2 months ago

Will the image show up on report portal?

In this state no, as the pytest report header is not included in the junit export, I am investigating how to add this.

azgabur commented 2 months ago

I tried adding this fixture which adds property info to junit file but report portal is not displaying it. Maybe in the future when report portal starts displaying this additional info we can add this, otherwise I did not find any other simple way.

@pytest.fixture(scope="session", autouse=True)
def log_global_env_facts(record_testsuite_property):
    """Adds Kuadrant version in property of testcase in junit."""
    for i, image in enumerate(kuadrant_version()):
        record_testsuite_property(f"KUADRANT_{i}", image[0])
azgabur commented 2 months ago

^ I added just the image info to metadata data structure which will be displayed in "Environment" section when generating html report.

trepel commented 2 months ago

@azgabur I tried using html=1 make ./testsuite/tests/singlecluster/gateway/test_basic.py and junit=1 make ./testsuite/tests/singlecluster/gateway/test_basic.py. Former worked (even tried with additional clusters), latter did not work for some reason. Any ideas what might have gone wrong?

azgabur commented 2 months ago

junit=1 make ./testsuite/tests/singlecluster/gateway/test_basic.py

In the end I did not implement the junit property carrying the kuadrant image string as the Report portal did not display it so thats why its not included. But just by adding this function (https://github.com/Kuadrant/testsuite/pull/511#issuecomment-2290971355) to top level conftest it can be later added if needed.

trepel commented 2 months ago

Ah, ok, sorry, I did not get that. If this meant to be about html only, it looks good to me then.