search

CFMTech / pytest-monitor

Pytest plugin for analyzing resource usage during test sessions
MIT License
173 stars 16 forks source link

Write monitor-output of tests to console #46

Open danielhuppmann opened 2 years ago

danielhuppmann commented 2 years ago

Is your feature request related to a problem? Please describe. Inspecting the database after every test-run to get the output of pytest-monitor seems to be quite a hassle.

Describe the solution you'd like Looking at pytest-benchmark, they write a summary of the output to the console, see screenshot below.

Describe alternatives you've considered None.

Additional context None.

js-dieu commented 2 years ago

Hello

Thanks for the proposition! Indeed, extracting data is a pain .... I see two points:

  1. metrics extracts for exploitation
  2. metrics summary

Point number 1 can be addressed through the monitor-server-api (see https://github.com/CFMTech/monitor-server-api.git)

However, point 2 needs a development. Should we propose a report with top consumers on a given resource or just dumps the output ? I believe a full report in someone's terminal is not a good idea... but we can definitely report for a given number of test given an axis (cpu, memory or time)

danielhuppmann commented 2 years ago

Thanks for the pointer to monitor-server-api, will take a deeper look - though this seems like a more sophisticated stack than what I had in mind...

About the metrics summary, yes, this is tricky... One thought, though: the main advantage why I would rather use pytest-monitor instead of pytest-benchmark is that is reports across multiple dimensions - so having a summary-output of only one dimension (cpu/memory/time) defeats its main advantage.

js-dieu commented 2 years ago

One thought, though: the main advantage why I would rather use pytest-monitor instead of pytest-benchmark is that is reports across multiple dimensions - so having a summary-output of only one dimension (cpu/memory/time) defeats its main advantage.

Got your point 👍🏻

joukewitteveen commented 2 years ago

Would something along the lines of pytest_terminal_summary be an option, where we list the tests along with their duration and peak memory usage?

joukewitteveen commented 2 years ago

I've made a little proof-of-concept: pytest-resource-usage. Feel free to copy anything from that experiment.

js-dieu commented 2 years ago

Hello @joukewitteveen

Thanks for the PoC. You are not the first one requesting this output. I was lacking time lately, but I'll make the implementation next week. I've seen the code you propose, I'll surely inspire myself from this. Thanks for that! 👍🏻