search

jumaffre / cimetrics

Track your metrics in GitHub PR to avoid unwanted regressions
MIT License
16 stars 2 forks source link

Add collapsed markdown table #117

Closed letmaik closed 3 years ago

letmaik commented 3 years ago

It's sometimes useful to look at the raw data.

cimetrics commented 3 years ago

letmaik/md-table@685 aka 20210402.14 vs main ewma over 11 builds from 637 to 669

Click to see table | build_id | build_number | Throughput (tx/s) ^ | Latency (ms) | Peak working set size (GB) | Accuracy (%) ^ | Error rate (%) | Memory fragmentation (%) | CPU usage (%) | New metric (U) | |-----------:|:---------------|----------------------:|---------------:|-----------------------------:|-----------------:|-----------------:|---------------------------:|----------------:|-----------------:| | 637 | 20210205.1 | 48261 | 10 | 46 | 0.68 | 0.05 | 0.02 | 0.28 | 189 | | 638 | 20210205.2 | 50232 | 11 | 51 | 0.69 | 0.04 | 0.019 | 0.32 | 101 | | 646 | 20210205.10 | 47148 | 10 | 55 | 0.72 | 0.05 | 0.02 | 0.31 | 129 | | 647 | 20210205.11 | 49154 | 10 | 49 | 0.68 | 0.05 | 0.022 | 0.32 | 197 | | 652 | 20210205.16 | 47145 | 9 | 52 | 0.69 | 0.06 | 0.02 | 0.3 | 100 | | 653 | 20210205.17 | 50877 | 10 | 55 | 0.7 | 0.04 | 0.022 | 0.28 | 100 | | 655 | 20210326.1 | 46562 | 10 | 51 | 0.69 | 0.06 | 0.021 | 0.31 | 155 | | 658 | 20210326.4 | 50718 | 9 | 53 | 0.68 | 0.05 | 0.021 | 0.29 | 128 | | 660 | 20210326.6 | 46897 | 9 | 48 | 0.73 | 0.05 | 0.022 | 0.28 | 133 | | 664 | 20210401.3 | 49393 | 11 | 53 | 0.73 | 0.05 | 0.019 | 0.29 | 188 | | 669 | 20210401.8 | 51450 | 9 | 54 | 0.72 | 0.06 | 0.022 | 0.29 | 191 |

images

letmaik commented 3 years ago

Hm... the Actions job fails due to DNS problems connecting to Cosmos DB and the DevOps job doesn't even trigger now. Oh well...

letmaik commented 3 years ago

OK, so the build number is not really working yet, a bit hard to test locally, but I think it would be nice to have it since that's what's shown in the plots and what you see in CI.

achamayou commented 3 years ago

Agreed!

letmaik commented 3 years ago

pandas uses tabulate for markdown formatting and by default tabulate tries to interpret strings as numbers. This can only be disabled for all columns, at which point it also disables nice formatting for floats (reasonable number of decimals). I opened an issue (https://github.com/astanin/python-tabulate/issues/121) about that since it should be possible to disable parsing for specific columns. Let's wait a bit and see if the maintainer is open to the idea. If not, then we may have to manually truncate floats in the dataframe before calling the markdown formatter.

EDIT: Hm, the formatting looks fine here, but locally I get a lot of extra decimals. I have the same pandas and tabulate versions, weird.

letmaik commented 3 years ago

Looking at the docstring, disable_numparse actually does support specifying individual columns, so all good.

EDIT: oh dear... still not working as it should... sorry about the spam

letmaik commented 3 years ago

OK, that was more fiddly than expected but it's all working now and ready to be merged.

achamayou commented 3 years ago

It looks great, thank you!