airspeed-velocity / asv

Airspeed Velocity: A simple Python benchmarking tool with web-based reporting
https://asv.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
874 stars 180 forks source link

Show print statement output in console #1354

Closed droumis closed 11 months ago

droumis commented 11 months ago

I'm having a hard time debugging my benchmarks. I'm trying to have python print statements appear in the console but I have not been successful with any options (-e, -v). Part of the problem is that the benchmarks are not hitting an error, they just aren't doing what I expect in the context of the asv machinery.

How can I have print statements show up in the console? Thanks!

ianthomas23 commented 11 months ago

Hi @droumis!

Use asv run -q -e, which will show print statements that are in the benchmark code in the console. But you need to be careful with print statements that are in the source code of your package as asv checks out whatever branch is specified in the asv.conf.json so you can't just add some extra print statements and re-run asv, you need to check in the changes.

Alternatively, use asv run -q -e --python=same which will run the benchmarks in your current environment rather than a new virtual environment, so if you have used pip install -e . for your package you don't need to check in the changes.

droumis commented 11 months ago

Thanks @ianthomas23, looks like the issue was specific to print statements within a setup function, and that is working now, somewhat magically. Might have been a timing thing, as fixing a setup function such that it didn't return so quickly made it work. 🤷

If anyone is interested in the context: The context was that at first we were reading standard string console messages from playwright's page.on("console", callback) -> msg.args. But then I had changed the upstream package (Bokeh) to emit console messages with trace level logging (see [issue](https://github.com/bokeh/bokeh/pull/13503#issuecomment-1789816075)), which was spitting out console messages in a different form. At first, I thought I could just filter the console messages by adjusting the length of msg.arg list, but that didn't seem to be working and I could not get asv to print anything from within a setup function so it was hard to debug. What a solution seemed to be was instead of just filtering the console messages via `msg.args` by length, I use a regex pattern to filter `msg.text` for a specific string ('PlotView._actual_paint' in this case). That seems to be working now and magically asv is releasing the print statements.