search

smarie / python-pytest-harvest

Store data created during your `pytest` tests execution, and retrieve it at the end of the session, e.g. for applicative benchmarking purposes.
https://smarie.github.io/python-pytest-harvest/
BSD 3-Clause "New" or "Revised" License
61 stars 8 forks source link

If only #63

Open Midnighter opened 10 months ago

Midnighter commented 10 months ago

If only pytest-harvest had already existed when we were working on MEMOTE. We found a fairly hackish way by storing information on the test function itself and then collecting that info with a plugin, but the result_bag makes so much more sense! And it even works with xdist.

I didn't know where else to communicate besides issues, so this is just to say, your package looks awesome. I wish we had it back then. I'm now considering re-writing MEMOTE, but I don't really have the time right now.

smarie commented 8 months ago

Hello @Midnighter !

This is a late answer but your kind words are very much appreciated ! Thanks a lot for taking the time to write this.

MEMOTE is exactly the kind of usage that I had in mind when I developed pytest-harvest and pytest-cases: data science benchmarks that are reproducible by anyone. I'd be curious to see where you actually store the reference datasets - that is the piece where I do not have an elegant solution, apart from using a blob storage or a data catalog such as OpenDataSoft (maybe using https://smarie.github.io/python-odsclient/ :) )

Anyway, kudos for MEMOTE and thanks again for your message!