[Question] How would testmon be used to compare code differences between classes?

[jacksongoode]

It seems testmon is able to evaluate whether or not any of the functions within a test have changed before executing the test itself. In my work, we have a scenario where we would like to know whether or not a function's code path has changed from one state to another (basically creating a hash of the function's code and comparing it the present hash).

Is there a way to achieve this with coverage.py methods - I've scanned through but thinking about this in a testing independent context is difficult.

[tarpas]

Function and function code path sound like a different thing so I’ll assume function: yes, we compute a hash of each function representation as well as the range of line numbers for each file in process_code.py . Coverage doesn’t concern itself with functions, just lines.

How are your functions identified for you? It´s name and location? Line numbers?

On Tue, 20 Dec 2022 at 01:02, Jackson Goode @.***> wrote:

It seems testmon is able to evaluate whether or not any of the functions within a test have changed before executing the test itself. In my work, we have a scenario where we would like to know whether or not a function's code path has changed from one state to another (basically creating a hash of the function's code and comparing it the present hash).

Is there a way to achieve this with coverage.py methods - I've scanned through but thinking about this in a testing independent context is difficult.

— Reply to this email directly, view it on GitHub https://github.com/tarpas/pytest-testmon/issues/199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABBBMAJLH33S2LLAVT33LDWODZSBANCNFSM6AAAAAATD5AMTM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jacksongoode]

Thanks for the response @tarpas. I think I have the line numbers for each process, but the (helper) functions that a function itself calls are tricky to determine? They are sometimes given by the opcode but not always. Does testmon recursively go through functions called by other functions?

[tarpas]

Does testmon recursively go through functions called by other functions? That's the task of the coverage.py which testmon uses. (and I wouldn't call it recursively, the result is a flat list of all executed lines) I feel you'll expect more from coverage + method checksumming than it can actually provide :/

Did you carefully read https://testmon.org/determining-affected-tests ? If there is anything unclear there I would be happy to get questions and try to explain. :)

[jacksongoode]

Yes, I did see that - I started digging a bit into process_code.py and (perhaps this is out of scope) but the way I currently see testmon operating is on files where tests are found themselves (through coverage.py). My desire would be to selectively generate a checksum of a function once run (I guess starting coverage before and stopping after).

def a():
    print('hello')

which returns some checksum using a function cov_checksum(a) and find that if I change the function to:

def a():
   print('goodbye')

a helper function compare_checksum(a) would return False indicating that the function has changed since it's last run.

This is sort of my imagined use-case. I recognize that while this is a testing specific implementation, I haven't found any other project that attempts to compare the full execution of a function from one state to another. Do you feel that some of the functions you've written within testmon might be exposed to accomplish this task?

I guess in theory you could make a test:

def test_func_a():
    a()

And then trigger testmon to see if that test needs to be rerun at any given time given potential changes made to a.

[jacksongoode]

@tarpas Just wondering if this would be possible given the constraints of how coverage.py? I'm attempting to de-structure the methods but am running into functions that are decorated upon pytest.