laike9m / Cyberbrain

Python debugging, redefined.
http://bit.ly/cyberbrain-features
MIT License
2.51k stars 159 forks source link

Should only create gold data files if cyberbrain ran successfully #149

Open laike9m opened 2 years ago

laike9m commented 2 years ago

Currently, even if cyberbrain fails, new json files will be created.

Relevant code: https://github.com/laike9m/Cyberbrain/blob/master/test/conftest.py#L128-L141

2000prath commented 2 years ago

How do we know whether cyberbrain ran successfully? if tracer_events==golden_filepath returns True indicates it ran successfully?

laike9m commented 2 years ago

@victorjzsun is supposed to improve this, and can provide more context

victorjzsun commented 2 years ago

yield will return whether the test ran successfully or unsuccessfully, we then immediately create/compare golden files with tracer_events.You'd need to find a way to check if it ran successfully, for example, using a hook or doing something like pytest-golden plugin and use a separate command for updating. I haven't done too much investigation regarding this but it might be hard to ensure every test passes before writing all of the golden files