Open 4kreed opened 9 months ago
rocallahan
commented
9 months ago I have a use case where I want to record the execution of all my tests and only keep the record if the test failed (and do that programatically).
rr returns the exit code of the application you recorded. So you can write a program (e.g. a shell script) that runs rr, checks the exit code, and delete the recording if the test failed.
4kreed
commented
9 months ago Thanks for your quick response!
Right, I was wondering about a more fine grained execution, so let's say I have a tester that runs 100 tests. I'd like to record each test individually and keep the record only if the test failed (and not the recording of the execution of the entire tester).
Also, and specially, in a fuzz testing scenario, where the same code is tested multiple times based on the generated inputs by the fuzz engine (a process that can't be stopped and relaunched, since metrics like coverage would be loss), I'd like to record the execution of the code for each generated input, and again discard the recording based on the result.
Is there any mechanism that could fit for this scenario?
Again, thanks for your time, any help is much appreciated.
rocallahan
commented
9 months ago rr can't split up a recording of a single process execution to just isolate one sub-test that failed.
For fuzzing, usually people take the failing fuzzing input, run some minimization algorithm on it, and then re-record that with rr for debugging.
4kreed
commented
8 months ago Thanks again for your response and sorry for the delay in mine.
I apologize if I haven't expressed myself clearly, it is not about splitting up a recording but making recordings in a more fine tune way. The idea would be to be able to start and stop recordings by calling some public API (let's say, recordStart() and recordStop()) from the source code and thus using rr as a library.
Nevertheless, I understand from your messages that this scenario/functionality is not currently provided by rr. Based on your knowledge about the tool, do you think this is something achievable? I'm thinking about implementing this functionality myself but I'm not even sure if it is possible at all. Any comment or hint about it would be greatly appreciated.
Again, thanks for your time, I appreciate a lot your help.
4kreed
commented
7 months ago @rocallahan Did you have the chance to see my latest comment? Any thoughts about it?
Greatly appreciated :)
rocallahan
commented
7 months ago The idea would be to be able to start and stop recordings by calling some public API (let's say, recordStart() and recordStop()) from the source code and thus using rr as a library.
It probably could be done but it would be pretty hard. The especially hard part is taking a snapshot of the current state at the start of the recording and getting everything set up. Right now we rely on setting everything (e.g. seccomp filters) up in an initial process and then all other tracees inheriting the right settings, and also on setting up various things whenever there's an execve.
Would it be possible to use rr-debugger as a library? I have a use case where I want to record the execution of all my tests and only keep the record if the test failed (and do that programatically).
I'm wondering if it is possible to call
RecordCommand::run()before executing my test. Maybe this doesn't make sense but I can see a real advantage of being able to at least record by using some APIs from a library.Thanks for your time.