mbj / mutant

Automated code reviews via mutation testing - semantic code coverage.
Other
1.96k stars 153 forks source link

Dependence on fork() #91

Closed L2G closed 10 years ago

L2G commented 11 years ago

At present I am using Ruby on Windows, which has no support for fork().

I'm setting out to find a way to make Mutant work without relying on fork(). One possibility I'm eyeing for Windows is the win32-process gem.

I just thought I'd call it out here in case anyone already has something like this in the works.

mbj commented 11 years ago

@L2G To put your request into the greater picture:

Mutant needs to kill mutations in a sandbox. This sandbox must have the following properties:

a) Each mutation of global state must be inversible b) Sandbox must spin up and down fast c) There should be the possiblity of parallel active sandboxes.

Reasons behind the inversible property: Mutant can generate unpredictable global state. Your test cannot cleanup all global state mutations mutant could have done to your test. Hence the need for a sandbox. If a global state mutation, such as a constant, leaks into later tests, these later tests results could be invalidated.

From what I know about the windows platform:

a) A win32-process "fork" would properly isolate mutations of global state. b) I heard it is slow, but I never programmed for windows. c) Dunno about resource consumption, and multiple sub-processes on windows.

I talked to @headius once about jruby and spinning up multiple environments there, so maybe the sandboxing does not have to be archived via system processes and can be done on the VM/language level.

So if you wanna work on windows support, I'll not stop you. I'd happily merge any windows supporting PR. But for myself I'll not work on this and focus on jruby support instead, this would result in windows support for free.

I'll leave this issue open as feature request.

L2G commented 11 years ago

Sounds good.

I made a fork to start playing with this. I freely admit that I have a very poor understanding of Ruby threads as well as threads in general. :smile_cat:

My changes so far are ad hoc and very naive. But I was able to get mutant's own integration tests to pass, and I was able to successfully(?) mutate code in another project with it--all on Windows 7 with MRI Ruby 1.9.3p392. So I'm encouraged by that (rightly or wrongly).

I think it passes your sandbox conditions (a) and (b), but (c) is debatable.

mbj commented 11 years ago

@L2G Please open your PR earily, I'm happy to help where I can.

mbj commented 10 years ago

Update: I think sonner than later mutant will not monkeypatch the mutation in anymore, because monkeypatching has some negative side effects.

I plan to load the mutated code directly. For this reason I'll spawn a pristine ruby interpreter for each mutation. Porting it to non fork() platforms will be more easy. If you add parallel mutation killers (via many kill-workers) it'll maybe get fast enough.

L2G commented 10 years ago

This issue is not a critical one to me anymore, as I'm no longer restricted to working in Windows (I'm with a different company now). But your plans sound good to me.

mbj commented 10 years ago

@L2G I'll not try to support windows directly. Okay as an accident. I'll leave this ticked open as a feature suggestion.

mbj commented 10 years ago

I'll close that one now. Its unlikely mutant will run on a platform without fork(2).