Mutator selection prioritization

[plotchy]

Ityfuzz selects mutators on random chance. There are two hyperparameters that can affect overall mutation:

• MAX_STACK_POW (used in libafl to determine how many times in a row to mutate - min of 2, max of 128)
• HAVOC_TIMES (used in mutators to determine how many iterations of libafl mutations to do - min of 1, max of 10)

The combination of these can produce a min of 2 or max of 1280 mutations to occur on a chosen input.

We are putting a lot of effort into mutating the input, can we do it in a smarter way? A paper like MOpt: Optimized Mutation Scheduling for Fuzzers shows that effort into tracking mutator effectiveness during a run can produce better results than always selecting randomly.

Like here the bitflip 1/1 is a really powerful mutator and should be selected more, but by default is equally selected.

They use a Particle Swarm Optimization to prioritize mutators. But i dont know what that means.. Apparently the strategy can be applied to all fuzzers.

[plotchy]

DARWIN paper has a different take on this using an evolution system.

tho oddly default AFL seems to perform very closely to DARWIN and better than MOpt. Maybe this is better to leave alone

[shouc]

A few other papers (https://www.ndss-symposium.org/wp-content/uploads/2022-162-paper.pdf, https://dl.acm.org/doi/pdf/10.1145/3510003.3510174) also have some results on this. It seems that this really depends on the target we are fuzzing. I'll try to implement something similar and see how it works on smart contracts.