JetBrains / lincheck

Framework for testing concurrent data structures
Mozilla Public License 2.0
564 stars 33 forks source link

Make parameter generators smarter #149

Closed ndkoval closed 1 year ago

ndkoval commented 1 year ago

We need custom parameter generators in rare cases, most of which are for testing concurrent Maps or similar data structures. To read/update the same key with higher probability, users often specify a small range for this parameter. Yet, the range size should also depend on the scenario size (the bigger scenario, the more different values).

Do we really need customizing parameters for such scenarios? I believe "no." We can solve the issue more smartly: with some constant probability (e.g., 50%), Lincheck could take an already used value, generating a new one otherwise.

Let's investigate whether this solution fits the practical needs and eliminate custom parameter generators if so.

alefedor commented 1 year ago

There are cases when parameters are not simple integers in some range (e.g., when there are certain requirements on them) and when parameters are of non-primitive types, so custom parameter generators can not be removed.

What I see is:

  1. Generators that re-use generated values with some probability to encourage interesting scenarios as default generators.
  2. Uniform generators (as before) as an option.
  3. Option to define custom generators (as before).
ndkoval commented 1 year ago

Hi @alefedor, thanks for the comment. Could you please provide use cases for (2) and (3)?

We are working on a new API, and supporting custom parameter generators is quite painful. Therefore, we are considering removing them entirely and adding them back when requested. However, this is still under discussion.

ndkoval commented 1 year ago

Let's keep custom generators for now, but use the suggested strategy in the default ones.