proper-testing / proper

PropEr: a QuickCheck-inspired property-based testing tool for Erlang
http://proper-testing.github.io
GNU General Public License v3.0
882 stars 167 forks source link

Add ability to limit the number of generated atoms #307

Closed Maria-12648430 closed 1 year ago

Maria-12648430 commented 1 year ago

As mentioned in #290, the usage of the atom generator (explicit or implicit) can lead to exhaustion of the atom table. While I understand that PropEr does not want to restrict itself to just afew atoms per se, it would be very useful if a user (like me) was given the choice to limit this if needed.

Maria-12648430 commented 1 year ago

When you do add some tests, you will discover that the code for the non-infinity case cannot run on older Erlang/OTP versions (e.g. 21 and 22 - I believe) because persistent_term is not available there. So, this code cannot be included as is as long as PropEr still supports these Erlang/OTP versions...

For that case, I suggest we put in a conditional that falls back to always using infinity (and ignore the setting) when used with an older OTP version that does not support persistent_term. Would that be acceptable?

EDIT: persistent_term was introduced in OTP 21.2.

kostis commented 1 year ago

EDIT: persistent_term was introduced in OTP 21.2.

OK - so then it's not that bad as I feared. In that case, yes, the conditional you suggest will do.

Maria-12648430 commented 1 year ago

I'll retract this PR for the time being. Having tried to make the existing tests work, I realized that it changes the behavior of the current atom generator to a considerable degree, especially when it comes to shrinking, which is currently tightly related to list and integer shrinking.

However, as stated in #290 and the follow-up discussion in the Erlang Forums, @juhlig's and my general opinion is that...

For those reasons, it is our opinion that the current way of atom generation is, to put it bluntly, pretty useless, and that it needs to be reconsidered and redesigned. Which is no easy task, because in a realm wherein anything is as good or bad as anything else, what can you reduce to and shrink on?