search

japgolly / nyaya

Random Data Generation and/or Property Testing in Scala & Scala.JS.
GNU Lesser General Public License v2.1
185 stars 19 forks source link

Doc request: compare and contrast to ScalaCheck #18

Open SethTisue opened 8 years ago

SethTisue commented 8 years ago

I would like to understand better how this compares to ScalaCheck. I suspect other prospective users would like that as well.

Two possible models for such a doc section: https://github.com/travisbrown/circe#why, http://rapture.io/blog/rapture-manifesto

japgolly commented 8 years ago

When I initially started documenting this project I did just that, but then it just came off as a hyper-negative rant again ScalaCheck. ScalaCheck has done good things and that's not the message I want to convey, so I deleted it and rewrote it trying to be less comparative and instead highlight what I think the benefits of Nyaya are.

My conclusive thought was that between the front-page and FEATURES.md it should be obvious to someone familiar with ScalaCheck that this does many additional things, and differently.

Do you think that approach isn't enough?

SethTisue commented 8 years ago

it should be obvious to someone familiar with ScalaCheck

perhaps, but then that restricts your potential audience greatly. I've used ScalaCheck for many years, but without deep knowledge of its full feature set or its limitations, so the differences aren't obvious to me.

I don't think the tone has to be "hyper-negative rant" — I think the Circe thing I linked to is a good of example of avoiding that while still doing a direct and informative comparison.

if you need help wordsmithing something to get the right tone, maybe open a PR with your draft and invite comment?

SethTisue commented 8 years ago

(I see there's https://github.com/scalaprops/scalaprops too...)

japgolly commented 8 years ago

Very fair point. I'll give it another try.

japgolly commented 8 years ago

Erm, well to this day, ScalaCheck frustrates the hell out of me. Nyaya definately isn't perfect and has its annoying parts that I'd like to change, but (sorry to sound arrogant here), ScalaCheck feels like prop-testing 101 which isn't very effective in big/real/large projects. Let's split it into three points. 1) Data generation. Nyaya's data generation (which is a standalone module) is much easier to use and customise the data, allows you to be much more expressive about what you generate, it's also orders-of-magnititude faster in its generation. Being standalone it's also great for stress/load testing, etc as well as just prop testing. 2) Prop testing. It was experimental so good & bad vs ScalaCheck here. Good: Faster and more efficient. Better failure reporting. Bad: Syntax more verbose, and a bit more annoying sometimes to write, than ScalaCheck. 3) Decouples testing from properties. At runtime in dev-mode I like to have complex data structures validate their properties. Testing should also be able to use the same properties.