search

disneystreaming / smithy4s

https://disneystreaming.github.io/smithy4s/
Other
341 stars 69 forks source link

SUGGESTION: first class scalacheck generators #967

Open caoilte opened 1 year ago

caoilte commented 1 year ago

It's pretty obvious to anyone who's been using smithy4s a few minutes that automatic scalacheck generators would be pretty neat. There seems to have been some work on this eg,

https://github.com/disneystreaming/smithy4s/tree/series/0.18/modules/scalacheck/src/smithy4s/scalacheck

but it's also been suggested by @kubukoz that "it's not published, plus i don't think it handles validations".

Regardless I don't think there is any documentation for current support or a roadmap for the remaining pieces that would make it first class so @kubukoz suggested I raise this issue to track.

kubukoz commented 1 year ago

IIRC the main issue is the validations - generating things marked with constraint traits and refinements would need to either:

Option 2 is easier as it's a general solution that works for virtually all constraints. The huge problem is that it may filter out too many values and cause Scalacheck to fail because it won't be able to generate sufficient amounts of valid values.

We could special-case for the refinements we know about (length, range, regex - generating these is a tough task but might be possible, swagger UI does it albeit with some bugs), which is option 1... but I don't know how to do it for user-defined refinements: the generator would sort of have to be built into the schema, so that the SchemaVisitor that constructs generators would see it.

Baccata commented 1 year ago

@kubukoz got it right : this is simply not something I'm willing to commit to providing a solution for, because the fact that we're letting users come with their own refinement rules (with custom constraints) makes it downright impossible to derive sensible scalacheck generators in a generic fashion.

We do have a limited implementation but that we use for our own tests but it's certainly not something I want published as a consumable artifact. Feel free to copy/paste/adapt the code to fit your own usecase though.

I also think that this is one of the few usecases that actually benefits from full-auto derivation with shapeless :

The only missing bits, besides a few instances for smithy4s.Document, smithy4s.ByteArray and smithy4s.Timestamp, would be that you'd have to write logic for deriving Gen instances from newtypes. You'd be able to trust the presence of a Bijection instance between the underlying type and the new type that could be use for deriving the Gen.

Of course, going down that road won't solve the validation problem because the implicit resolver won't be able to know about the constraints that do not use refined types.

tl;dr this is a much more complex problem than it seems and I'm not comfortable trying to solve it at this time.