haf / expecto

A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Apache License 2.0
663 stars 96 forks source link

Q: How to reflect FsCheck related CLI option values to the custom configuration? #335

Open objectx opened 4 years ago

objectx commented 4 years ago

There seem no easy ways to inject settings from CLI when using a custom FsCheck configuration.

FsCheckConfig.defaultConfig does not reflect the values from CLI options (i.e. --fscheck-max-tests etc...), So the code following uses the default maxTests (=200) even when --fscheck-max-tests 10 supplied.

let config = { FsCheckConfig.defaultConfig with arbitrary = [typeof<MyClass>] }

let test = testPropertyWithConfig config <| fun () -> …

Please tell me how to inject CLI settings when using a custom FsCheckConfig if possible.

haf commented 4 years ago

I believe they'd (the command-line-option DU cases) have to be merged. Do you feel up for adding that support? If you want to add it, you could consider writing about how you're considering architecting the new code in this issue for a bit of back-and-forth.

objectx commented 4 years ago

Once using the custom Arbitrary, We lost parameters from the CLI option. I've struck this when testing complex tree structure (required 5min to finish with using the default maxTest settings and no way other than recompiling to change settings). This is the reason for creating this issue.

One possible design is like Fake's MSBuild module (cf. MSBuild.fs.) Using a function FsCheckConfig -> FsCheckConfig as a modifier.

haf commented 4 years ago
objectx commented 4 years ago

Seems to be good enough to reflect CLI settings, and solves problems when using a custom arbitrary.