fscheck / FsCheck

Random Testing for .NET
https://fscheck.github.io/FsCheck/
BSD 3-Clause "New" or "Revised" License
1.15k stars 154 forks source link

Allow asynchronous tests to run on Task<'a> #633

Closed ploeh closed 11 months ago

ploeh commented 1 year ago

That FsCheck can run asynchronous tests is one of its major advantages over competitors. It'd be nice, however, if it could run methods/actions that return Task<'a> instead of simply Task.

Particularly, with the new task computation expression, one can easily write an asynchronous (integration) test, but even if such a test body ends with an assertion, the expression has the type Task<unit>, rather than Task.

One can get around this issue by upcasting the expression with :> Task. If, however, one wishes to avoid adding an extra line of code for only that purpose, one runs into this issue.

If, currently, one tries to pass a Task<unit> to Prop.forAll if compiles fine, but fails at run time with this error message:

No instances of class FsCheck.Testable+ITestable`1[T] for type
System.Threading.Tasks.Task`1[Microsoft.FSharp.Core.Unit]

This is on FsCheck.Xunit 3.0.0-beta2.

Since Task<'a> derives from Task is seems to me that a fix should be a fairly straightforward.

If you agree that such a fix is desired, I may try my hand at it myself. (I've been away from contributing to FsCheck for years, now, so I hope I'm not biting off more than I can chew.)

kurtschelfthout commented 1 year ago

Agree that a fix is desired. Actually not sure why it doesn’t already work!

Let me know if you get stuck, happy to assist (although it’s quite busy these days, so may take a couple days to reply)

ploeh commented 1 year ago

I've now cloned the repo and executed build.sh. It looks fairly promising, but does report this error:

Script reported an error:
-> BuildFailedException: Target 'Build' failed.
-> One or more errors occurred. (Unsupported log file format. Latest supported version is 14, the log file has version 16.)
-> NotSupportedException: Unsupported log file format. Latest supported version is 14, the log file has version 16.

Is this a know issue?

kurtschelfthout commented 1 year ago

tldr: make sure you have .NET 6 SDK installed (not 7), git checkout fscheck3, git pull, build.sh and you should be good.

It's known now, in that I see it too :)

Afaik this is due to some incompatibility between fake and installed MSBuild versions. This kind of thing unfortunately happens a lot. For a project like FsCheck which is essentially untouched, the amount of churn introduced by .NET, Fake, paket and all the build paraphernelia keeps surprising me. Excuse the rant.

In any case it's moot, because since you want to modify 3.0, you need to check out (and PR to) the fscheck3 branch. It is substantially different (i.e. includes the async support...whereas master doesn't).

For that, as well as the eternal prerelease state of 3.0, there is no one to blame but myself.

In any case I managed to update the fscheck3 branch to the latest paket and fake, and after pinning to .NET 6 that builds fine, also in CI: https://ci.appveyor.com/project/kurtschelfthout/fscheck/builds/47399941

Hope this helps!

ploeh commented 1 year ago

All looks green now, thank you!

bartelink commented 1 year ago

aside: While I'm a poster child of what not to do (I'm up to -rc.11 on some prominent public things, though most people can work out the story from looking at the previous full releases being ~2y ago!) I would plead to do as I say, not as I do and make fscheck3 become main, fork off a v2 maintenance badge, and then badge the nuget 3.0.0-rc.1

There's enough info out there (e.g. @pblasucci and Mark's blogs etc) for people with appetite/aptitude for FsCheck to move on past V2. My main reasoning is that there are enough quality of life/usability fixes that the OOTB experience is simply better. And if anyone wants to complain about that, simply put an Issue on GH stating that some docs are not yet in sync with V3 - ultimately if nobody has capacity to do a real complete release with all that might entail on paper, that should not be a blocker on doing the right thing.

kurtschelfthout commented 1 year ago

@bartelink yes, I think it's time for that. I'll probably just do 3.0.0 even. If the other plans ever materialize, there'll just have to be another version.

C0DK commented 1 year ago

When will this be released? :D

kurtschelfthout commented 11 months ago

Released in 3.0.0-rc1.

Also merged fscheck3 branch to master.