haf / expecto

A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Apache License 2.0
663 stars 96 forks source link

add options to list focused tests #395

Closed vilinski closed 3 years ago

vilinski commented 3 years ago

following PR to enhancement #392

Proposed changes and usage:

P pending_test.case.one 
F focused_test.case.two
N normal_test.case.three 
vilinski commented 3 years ago

Hope I understand you right.

also please check wording, my english is not that strong

haf commented 3 years ago

Build needs fixing and there's a printf: wouldn't it be better to go through the logger like everywhere else? How available is the logger where you've made your change?

vilinski commented 3 years ago

Build needs fixing

Oh, I see, there is a bug in NuGet/paket which I'm also struggling with. https://github.com/NuGet/Home/issues/9786 Workaround with commenting out a line in paket.Restore.targets didn't work. No clue how to fix this until it's done in NuGet

and there's a printf: wouldn't it be better to go through the logger like everywhere else? How available is the logger where you've made your change?

printfn is a shadowed function which was used here before. Impl.logger is also awailable from listTests function. But I'm failed to figure out how to use it right. Is logger.info a right thing to do here? My last commit results in no output at all. actually it does if I append printfn after log or Async.RunSynchronously. Seems it's not get flushed before program ends.

haf commented 3 years ago

@vilinski You can do an Async.RunSynchronously for now; the most important bit is that it goes through the logger so we can quelch output, should we wish to. I'm not too happy on shadowed stdlib functions.

vilinski commented 3 years ago

Async.RunSynchronously and logWithAck doesn't help. it only works with printfn "" at end ... An alternative would be to return the string from listTests instead of logging it. also all other occurences of shadowed printfn are in the calling function runTestsWithArgsAndCancel.

haf commented 3 years ago

The solution here is not to add a "fake" printf, but to hunt down why the logger's flush doesn't work. I'm afraid I don't have time to spare right now, but I would very much appreciate if you feel up for it.

It's most likely due to the coloured console + progress printer interaction; look for the Monitor.Enter calls and the buffering by overriding Console's Stdout

vilinski commented 3 years ago

tried to look into code and debug. Monitor.Enter and Monitor.Exit calls in progress printer are looking plausible for me.

however noticed that the Logger.log and Logger.logWithAck are identical in all leaf implementations LiterateConsoleTarget, TextWriterTarget, OutputWindowTarget

I have added a flush into LiterateConsoleTarget:

$ dotnet run --framework netcoreapp3.1 --  --list-tests pending
[00:37:38 INF] 
P findFastest.find fastest sleep
P all focused tests/ignored by attribute
P performance.same function is faster should fail
P findFastest.find fastest lo
P findFastest.find fastest hi
 <Expecto>

Comparet to no output at all for the above command it's a progress. But can you please confirm this is the right way to do? If not I'm afraid I can't understand how to fix this either.

haf commented 3 years ago

I think we can live with that flush call; if all other tests are passing. What happens if you disable progress printing? Flush still works, but is a no-op?

vilinski commented 3 years ago

I think we can live with that flush call; if all other tests are passing.

No failing tests in Expecto.Tests. What about other two logWithAcks? Do they also need a flush?

What happens if you disable progress printing?

--no-spinner option? I see the same output without progress spinner in either cases, maybe because there is just one log call for --list-tests

Still don't know hot to fix the build though

haf commented 3 years ago

/rebase

haf commented 3 years ago

Thanks @vilinski ! 🎊