Closed Bodigrim closed 1 year ago
I'd like to re-propose two suggestions I made that were put on hold for including breaking changes. I think adding --show-all-options
and a TestMeta
structure would make tasty nicer/easier to extend. Could we revive the discussion in those issues on the merits of the proposal, now that breaking changes are less of a concern?
EDIT: It would also be great to add a time frame to the initial post, so people have an idea when to submit "last calls" by.
No requests on part of tasty-tmux.
@brandonchinn178 class IsTest
remains as is since tasty-0.1
, I'm very reluctant to change it. But breaking Ingredients
is on the table, we've changed their interface already. We still aim to minimise breakage however.
It would also be great to add a time frame to the initial post, so people have an idea when to submit "last calls" by.
It depends on the activity in this thread. By default I'd wait for responses up to mid August. If someone has an important suggestion, but off for vacation and cannot contribute immediately, we can wait a little longer.
In the meantime you can test your packages against tasty
HEAD with this cabal.project
:
packages: .
source-repository-package
type: git
location: https://github.com/UnkindPartition/tasty
subdir: core
source-repository-package
type: git
location: https://github.com/UnkindPartition/tasty
subdir: hunit
source-repository-package
type: git
location: https://github.com/UnkindPartition/tasty
subdir: quickcheck
source-repository-package
type: git
location: https://github.com/UnkindPartition/tasty
subdir: smallcheck
source-repository-package
type: git
location: https://github.com/Bodigrim/tasty-bench
source-repository-package
type: git
location: https://github.com/ocharles/tasty-rerun
No special requests from me
I don't believe any of my libraries need anything, so no requests here either
There's no way to add arbitrary data to Result
. tasty-bench
and tasty-papi
resort to hacks like hiding data in resultDescription
and manipulating it. I propose adding field resultExtraData :: [Dynamic]
. This way downstream libraries could easily add any data they like to test result.
It is breaking change but it seems that impact should be limited since preferred way of construction Result
is testPassed
and similar function.
@Shimuuar I like the idea, but I wonder whether extending tasty
for the purposes of tasty-bench
/ tasty-papi
is a right thing to do. One might argue that the latter are abusing tasty
. OTOH I recall @Kleidukos asking for the same functionality, an ability to store data after test execution, but I don't recall for what purpose.
The idea was to store data produced by the test in order to render it, for later exploration. So, each test would get a TVar to a data structure that would get enhanced with the results of each test, and this data structure would be rendered in a nice format like HTML so that it can be visualised later.
@Bodigrim It could be abuse but that's what happens when library is released to public. User always have ideas. And building tree of things to do executing them, collating reports doesn't seem to be too far off. In any case it would be much better if we could abuse it without horrible hacks. Also such addition seems to be generally useful
Thank you for checking in, @Bodigrim .
The tasty-checklist
and tasty-sugar
packages are both fine with the new tasty
candidate and I don't foresee any benefits for them to changes in tasty
itself.
The tasty-expected-failure
package also works fine with the new candidate, but it does contain this workaround https://github.com/nomeata/tasty-expected-failure/blob/master/Test/Tasty/ExpectedFailure.hs#L54-L60 (see also the note starting line 66), which has further documentation in https://github.com/nomeata/tasty-expected-failure/issues/24. The summary is that the test might fail, but the subsequent Show
of the test results might differently fail (via exception) and that failure occurs outside of the WrappedTest
handling. If the Show
of the results was forced within tasty
prior to exiting the WrappedText
context then the workaround noted above would no longer be necessary.
And building tree of things to do executing them, collating reports doesn't seem to be too far off.
@Shimuuar Yeah, I often use tasty
as a general task manager. If you are interested enough, feel free to open an issue to discuss design. Not sure it will make it into tasty-1.5
, this is pretty big and I'd like a consensus from other maintainers that it's a right thing to do, but at least we get started.
In any case it would be much better if we could abuse it without horrible hacks.
Playing devil's advocate, is Dynamic
and fromDynamic
massively better than String
and readMaybe
?..
If the
Show
of the results was forced withintasty
prior to exiting theWrappedText
context then the workaround noted above would no longer be necessary.
@kquick I must admit I'm generally illiterate about Haskell exceptions. Could you possibly raise a PR to discuss it further? I imagine the patch is not massive and it could be easier to discuss a patch than its verbal description, but if I'm wrong feel free to raise an issue instead.
Playing devil's advocate, is Dynamic and fromDynamic massively better than String and readMaybe?..
It is! Maybe not massively. 1) it doesn't break existing printers of results. 2) It actually composes so it's possible to mix tests from different libraries which both want to add some data to Result.
I think it's easier to discuss concrete code, so I made draft PR #381
Unless there are specific objections, I'll cut a release candidate next week.
Apologies for not responding to your request for a patch: got swamped and forgot. Looking into this now, it appears that the concern was addressed in https://github.com/UnkindPartition/tasty/commit/fcfccc8ea991583d649f5129b83660d6cee38d8b , so that addresses my stated concern and thumbs-up for a release.
@ocharles @andreasabel could you please grant me upload rights for tasty-hunit
, tasty-smallcheck
and tasty-quickcheck
? I'd like to publish release candidates.
@Bodigrim Added to all three.
It's been almost 3 years since
tasty-1.4
release, and we accumulated plenty of contributions waiting for a major release. It would be great to releasetasty-1.5
soon, but before we proceed with an RC, I'd like to issue the last call for contributions. Ideally oncetasty-1.5
is out, we'll refrain from releasing breaking changes for another 2-3 years, so if there is anything you'd like to change let's change it now.CC @brandonchinn178 @gwils @nomeata @ocharles @kquick @jstolarek @mitchellwrosen @newhoggy @tarleb @barrucadu @414owen @Shimuuar @frasertweedale on behalf of various
tasty-*
packages.