Failing bitrotted tests

Raku / whateverable

🤖 Different IRC bots that operate on a bunch of prebuilt Rakudo versions

https://gist.github.com/Whateverable

GNU Affero General Public License v3.0

18 stars 14 forks source link

Failing bitrotted tests #340

Closed AlexDaniel closed 4 years ago

AlexDaniel commented 6 years ago

Was doing release prep for rakudo and realized that many whateverable test files no longer pass (although they used to a long time ago). I can't spot any rakudo issues, mostly it's just bitrot in the bots themselves. Most commonly used bots seem to be OK and their tests pass, but other bots need more love.

[ ] benchable.t
[x] bisectable.t
[ ] bloatable.t
[ ] committable.t
[ ] coverable.t
[x] evalable.t
[ ] greppable.t
[ ] nativecallable.t
[x] notable.t
[ ] quotable.t – requires irc log files which are not public
[ ] releasable.t – fails due to lack of mocks (it fetches real data which changes all the time)
[ ] statisfiable.t
[x] tellable.t
[x] unicodable.t

AlexDaniel commented 6 years ago

@MasterDuke17 I think issues with Benchable are because of your changes, can you take a look? FWIW benchable should work perfectly locally, so you don't even need to do anything on the server as long as you're running some 64-bit linux.

taboege commented 5 years ago

I also see this on current master from commitable.t

not ok 108 - “releases” query
# Failed test '“releases” query'
# at /tmp/whateverable-repo/xt/lib/Testable.pm6 (Testable) line 118
# expected: [/^ $($t.our-nick) ‘, ¦releases (’\d+‘ commits): «Perl 6 (6.c)␤»’ $/]
#  matcher: 'infix:<~~>'
#      got: ["testable739686, https://whatever.able/fakeupload"]
# Test failed. Stopping test suite, because PERL6_TEST_DIE_ON_FAIL environmental variable is set to a true value.
# Bot process finished (exit code=0, signal=1)

AlexDaniel commented 4 years ago

Same as https://github.com/Raku/whateverable/issues/377, this is now a bit outdated.