search

Test-More / TB2

Test::Builder version 2, the next generation of building testing modules in Perl
Other
1 stars 0 forks source link

100 candidates having bad experiences with 1.005000_002 #41

Open schwernbot opened 10 years ago

schwernbot commented 10 years ago

From: @andk Date: Thursday Dec 29, 2011 at 11:44 GMT Orig: https://github.com/Test-More/test-more/issues/249

The following list is compiled from results of the regression testing done on analysis.cpantesters.org. In column 2 there is the R^2 denoting the quality of the finding. The closer it is to 1 the less disturbing influences have been identified, and the more likely it is that the measured effect is relevant. In column 3 there is the Theta, the direction and amplitude of the influence. I've filtered column three for values < 0.75 and column 2 to values > 0.35. Arbitrary decisions. 103 lines seemed more than enough to me and indeed the last items in the list are already lots of influences mixed in.

Hope this is interesting to you.

XML-LibXML-SAX-ChunkParser-0.00004            1.000        -1.0000
URI-Escape-XS-0.08                            1.000        -1.0000
UNIVERSAL-DOES-0.004                          1.000        -1.0000
Test-Script-Shebang-0.02                      1.000        -1.0000
Test-More-Prefix-0.001                        1.000        -1.0000
Test-Group-0.19_01                            1.000        -1.0000
Test-Flatten-0.07                             1.000        -1.0000
Test-File-1.28_01                             1.000        -1.0000
Test-Classy-0.09                              1.000        -1.0000
Test-Able-0.11                                1.000        -1.0000
RedisDB-0.24                                  1.000        -1.0000
Proc-Lite-0.09                                1.000        -1.0000
Proc-Lite-0.09_01                             1.000        -1.0000
Proc-Guard-0.06                               1.000        -1.0000
Plack-App-Proxy-Selective-0.10                1.000        -1.0000
Net-Google-DataAPI-0.25                       1.000        -1.0000
Log-Sigil-0.15                                1.000        -1.0000
Log-Dispatch-Config-TestLog-0.02              1.000        -1.0000
Locale-Maketext-1.21                          1.000        -1.0000
IO-Prompt-Simple-0.04                         1.000        -1.0000
HTML-LinkFilter-0.03                          1.000        -1.0000
HTML-Barcode-Code93-0.10                      1.000        -1.0000
Goose-0.004                                   1.000        -1.0000
Game-TextPacMonster-0.03                      1.000        -1.0000
DBM-Deep-2.0004                               1.000        -1.0000
DBIx-Simple-Inject-0.03                       1.000        -1.0000
DBI-Easy-0.23                                 1.000        -1.0000
Date-Holidays-PAK-0.06                        1.000        -1.0000
Data-Monad-0.02                               1.000        -1.0000
CGI-Emulate-PSGI-0.14                         1.000        -1.0000
Carp-1.23                                     1.000        -1.0000
App-mgen-0.16                                 1.000        -1.0000
AnyEvent-Twitter-0.62                         1.000        -1.0000
AnyEvent-Groonga-0.07                         1.000        -1.0000
Algorithm-Kmeanspp-0.03                       1.000        -1.0000
Algorithm-KernelKMeans-0.03_02                1.000        -1.0000
Algorithm-FuzzyCmeans-0.02                    1.000        -1.0000
POE-Component-ResourcePool-0.04               0.969        -1.0000
Test-SFTP-1.10                                0.957        -1.0000
Test-Dependencies-0.12                        0.942        -1.0000
Aspect-1.01                                   0.873        -1.0000
IO-BindHandles-0.005                          0.872        -1.0000
Module-Release-2.06_01                        0.870        -0.9946
Role-Basic-0.12                               0.865        -1.0000
WebService-8tracks-0.01                       0.854        -1.0000
Teng-0.14_04                                  0.853        -0.9953
ShipIt-Step-ChangeAllVersions-0.007           0.853        -1.0000
Data-ModeMerge-0.27                           0.810        -1.0000
WWW-Search-MSN-0.0106                         0.789        -1.0000
DBIx-CouchLike-0.16                           0.781        -1.0000
Data-YUID-0.06                                0.771        -1.0000
POEx-Role-SessionInstantiation-1.102610       0.759        -1.0000
Teng-0.14_03                                  0.747        -1.0000
Devel-Cover-Report-Clover-0.31                0.742        -1.0000
DBIx-Class-0.08196                            0.742        -1.0000
Bison-0.05                                    0.729        -1.0000
Makefile-DOM-0.006                            0.723        -1.0000
Backed_Objects-1.0                            0.713        -1.0000
CGI-Session-ID-crypt_openssl-1.01             0.711        -1.0000
Barcode-DataMatrix-0.03                       0.693        -1.0000
HTML-GUI-0.04                                 0.680        -0.9545
Moose-2.0401                                  0.665        -1.0000
XSLoader-0.15                                 0.664        -1.0000
Parallel-ForkControl-0.5                      0.663        -1.0000
Test-Dist-0.01                                0.641        -1.0000
KiokuDB-Backend-Files-0.06                    0.630        -1.0000
Plack-0.9985                                  0.621        -1.0000
Furl-0.38                                     0.610        -1.0000
Module-Build-Bundle-0.07                      0.600        -1.0000
Spawn-Safe-2.001                              0.596        -1.0000
MooX-Option-0.3                               0.593        -0.8889
Parse-HP-ACU-0.03                             0.579        -1.0000
DBIx-Skinny-Pager-0.11                        0.571        -1.0000
Alien-SVN-v1.6.12.1                           0.556        -1.0000
Parse-CSV-Colnames-0.04                       0.544        -1.0000
DBIx-Skinny-0.0742                            0.543        -1.0000
IPC-Cmd-0.72                                  0.528        -1.0000
Catalyst-Action-REST-0.94                     0.521        -0.9759
ASP4x-Captcha-Imager-0.003                    0.507        -1.0000
Ledger-Parser-0.03                            0.493        -0.8571
WiX3-0.011                                    0.455        -1.0000
signatures-0.06                               0.455        -1.0000
MediaWiki-Bot-v3.4.2                          0.448        -1.0000
WebService-FreeDB-0.78                        0.445        -1.0000
Pod-HTMLEmbed-0.04                            0.445        -1.0000
Test-Story-0.07                               0.440        -1.0000
Acme-Perl-VM-0.006                            0.437        -1.0000
Data-ObjectDriver-0.09                        0.431        -1.0000
Data-MessagePack-0.39                         0.426        -0.9653
Event-ExecFlow-0.64                           0.421        -1.0000
Log-Any-Adapter-Mojo-0.04                     0.418        -1.0000
Encode-JP-Mobile-0.27                         0.410        -1.0000
Net-API-Gett-0.01                             0.391        -0.7812
CPAN-Mini-Extract-1.21                        0.391        -1.0000
BSON-0.06                                     0.390        -1.0000
forks-0.34                                    0.386        -1.0000
Catalyst-View-JavaScript-0.995                0.382        -1.0000
App-Cpan-1.57                                 0.373        -1.0000
Mojolicious-Plugin-HamlRenderer-1.0           0.360        -1.0000
PLP-3.23                                      0.358        -1.0000
MooseX-Daemonize-0.13                         0.358        -1.0000
Ukigumo-0.01                                  0.356        -1.0000
IO-Async-Loop-AnyEvent-0.03                   0.356        -1.0000

Oh, and splitting into individual tickets is left as an exercise:)

schwernbot commented 10 years ago

From: @andk Date: Thursday Dec 29, 2011 at 11:46 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-3300001

I've filtered column three for values < 0.75

s/0.75/-0.75/

schwernbot commented 10 years ago

From: @schwern Date: Friday Dec 30, 2011 at 20:04 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-3316567

Thanks! Uhh... wow! Yes, this is very interesting and useful.

I was hoping they'd mostly be the fault of broken Test::* dependencies. Nope... they all seem to be a bit different. Poop.

schwernbot commented 10 years ago

From: @doherty Date: Tuesday Feb 28, 2012 at 04:17 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-4209365

MediaWiki-Bot-v3.4.2 is almost certainly not failing because of TB2. It is almost certainly failing because I have bugs in my test suite and/or the live site we test against has gone funky again.

schwernbot commented 10 years ago

From: @petdance Date: Tuesday Feb 28, 2012 at 20:24 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-4224623

Suggestion/request: Rerun it with our CPAN IDs on it. Make it easy for authors to find their names and problems without having to scan the listing of 100.

schwernbot commented 10 years ago

From: @shlomif Date: Friday Mar 16, 2012 at 11:38 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-4538049

I agree with petdance - the CPAN IDs would be useful.

schwernbot commented 10 years ago

From: @schwern Date: Friday Mar 30, 2012 at 15:32 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-4844832

URI::Escape::XS appears to be a false positive. I can't get it to fail on 1.5.0a2.

schwernbot commented 10 years ago

From: @schwern Date: Friday Mar 30, 2012 at 16:34 GMT Orig: https://github.com/Test-More/test-more/issues/249#issuecomment-4846181

UNIVERSAL::DOES has been fixed by 78f0c977ca3b79a7a33592557b77b4f14ca61f32