search

beyondgrep / ack2

**ack 2 is no longer being maintained. ack 3 is the latest version.**
https://github.com/beyondgrep/ack3/
Other
1.48k stars 140 forks source link

Ubuntu Artful doesn't contain ack (nor ack-grep) #652

Closed n1vux closed 5 years ago

n1vux commented 7 years ago

discussion at https://bugs.launchpad.net/ubuntu/+source/ack/+bug/1707979

may be an issue between Debian and Ubuntu, but in case we need to fix something, opening a ticket here for tracking

Affected: ack 2.14 removed (to allow Perl upgrade?? why?) and 2.18 proposed but not added to Ubuntu Artful 17.10 (because test failures)

xtaran commented 6 years ago

This is very likely an Ubuntu-specific issue as the test suite of ack fails on Ubuntu, but not on Debian, for the very same ack package. It's though still unclear why.

petdance commented 6 years ago

Thanks for the tip, @xtaran. It looks like the failing test was a spelling issue. I think I'm going to release a new version of ack just to get past that.

stephenostermiller commented 6 years ago

The best workaround that I've found is to install it from CPAN on Ubuntu 17.10 Artful:

sudo apt install cpanminus
sudo cpan install App::Ack

This is the same workaround I posted to the related question on Ask Ubuntu: https://askubuntu.com/questions/972083/unable-to-find-ack-in-ubuntu-17-10-repositories

petdance commented 6 years ago

Many alternatives at https://beyondgrep.com/install/, including downloading the all-in-one-file version and dropping it in your ~/bin or wherever.

xtaran commented 6 years ago

@petdance: I'm sorry, but I don't see any relation between commit 66009a140a9d05302546bcd4c1d85fd5d66388f2 and e.g. this test suite failure (linked indirectly from https://autopkgtest.ubuntu.com/packages/ack):

t/ack-f.t .................... 
1..6
# Subtest: ack_sets_match( DEFAULT_DIR_EXCLUSIONS )
    1..2
    ok 1 - Should have no output to stderr: ack -f t/swamp
    # Subtest: sets_match( DEFAULT_DIR_EXCLUSIONS )
        1..1
        # Subtest: lists_match( DEFAULT_DIR_EXCLUSIONS )
            1..1
            not ok 1 - DEFAULT_DIR_EXCLUSIONS

            #   Failed test 'DEFAULT_DIR_EXCLUSIONS'
            #   at t/Util.pm line 412.
            # +----+-------+----+---------------------------------------------+
            # | Elt|Got    | Elt|Expected                                     |
            # +----+-------+----+---------------------------------------------+
            # |   0|[      |   0|[                                            |
            # *   1|  '-'  *   1|  't/swamp/0',                               *
            # |    |       *   2|  't/swamp/CMakeLists.txt',                  *
            # |    |       *   3|  't/swamp/Makefile',                        *
            # |    |       *   4|  't/swamp/Makefile.PL',                     *
            # |    |       *   5|  't/swamp/MasterPage.master',               *
            # |    |       *   6|  't/swamp/Rakefile',                        *
            # |    |       *   7|  't/swamp/Sample.ascx',                     *
            # |    |       *   8|  't/swamp/Sample.asmx',                     *
            # |    |       *   9|  't/swamp/c-header.h',                      *
            # |    |       *  10|  't/swamp/c-source.c',                      *
            # |    |       *  11|  't/swamp/crystallography-weenies.f',       *
            # |    |       *  12|  't/swamp/example.R',                       *
            # |    |       *  13|  't/swamp/file.bar',                        *
            # |    |       *  14|  't/swamp/file.foo',                        *
            # |    |       *  15|  't/swamp/fresh.css',                       *
            # |    |       *  16|  't/swamp/groceries/another_subdir/fruit',  *
            # |    |       *  17|  't/swamp/groceries/another_subdir/junk',   *
            # |    |       *  18|  't/swamp/groceries/another_subdir/meat',   *
            # |    |       *  19|  't/swamp/groceries/dir.d/fruit',           *
            # |    |       *  20|  't/swamp/groceries/dir.d/junk',            *
            # |    |       *  21|  't/swamp/groceries/dir.d/meat',            *
            # |    |       *  22|  't/swamp/groceries/fruit',                 *
            # |    |       *  23|  't/swamp/groceries/junk',                  *
            # |    |       *  24|  't/swamp/groceries/meat',                  *
            # |    |       *  25|  't/swamp/groceries/subdir/fruit',          *
            # |    |       *  26|  't/swamp/groceries/subdir/junk',           *
            # |    |       *  27|  't/swamp/groceries/subdir/meat',           *
            # |    |       *  28|  't/swamp/html.htm',                        *
            # |    |       *  29|  't/swamp/html.html',                       *
            # |    |       *  30|  't/swamp/incomplete-last-line.txt',        *
            # |    |       *  31|  't/swamp/javascript.js',                   *
            # |    |       *  32|  't/swamp/lua-shebang-test',                *
            # |    |       *  33|  't/swamp/not-an-#emacs-workfile#',         *
            # |    |       *  34|  't/swamp/notaMakefile',                    *
            # |    |       *  35|  't/swamp/notaRakefile',                    *
            # |    |       *  36|  't/swamp/notes.md',                        *
            # |    |       *  37|  't/swamp/options-crlf.pl',                 *
            # |    |       *  38|  't/swamp/options.pl',                      *
            # |    |       *  39|  't/swamp/parrot.pir',                      *
            # |    |       *  40|  't/swamp/perl-test.t',                     *
            # |    |       *  41|  't/swamp/perl-without-extension',          *
            # |    |       *  42|  't/swamp/perl.cgi',                        *
            # |    |       *  43|  't/swamp/perl.handler.pod',                *
            # |    |       *  44|  't/swamp/perl.pl',                         *
            # |    |       *  45|  't/swamp/perl.pm',                         *
            # |    |       *  46|  't/swamp/perl.pod',                        *
            # |    |       *  47|  't/swamp/pipe-stress-freaks.F',            *
            # |    |       *  48|  't/swamp/sample.asp',                      *
            # |    |       *  49|  't/swamp/sample.aspx',                     *
            # |    |       *  50|  't/swamp/sample.rake',                     *
            # |    |       *  51|  't/swamp/service.svc',                     *
            # |    |       *  52|  't/swamp/stuff.cmake',                     *
            # |    |       *  53|  't/swamp/swamp/ignoreme.txt'               *
            # |   2|]      |  54|]                                            |
            # +----+-------+----+---------------------------------------------+
            # actual[
            #   '-'
            # ]
            # expected[
            #   't/swamp/0',
            #   't/swamp/CMakeLists.txt',
            #   't/swamp/Makefile',
            #   't/swamp/Makefile.PL',
            #   't/swamp/MasterPage.master',
            #   't/swamp/Rakefile',
            #   't/swamp/Sample.ascx',
            #   't/swamp/Sample.asmx',
            #   't/swamp/c-header.h',
            #   't/swamp/c-source.c',
            #   't/swamp/crystallography-weenies.f',
            #   't/swamp/example.R',
            #   't/swamp/file.bar',
            #   't/swamp/file.foo',
            #   't/swamp/fresh.css',
            #   't/swamp/groceries/another_subdir/fruit',
            #   't/swamp/groceries/another_subdir/junk',
            #   't/swamp/groceries/another_subdir/meat',
            #   't/swamp/groceries/dir.d/fruit',
            #   't/swamp/groceries/dir.d/junk',
            #   't/swamp/groceries/dir.d/meat',
            #   't/swamp/groceries/fruit',
            #   't/swamp/groceries/junk',
            #   't/swamp/groceries/meat',
            #   't/swamp/groceries/subdir/fruit',
            #   't/swamp/groceries/subdir/junk',
            #   't/swamp/groceries/subdir/meat',
            #   't/swamp/html.htm',
            #   't/swamp/html.html',
            #   't/swamp/incomplete-last-line.txt',
            #   't/swamp/javascript.js',
            #   't/swamp/lua-shebang-test',
            #   't/swamp/not-an-#emacs-workfile#',
            #   't/swamp/notaMakefile',
            #   't/swamp/notaRakefile',
            #   't/swamp/notes.md',
            #   't/swamp/options-crlf.pl',
            #   't/swamp/options.pl',
            #   't/swamp/parrot.pir',
            #   't/swamp/perl-test.t',
            #   't/swamp/perl-without-extension',
            #   't/swamp/perl.cgi',
            #   't/swamp/perl.handler.pod',
            #   't/swamp/perl.pl',
            #   't/swamp/perl.pm',
            #   't/swamp/perl.pod',
            #   't/swamp/pipe-stress-freaks.F',
            #   't/swamp/sample.asp',
            #   't/swamp/sample.aspx',
            #   't/swamp/sample.rake',
            #   't/swamp/service.svc',
            #   't/swamp/stuff.cmake',
            #   't/swamp/swamp/ignoreme.txt'
            # ]
            # Looks like you failed 1 test of 1.
        not ok 1 - lists_match( DEFAULT_DIR_EXCLUSIONS )

        #   Failed test 'lists_match( DEFAULT_DIR_EXCLUSIONS )'
        #   at t/Util.pm line 456.
        # Looks like you failed 1 test of 1.
    not ok 2 - sets_match( DEFAULT_DIR_EXCLUSIONS )

    #   Failed test 'sets_match( DEFAULT_DIR_EXCLUSIONS )'
    #   at t/Util.pm line 472.
    # $ ack -f t/swamp
    # Looks like you failed 1 test of 2.
not ok 1 - ack_sets_match( DEFAULT_DIR_EXCLUSIONS )

#   Failed test 'ack_sets_match( DEFAULT_DIR_EXCLUSIONS )'
#   at t/ack-f.t line 73.

(There are many more of this type of test suite failures in Ubuntu's autopkgtests while Debian's autopkgtest passes. The infrastructure setups of their autopkgtest services seem to differ, but it's not clear to me where they differ and why that difference makes such a difference for ack.)

petdance commented 6 years ago

The test failure you're showing me is not one I've seen before. I don't know why that's failing, but I also don't know how you're running the tests.

xtaran commented 6 years ago

Unfortunately that's the kind of test failure which keeps ack out of Ubuntu at the moment.

The other one you fixed was just a pedantic nitpicking about potential typos (which even has been explicitly declared as non-error by Debian) and shouldn't cause any package to not be included anywhere.

But thanks for the feedback anyway. It means that Ubuntu needs to figure out on their own, what's the difference between their autopkgtest-setup and Debian's so that ack's test suite passes the as-installed testing on Debian, but not on Ubuntu. Because the Debian Perl Team (who wonders about this issue, too) has no more ideas either at the moment.

P.S. to @petdance: I've allowed myself to subscribe you to the according Ubuntu bug report, so you should get informed about any update there. I hope, that was ok. If not, feel free to either tell me or unsubscribe yourself.

petdance commented 6 years ago

Unfortunately that's the kind of test failure which keeps ack out of Ubuntu at the moment.

What can I do to help solve the problem? Is there some way I can run it to reproduce it so I can help figure it out? Right now, the situation is that everything is fine for me, but Ubuntu is running tests differently, and I can't do anything to fix it. Help me help you.

xtaran commented 6 years ago

Good question. autopkgtest (apt-get install autopkgtest pkg-perl-autopkgtest; the latter contains common autopkgtest infrastructure used by most perl-related packages) runs a package's test suite against the installed package (compared to build-time testing) and that way makes sure that test suite also passes with the paths and dependencies of the installed package (compared to build-dependencies and blib).

To be able to test ack installed at /usr/bin/ack, we (Debian Perl Team) "misuse" the nice ability of ack's test suite to run against either the normal ack or the standalone ack. For that we slightly patch the test suite to run either against the freshly built ack or /usr/bin/ack since binary-based distributions don't need the standalone version.

That works well in Debian (independent of the autopkgtest version) and worked in Ubuntu up to autopkgtest 4.x. But in Ubuntu (and only there) that no more works since approximately autopkgtest 5.0.

autopkgtest can run the tests either in a virtual machine or chroot (usually more accurate) or on the local system (then it might happen that more packages than minimally needed are installed and might cause unwanted side-effects.

On my laptop I run autopkgtest without virtual machine or chroot like this (null means no virtualization or chroot):

/tmp → apt-get source ack
Reading package lists... Done
NOTICE: 'ack' packaging is maintained in the 'Git' version control system at:
https://anonscm.debian.org/git/pkg-perl/packages/ack.git -b debian-stretch
Please use:
git clone https://anonscm.debian.org/git/pkg-perl/packages/ack.git -b debian-stretch
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 263 kB of source archives.
Get:1 https://debian.ethz.ch/debian sid/main ack 2.19.01-1 (dsc) [2124 B]
Get:2 https://debian.ethz.ch/debian sid/main ack 2.19.01-1 (tar) [250 kB]
Get:3 https://debian.ethz.ch/debian sid/main ack 2.19.01-1 (diff) [11.4 kB]
Fetched 263 kB in 0s (334 kB/s)
dpkg-source: info: extracting ack in ack-2.19.01
dpkg-source: info: unpacking ack_2.19.01.orig.tar.gz
dpkg-source: info: unpacking ack_2.19.01-1.debian.tar.xz
dpkg-source: info: applying no-standalone-test-but-autopkgtests.patch
/tmp → cd ack-*/
/tmp/ack-2.19.01 → autopkgtest -B ./ -- null

autopkgtest [19:48:46]: version 5.1
autopkgtest [19:48:46]: host c-cactus2; command line: /usr/bin/autopkgtest -B ./ -- null
autopkgtest [19:48:46]: testbed dpkg architecture: amd64
autopkgtest [19:48:47]: testbed running kernel: Linux 4.13.0-1-amd64 #1 SMP Debian 4.13.13-1 (2017-11-16)
autopkgtest [19:48:47]: @@@@@@@@@@@@@@@@@@@@ unbuilt-tree ./
autopkgtest [19:48:47]: testing package ack version 2.19.01-1
autopkgtest [19:48:47]: build not needed
autopkgtest [19:48:47]: test command1: preparing testbed
autopkgtest: WARNING: The following dependencies cannot be handled in reduced "unpack to temporary directory" mode: perl (>= 5.13.11) | libtest-simple-perl (>= 0.98)
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
autopkgtest [19:48:48]: test command1: /usr/share/pkg-perl-autopkgtest/runner build-deps
autopkgtest [19:48:48]: test command1: [-----------------------
t/00-load.t .................. 
1..1
ok 1 - All modules loaded
# Testing ack version 2.19_01 under Perl 5.026001, /usr/bin/perl
# Using File::Next 1.16
# Using File::Spec 3.67
# Using Getopt::Long 2.49
# Using Pod::Usage 1.69
# Using Test::Harness 3.38
# Using Test::More 1.302113
ok
t/ack-1.t .................... 
1..12
ok 1 - Should have no output to stderr: ack Montresor -1 -h --sort-files t/text
# Subtest: lists_match( Looking for first instance of Montresor! )
    1..1
    ok 1 - Looking for first instance of Montresor!
ok 2 - lists_match( Looking for first instance of Montresor! )
ok 3 - Should have no output to stderr: ack c -1 -h -v t/text/raven.txt
[…]
    ok 16 - /usr/bin/perl -wc /usr/share/perl5/App/Ack/Filter.pm exited successfully
    ok 17 - /usr/bin/perl -wc /usr/share/perl5/App/Ack/Resource.pm exited successfully
    ok 18 - /usr/bin/perl -wc /usr/share/perl5/App/Ack/Resources.pm exited successfully
    # Name "App::Ack::orig_program_name" used only once: possible typo at /usr/share/perl5/App/Ack.pm line 435.
    ok 19 - /usr/bin/perl -wc /usr/share/perl5/App/Ack.pm exited successfully
ok 4 - all modules in ack pass the syntax check
ok 5 - Package ack-grep is known to dpkg
ok 6 - Got status information for package ack-grep
ok 7 - Got file list for package ack-grep
ok 8 # skip no perl modules to test in ack-grep
ok
All tests successful.
Files=1, Tests=8,  1 wallclock secs ( 0.02 usr  0.00 sys +  0.87 cusr  0.10 csys =  0.99 CPU)
Result: PASS
autopkgtest [19:48:30]: test command3: -----------------------]
autopkgtest [19:48:30]: test command3:  - - - - - - - - - - results - - - - - - - - - -
command3             PASS
autopkgtest [19:48:30]: @@@@@@@@@@@@@@@@@@@@ summary
command1             PASS
command2             PASS
command3             PASS
autopkgtest -B ./ -- null  44.11s user 4.57s system 97% cpu 49.890 total

It is still unclear to, if the issue is a general autopkgtest issue on Ubuntu (i.e. happens independently of the way autopkgtest is called) or if it's specific to the way autopkgtest is called in Ubuntu's infrastructure for automatically testing all packages which declare that they're testable by autopkgtest.

So the next step is IMHO to figure out if the issue is already reproducible with the steps I did show above, just on Ubuntu and not Debian.

I intent to do this inside an Ubuntu chroot on one of my Debian boxes. Will keep you posted.

Oh, and JFTR: 2.19_01 still fails in Ubuntu's autopkgtest, at least on the amd64 architecture (didn't check all architectures). But there seems more than just "it started to fails since Ubuntu 17.10":

This doesn't really look deterministic to me. :-(

xtaran commented 6 years ago

Using pbuilder I've set up a minimal chroot for Ubuntu 18.04 Bionic as of today (i.e. a development snapshot). Inside that chroot, I installed autopkgtest, pkg-perl-autopkgtest and ack (version 2.18+dfsg-1, 2.19.01-1 doesn't seem to be on my mirror yet) from bionic-proposed and ran autopkgtest ./ -- null inside the unpacked source package.

"Unfortunately" all tests passed. So I can't reproduce the issue Ubuntu finds on their autopkgtest infrastructure by simply running autopkgtest inside a minimal chroot. So I still have no idea what goes wrong. (Will now add this information to the Ubuntu bug report, too.)

petdance commented 6 years ago

Is there anyone in Ubuntu-land we can nudge? Is there someone in charge of the package? Or who created the package?

petdance commented 6 years ago

I've allowed myself to subscribe you to the according Ubuntu bug report

Yes, that's fine, @xtaran. Thanks for the heads-up.

xtaran commented 6 years ago

Is there anyone in Ubuntu-land we can nudge?

I hope that declaring the Ubuntu bug as also being a bug in autopkgtest will make the autopkgtest maintainers of Ubuntu look at it. (autopkgtest AFAIK has its roots in Ubuntu and hence the package flow outlined for ack below does not apply.)

Is there someone in charge of the package?

Ubuntu automatically imports the ack package (as most other packages, too) from Debian Unstable into the "universe" archive of their development branch on a daily base. But the automatism stops e.g. if autopkgtests fail (obviously) or e.g. if there are ubuntu-specific patches in place which might need manual updating.

AFAIK there is no specific person per package (at least not in Ubuntu's "universe" archive) as it's the case in Debian. The packages in the "universe" archive are supervised by the "Masters of the Universe" (pun probably intended). But I don't know anything about their non-automatic workflow.

Or who created the package?

The package has been initially created in 2007 for Debian and since then is maintained and updated there for Debian. It's currently maintained by the Debian Perl Team with me taking care of it most of the time for a few years now.

We (Debian Perl Team) do also care about bug reports against our packages in Ubuntu because the bugs found in Ubuntu often also exist in Debian. But we usually can't do much about ubuntu-specific issues like this one.

petdance commented 6 years ago

Here's something interesting: Many of the tests I see at https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-artful/artful/amd64/a/ack/20171024_160017_5333d@/log.gz are showing - as filenames found in tests where actual filenames are expected, like so:

                #   Failed test 'File on command line is always searched'
                #   at t/Util.pm line 412.
                # +----+-------+---------------------------------+
                # | Elt|Got    |Expected                         |
                # +----+-------+---------------------------------+
                # |   0|[      |[                                |
                # *   1|  '-'  |  't/swamp/#emacs-workfile.pl#'  *
                # |   2|]      |]                                |
                # +----+-------+---------------------------------+
                # actual[
                #   '-'
                # ]
                # expected[
                #   't/swamp/#emacs-workfile.pl#'
                # ]
                # Looks like you failed 1 test of 1.

It's a pretty consistent pattern happening throughout. Maybe there's something in how the tests get run that confuse STDIN with the filesystem?

xtaran commented 6 years ago

@petdance: Thanks! I've forwarded it to the Ubuntu bug report.

petdance commented 6 years ago

ack 2.20 has now been released, and is on master here.