search

freebsd / atf

Libraries to write tests in C, C++ and shell
Other
127 stars 44 forks source link

Why are tests being installed? #33

Closed Earnestly closed 7 years ago

Earnestly commented 7 years ago

Currently when using make install I'm finding a lot of tests are being installed to the DESTDIR/PREFIX/tests. Is this intended?

When using --prefix=/usr it results in files installed to /usr/tests which seems wrong.

If this is intentional where does upstream consider a good place to install these tests?

jmmv commented 7 years ago

Yes, this is intentional.

I think the original documentation page that explained this has been lost... but for some context: https://wiki.netbsd.org/tutorials/atf/#index4h1

Earnestly commented 7 years ago

That didn't seem to answer why /usr/tests was chosen over something like /usr/share/atf/tests which seems more consistent with other software projects.

Although the context is helpful in understanding both why someone would do this in such an environment and why it's not necessary to include in the final install directory.

However am I still able to run the self tests prior to make install?

Edit: It appears I cannot run tests without installing this software...

jmmv commented 7 years ago

/usr/share/atf/tests makes no sense. From hier(7):

       /usr/share
              This directory contains subdirectories with specific application data, that can be shared among different architectures of the same OS.  Often one finds stuff here that used to live in /usr/doc or /usr/lib or /usr/man.

Tests cannot be shared among different architectures. You could abuse Linux's approach of /usr/lib/atf/tests/ which doesn't make much sense either, but at least it's not broken from the point of view of sharing — which is what I did for the Fedora RPM packages. Note that, in this latter case, the tests are still installed, but they are made available via a separate -tests subpackage.

And no, you cannot run the tests prior installation at this point. It could be done (the Kyua package allows this), but it's not yet.

Earnestly commented 7 years ago

Yes, fair point about /usr/share. I think none of the locations make sense because the premise of installing the tests doesn't. For now I'll just forgo testing entirely as it cannot be done nicely in an automated manner prior to installation.

jmmv commented 7 years ago

May I ask what you are trying to do?

Earnestly commented 7 years ago

Simply packaging this software because https://github.com/pkgconf/pkgconf is intending on using your kyua system for their tests which showed up in their recent effort to use meson.

The packaging process lets me run any test suites or the like during the construction of the package, with a specific phase dedicated to doing so. The way atf has been uniquely architected is proving difficult to integrate as the assumption is software is tested before it is installed. Because I'm doing this only to see if I can get kyua working with pkgconf I'm not too concerned.

jmmv commented 6 years ago

And may I ask what you are packaging this software for, assuming it's a public Linux distribution or other system?

Yes, pre-installation testing is pretty common. The reason ATF doesn't have that is historical... It really should have this feature; it's probably not too hard to do at this point, so maybe you'd file a separate bug to track this request?

But regarding pkgconf: I'm assuming they'll make the dependency on ATF optional? If they do, you'd just disable the build of their tests...