PhilterPaper / Perl-PDF-Builder

Extended version of the popular PDF::API2 Perl-based PDF library for creating, reading, and modifying PDF documents
https://www.catskilltech.com/FreeSW/product/PDF%2DBuilder/title/PDF%3A%3ABuilder/freeSW_full
Other
6 stars 7 forks source link

CPAN Build issues #92

Open PhilterPaper opened 6 years ago

PhilterPaper commented 6 years ago

I always try to get a nice clean CPAN upload, so that PDF::Builder looks good to someone browsing it in CPAN. Sometimes a minor POD error will slip through, for which I will not necessarily submit a new upload (version). However, I will fix the problem in GitHub so that it should be clean next time.

There are a few persistent things in CPAN, particularly the "Kwalitee" report, that are causing concerns:

    Kwalitee Indicator: buildtool_not_executable <core>

    The build tool (Build.PL/Makefile.PL) is executable. This is bad because you should specify which perl you want to use while installing.

    How to fix

    Change the permissions of Build.PL/Makefile.PL to not-executable. 

Um, OK, what exactly are they asking for here? Are they saying that the Makefile.PL I supply is bad because it's executable on a Linux system? It's created on a Windows system, so I don't have any control (that I know of) of permissions when it gets on to the CPAN system. If you build on a Linux box, do you make sure the file is 644 and magically this error goes away? On my system, I have the .pl file extension associated with "perl", so I don't have to type in "perl" on the command line. Is that "executable" being passed to the CPAN build? I'll have to remember to check at an intermediate stage, whether Makefile.PL has its permissions explicitly stated somewhere, and if that's something I can change (e.g., "1755" -> "1644").

By the way, Kwalitee's error labels are stupid. They should state what the error condition is, not what the fix is! So this one should be _buildtool_isexecutable.

    1 Extra Issue

    use_warnings

Add 'use warnings' (or its equivalents) to all modules (this will require perl > 5.6), or convince us that your favorite module is well-known enough and people can easily see the modules warn when something bad happens.

Error: PDF::Builder, PDF::Builder::Annotation, PDF::Builder::Basic::PDF::Dict, PDF::Builder::Basic::PDF::File, PDF::Builder::Basic::PDF::Filter::FlateDecode, PDF::Builder::Basic::PDF::Filter::LZWDecode, PDF::Builder::Basic::PDF::Literal, PDF::Builder::Basic::PDF::Page, PDF::Builder::Basic::PDF::Pages, PDF::Builder::Content, PDF::Builder::Lite, PDF::Builder::NamedDestination, PDF::Builder::Outline, PDF::Builder::Page, PDF::Builder::Resource::BaseFont, PDF::Builder::Resource::CIDFont, PDF::Builder::Resource::CIDFont::CJKFont, PDF::Builder::Resource::CIDFont::TrueType, PDF::Builder::Resource::CIDFont::TrueType::FontFile, PDF::Builder::Resource::ColorSpace, PDF::Builder::Resource::ColorSpace::DeviceN, PDF::Builder::Resource::ColorSpace::Indexed, PDF::Builder::Resource::ColorSpace::Indexed::ACTFile, PDF::Builder::Resource::ColorSpace::Indexed::Hue, PDF::Builder::Resource::ColorSpace::Indexed::WebColor, PDF::Builder::Resource::ColorSpace::Separation, PDF::Builder::Resource::ExtGState, PDF::Builder::Resource::Font, PDF::Builder::Resource::Font::BdFont, PDF::Builder::Resource::Font::CoreFont, PDF::Builder::Resource::Font::Postscript, PDF::Builder::Resource::Font::SynFont, PDF::Builder::Resource::UniFont, PDF::Builder::Resource::XObject::Image::GD, PDF::Builder::Resource::XObject::Image::GIF, PDF::Builder::Resource::XObject::Image::PNM, PDF::Builder::UniWrap, PDF::Builder::Util

Many modules still use "no warnings" with a list of what I presume are exception conditions. For instance, Builder.pm has

        use strict;
        no warnings qw[ deprecated recursion uninitialized ];

Is it asking me to stick use warnings; before no warnings? Would "warnings" then be the default condition, with a few specific conditions to be ignored? Or do I need to do some major fixes to the code so that "no warnings" can be replaced by "use warnings"? I'm reluctant to change any code until I understand how these things are supposed to interact. All code now is "use warnings".

    Kwalitee Indicator: meta_yml_has_provides <experimental>

    This distribution does not have a list of provided modules defined in META.yml.

    How to fix

    Add all modules contained in this distribution to the META.yml field 'provides'. Module::Build or 

Dist::Zilla::Plugin::MetaProvides do this automatically for you.

Since META.yml seems to be built on the fly, I don't think it will be easy (though not impossible) to insert some sort of additional content into the file. I use ExtUtils::MakeMaker to do the building, so is there a way to get the desired content into META.yml that way? If not, what exactly are they looking for — a list of every .pm file (and in what format), or something more manageable?

    Kwalitee Indicator: meta_yml_has_repository_resource <experimental>

    This distribution does not have a link to a repository in META.yml.

    How to fix

    Add a 'repository' resource to the META.yml via 'meta_add' accessor (for Module::Build) or 

META_ADD parameter (for ExtUtils::MakeMaker).

This one I think I figured out. The next build, I'm trying the following in Makefile.PL:

          META_MERGE        => {

            "meta-spec" => { version => 2 },

            resources => {

              homepage => "https://www.catskilltech.com",

              repository => {
                  type => 'git',
                  url => 'git://github.com/PhilterPaper/Perl-PDF-Builder.git',
                  web => 'https://github.com/PhilterPaper/Perl-PDF-Builder',
              },

            },

          },

and hopefully this will add the repository information it wants to META.yml. I'll have to make a note to change my website when it goes https sometime soon (I hope).

Website is now https://www.catskilltech.com.

PhilterPaper commented 6 years ago

In each of my releases, I've managed to forget to properly handle a new prerequisite. It would be really nice to be able to do a private "test" upload to CPAN, to see if everything checks out clean, before doing the public upload under a new version number. In RT 123123,

  > If anyone knows a way to make a "test" or "trial" CPAN upload, where
  > some testing is done but the distribution is not made public (so I
  > don't waste a version number if something goes wrong), I'd appreciate
  > a pointer to the instructions to do this.

<from Andreas Koenig>
Frequently used for this kind of question: Travis. 

Does anyone have some clear instructions for using Travis that they can point me to? I think the original PDF::API2 had some sort of "Travis" file, but I don't know how to use it (I recall that I was getting some errors before disabled .travis.yml by changing the name). Is Try Travis CI with your CPAN distributions a good start? Will it give me what I am looking for?

Of course, while the preliminary checkout is done pretty quickly, it takes a day or two for the testing to get going, and this is where missing prerequisites seem to bite me. Any suggestions for building on Windows (Win7 currently) and testing the resulting upload (especially for missing prerequisites and POD errors) would be appreciated.

Now on Windows 10. I have disabled Travis for now because the last time I left the file name "correct", it apparently tried running Travis and I got ugly error messages.

Note that I really need an upload to CPAN to do a test install, without burning another release number. I know how to use cpan and cpanm to install a candidate package, but as the new prereq will have already been installed, I don't want to have to go and delete a bunch of prereq packages to make sure they install anew! Finally, I seem to recall hearing that Travis has been discontinued anyway.

PhilterPaper commented 6 years ago

Regarding the Kwalitee issues, there seems to be a lot of overlap with what perlcritic is reporting. Hopefully fixing a lot reported by perlcritic (see pc.bat) will make Kwalitee happier. The "no warnings" issue alone will clean up a lot of both systems' complaints.

PhilterPaper commented 5 years ago

Starting in release 3.011, I added a line to Makefile.PL to direct bug reports from the CPAN RT system to GitHub. Unfortunately, it appears to have not worked. Some day I'll do some more research to see if I was given incorrect information.

PhilterPaper commented 5 years ago

I have moved the README file back to the root. This is to shut up Kwalitee's complaint about "no README file". I also changed the "bugtracker" entry format in Makefile.PL to see if this will now point CPAN Issues to GitHub ticket system instead of the RT ticket system.

PhilterPaper commented 5 years ago

CPAN seems to be happy now with using GitHub for Issues. I'm still getting the "no warnings" scolding, and the one about META.yml does not have "provide". After I moved LICENSE to INFO/, it complains that there is a "no separate LICENSE file". I wish there was a way to tell CPAN/Kwalitee to look in INFO/ for stuff, before complaining.

One thing that's missing is a way to tell the system about "optional" libraries. I don't want to require installation of something like Graphics::TIFF, as very few people use TIFF images, but I do want to show up in their "reverse dependencies" list as 1) reassurance to the author that someone is using their package, and 2) advertising for this product! There is supposedly a "recommends" entry for the Makefile.PL, but I have not found any information on what it does or the best way to use it. If you have good information on this, please feel free to comment!

PhilterPaper commented 5 years ago

The latest build (3.013) seems to be doing well, except for one thing. It appears that the four new .pl test scripts (1_pc.pl, etc.) are being installed in the root, alongside Builder.pm. When they were .bat files, they did not get installed into the Perl system (only lib/PDF/ stuff, and temporarily, t/). The MANIFEST did not change their locations, only their names.

This does not appear to do any harm, but is a minor annoyance. It is OK to erase the four .pl files, as they're not going to do any good there. If anyone has any idea why this happened, please chime in and I will fix it in the next build.

PhilterPaper commented 5 years ago

I have updated Makefile.PL (and README and PDF::Builder::Docs) to use the "recommends" section for the optional libraries. In many cases, the installer will attempt to install these optional libraries. If one fails to install, no need to panic — PDF::Builder should still install OK, and will still run. If you want to make use of the new features associated with that library, you'll need to manually initiate an installation (which could fail if the automatic installation failed). If you don't want an optional library, feel free to delete it from your Perl installation in order to free up space.

  1. Users don't have to manually install an optional library if they want to use advanced features.
  2. PDF::Builder should show up in the optional libraries' "Reverse dependencies" list (let their authors know their work is appreciated and being used, and advertise PDF::Builder).

This should appear in CPAN with release 3.014, later this Spring.

PhilterPaper commented 5 years ago

In release 3.014, the four offending Perl test routines (pc, t-tests, examples, contrib) were moved to a new tools/ directory. They no longer get installed into lib/PDF/.

Unfortunately, the "optional" prereqs are still not showing up in the Reverse Dependencies lists of those packages, as hoped for. I'll have to look around some more for how to do that. No idea yet if these "optionals" are installed on a fresh install.

PhilterPaper commented 4 years ago

In 3.017, I removed the optional "recommends" library installations. While I think they were installing, the potential concern to users if such an optional installation failed outweighed the convenience. In addition, I saw no evidence that these optional libraries notified their owners that someone was using them (via "Reverse Dependencies"), which was my prime objective.

So, we're back to where you have to manually install optional libraries (currently two) if you want to use them.

PhilterPaper commented 3 years ago

As of yesterday's 3.020 release, Makefile.PL asks you what optional libraries you want to try to install (default = all). There are three optional libraries at this point: Graphics::TIFF (libtiff.a support), Image::PNG::Libpng (libpng.a support), and HarfBuzz::Shaper (HarfBuzz glyph shaping). In the "near" future I may add Text::Hyphen for improved hyphenation, and Text::KnuthPlass for paragraph shaping.

PhilterPaper commented 3 years ago
  1. CPAN Testers matrix display has been having troubles, but I assume they'll get that fixed. It's hard to fix stuff when you can't see which tests broke.
  2. Kwalitee seems to have gone away, so any issues related to that are moot. It seems to have a lot of overlap with PerlCritic, anyway (see tools/1_pc.pl). Not to say that I don't find some of PerlCritic's complaints to be bizarre.
  3. I have enabled warnings (remove "no warnings", add "use warnings") everywhere, and cleaned up some reported problems. Please report anything that you think may be a consequence of fully enabling warnings.
  4. I'm switching over to README.md instead of README, as it permits formatting (MarkDown). GitHub code pages show README.md, while CPAN shows the formatted POD of the main routine.
  5. A chronic problem (of late) has been failure to match unformatted floating point results, when someone tests with a Perl built with long doubles or quadmath libraries. They give more decimal digits than the standard doubles, and there is no consistency among them (no standards). As I encounter these, I am rounding floating point results to 6 or so digits of precision (in t-tests only), and comparing against that. That should work even if someone tests on an IBM mainframe (360/370/390/z-Series), as it has its own floating point format (not the same as IEEE-754).
  6. Since CPAN seems to have been a bit wobbly of late, I am including the .tar.gz installation images (as sent to CPAN) on the GitHub repositories, under releases/, just as a backup. The last three releases will be kept there.
PhilterPaper commented 3 years ago

Just an aside on something I came across: according to https://neilb.org/2015/10/18/spotters-guide.html you can have an xt/ directory for extended t-tests, that are not normally run during install/upgrade. This would enable full(er) test coverage, without burdening everyone with an enormous test suite for every minute edge case and long-ago bug regression test. Worth thinking about.

I have not tested an xt/ directory.

Update: Author-only tests have been moved to xt/

PhilterPaper commented 1 year ago

Plans:

  1. Remove "optional" library installs from the install process. Suggest when they should be used, in the documentation (especially README) and check when used if they were actually installed (including a suggestion to install them). This will be for Graphics::TIFF, Image::PNG::Libpng, HarfBuzz::Shaper, and upcoming library usage for markup, SVG support, hyphenation, paragraph shaping, etc. Should it be a fatal error to use uninstalled modules? t-tests and examples should simply skip over anything requiring an uninstalled library.
  2. The current build tools seem to mess up the META.json and META.yml, removing the "provides" statements that I so laboriously included. If I can't fix the tools to automatically provide "provides", I will have to add some tool code to semi-manually add provided names to the MYMETA files (once generated). "provides" seems to be only of interest to CPAN "kwalitee", AFAICT.

I went ahead and removed the optional library installs, and from now on will merely list the suggested libraries. I went ahead and removed the "provides" stuff, as there's no documentation on how to do it better than what CPAN does by itself.