Questions about general capabilities

I maintain the PDF::Builder package, for creating (and editing) PDF documents. One of the things on my wishlist is a comprehensive data plotting package for PDF::Builder. I just discovered PDL::Graphics::Gnuplot, and it looks promising.

Apparently it can produce PNG image files as output. Is the size scalable, so that an image can be produced at the desired final dimensions and incorporated into a document without scaling it (thus degrading quality)?
For a PNG image (are there other formats, such as GD, TIFF, GIF, SVG?), is there any difference between a standalone Gnuplot installation's output, and PDF::Graphics::Gnuplot? I don't want to get all excited by writing code to pipe in/out of PDL::Graphics::Gnuplot and then discover that it can't do three-quarters of the amazing examples online.
Is there any scalable vector (e.g., SVG) output available? This would be desirable if a user wants to be able to zoom into a plot to look at it in detail, using a PDF Reader's zoom/scale function. A raster image-only output is acceptable, however.
It's no skin off my nose if a user has to produce a plot outside of the PDF::Reader production environment (running standalone Gnuplot, or a Perl program using PDF::Graphics::Gnuplot), and explicitly import it as an image (e.g., PNG file), although it would be really nice to be able to produce plots on-the-fly from data within the Perl program creating a PDF document. Does PDL::Graphics::Plot have this capability (two-way pipes, I guess)? I'd probably have to read back any piped-in image file into a file, but I might be able to directly read some formats into PDF::Builder.

I've just started poking around this package, and looking at online Gnuplot demos, so please accept my apologies if the answers are all on page one of the documentation! BTW, PDF::Reader does not yet have SVG image capability, but I'm going to implement it this year.

Thanks, and I might have more questions once I see your reply!

Yes, the output files will be of different file sizes (here I make 400x400 and 800x800 pixel images):

pdl> $m51 = rfits('~/Build/PDL/m51.fits') #your file location will vary
Reading IMAGE data...
BITPIX =  32  size = 147456 pixels 
Reading  589824  bytes
BSCALE = 1 &&  BZERO = 0

pdl> use PDL::Graphics::Gnuplot

pdl> $w=gpwin("pngcairo",size=>[4,4],output=>"img44.png")
pdl> $w->imag($m51);
pdl> $w->close

pdl> $w=gpwin("pngcairo",size=>[8,8],output=>"img88.png")
pdl> $w->imag($m51);
pdl> $w->close

pdl> exit

$ ls -sk1 img??.png
212 img44.png
372 img88.png

$ file img??.png
img44.png: PNG image data, 400 x 400, 8-bit/color RGB, non-interlaced
img88.png: PNG image data, 800 x 800, 8-bit/color RGB, non-interlaced

PDL::Graphics::Gnuplot uses the system gnuplot installation. So whatever gnuplot commands P::G:::G outputs should result in the same output whether they were generated with PDL first, or if your script just output the same Gnuplot commands. P::G::G simplifies some things (as the example above shows), but it's possible that not every Gnuplot feature has an analogue in P::G::G. And Yes, there are other formats. They are called "terminals" in gnuplot parlance. You typically select what terminals will be available when gnuplot is installed. Then PDL::Graphics::Gnuplot can use those. See "Gnuplot supports many different types of output" on the Gnuplot homepage and Part IV of the Gnuplot Manual.
On the Gnuplot homepage it says that svg is an option, and I see that is available for me locally and my PDL::Graphics::Gnuplot installation, the Perl command PDL::Graphics::Gnuplot::terminfo(); output tells me that svg is an option. (so does the gnuplot command 'set term'). Whether that matches the svg standard that PDF::Builder needs is another thing--I've had mixed luck with svg in the past. PDF file output might also work for you as a vector format--that's straightforward to use in another PDF document.
I'm not sure exactly what you mean here and I have limited experience with piping data back and forth like this so I will leave that answer to others.

Thanks for the speedy response!

Apparently it can produce PNG image files as output. Is the size scalable, so that an image can be produced at the desired final dimensions and incorporated into a document without scaling it (thus degrading quality)?

Yes, the output files will be of different file sizes (here I make 400x400 and 800x800 pixel images):

OK, it's good that the image size can be selected. I'll have to do some playing around and come up with some suggested guidelines for picking the proper size in pixels to fit a desired size in points in the document. I want to avoid possible image degradation due to rescaling.

For a PNG image (are there other formats, such as GD, TIFF, GIF, SVG?), is there any difference between a standalone Gnuplot installation's output, and PDF::Graphics::Gnuplot? I don't want to get all excited by writing code to pipe in/out of PDL::Graphics::Gnuplot and then discover that it can't do three-quarters of the amazing examples online.

PDL::Graphics::Gnuplot uses the system gnuplot installation. So whatever gnuplot commands P::G:::G outputs should result in the same output whether they were generated with PDL first, or if your script just output the same Gnuplot commands. P::G::G simplifies some things (as the example above shows), but it's possible that not every Gnuplot feature has an analogue in P::G::G. And Yes, there are other formats. They are called "terminals" in gnuplot parlance. You typically select what terminals will be available when gnuplot is installed. Then PDL::Graphics::Gnuplot can use those. See "Gnuplot supports many different types of output" on the Gnuplot homepage and Part IV of the Gnuplot Manual.

Looking through Part IV, I see support for PNG and SVG, which should cover my needs. I think most users will want to produce PNG fixed scale images, but some may want to create scalable vector plots so that the viewer can zoom in on them (to a degree, anyway -- I'll have to try it out and see how detailed a plot is). It sounds like PGG writes to a "device", which would typically be a file. I don't see any problem with having a PNG or SVG file -- that's normally how images are brought into PDF::Builder. I'll just have to remember to erase these files afterwards, unless the user has signaled that they want to keep them for other purposes (including reuse later in the document).

Is there any scalable vector (e.g., SVG) output available? This would be desirable if a user wants to be able to zoom into a plot to look at it in detail, using a PDF Reader's zoom/scale function. A raster image-only output is acceptable, however.

On the Gnuplot homepage it says that svg is an option, and I see that is available for me locally and my PDL::Graphics::Gnuplot installation, the Perl command PDL::Graphics::Gnuplot::terminfo(); output tells me that svg is an option. (so does the gnuplot command 'set term'). Whether that matches the svg standard that PDF::Builder needs is another thing--I've had mixed luck with svg in the past. PDF file output might also work for you as a vector format--that's straightforward to use in another PDF document.

It's not a great loss if the SVG capabilities are too limited; I'll just have to try them out and see. About the only advantage over a raster format like PNG is that there is the potential to cleanly zoom in to examine details (provided they're there in the first place!). As for what PDF::Builder "needs" -- that's an open question, as I'm just now playing around with SVG for some other purposes (including MathJax equation formatting).

It's no skin off my nose if a user has to produce a plot outside of the PDF::Reader production environment (running standalone Gnuplot, or a Perl program using PDF::Graphics::Gnuplot), and explicitly import it as an image (e.g., PNG file), although it would be really nice to be able to produce plots on-the-fly from data within the Perl program creating a PDF document. Does PDL::Graphics::Plot have this capability (two-way pipes, I guess)? I'd probably have to read back any piped-in image file into a file, but I might be able to directly read some formats into PDF::Builder.

I'm not sure exactly what you mean here and I have limited experience with piping data back and forth like this so I will leave that answer to others.

What I was asking about was whether it's a one-way street from the user's program + PDF::Builder to Gnuplot, and then retrieve the produced image file discretely and treat it as a normal image file, or if it's a potential two-way street, where data and commands go out to Gnuplot, and the produced image file is piped back in, eliminating the use of a separate image file. As I said earlier, it's no big deal to read in an image file and (usually) erase it. BTW, MathJax is a separately running NodeJS program that can pipe equation markup from a Perl program and pipe the resulting SVG back to the program for rendering. Capability like that would be nice, but is not vital.

Overall, PDF::Builder is a library of routines called from a Perl program to create a PDF document. It sounds like Gnuplot is a separately running program that you provide glue routines to, for feeding data and commands to it, and it creates an image file in the desired format (which PDF::Builder can be told about and suck it in for rendering).. Is my understanding more or less correct? PGG just makes it easier to feed data and commands to Gnuplot from a running Perl program, as opposed to running Gnuplot discretely and then telling the user program what the image file is? I may even go so far as to permit the user Perl program to hand the data and commands to PDF::Builder, to then be sent via PGG to Gnuplot, etc., but this isn't a deal-breaker. It would be nice to even have a user-supplied Perl program produce the data and generate the plot via PGG (either directly or via PDF::Builder), and then use PDF::Builder to package everything into a nicely formatted PDF document.

You don't have to experiment to get things a certain number of points in size. You can specify sizes in pixels (assumed 100dpi), points, inches, or cm. From the manual, on terminal options:

size Most drivers support a "size" option to specify the size of the output plotting surface. The format is [$width, $height, $unit]; the trailing unit string is optional but recommended, since the default unit of length changes from device to device. The unit string can be in, cm, mm, px, char, or pt. Pixels are taken to be 1 point in size (72 pixels per inch) and dimensions are computed accordingly. Characters are taken to be 12 point in size (6 per inch).

Well, that will be nice if the dimensions can be directly specified in points, bypassing all the busy-work of figuring how many pixels that corresponds to. I'll have to play with it soon.

If I have any further specific questions, I'll get back to you with a new ticket. Otherwise, I'll post here with the final results, so you can see how I used your code, some time later this year (I hope). Thanks again for the help!

As I am able to run the demos in batch mode (see #96), I am left wondering whether the full PDL::Graphics::Gnuplot would be massive overkill for my purposes, and would I be better off just using the system() call to call Gnuplot directly (command line interface)? I think the usual methodology for a user of PDF::Builder would be to prepare a "template" file giving the layout and fixed data/labels (similar to a demo/*.dem file), and reading in any "live" data from a just-prepared file they've made. Then PDF::Builder can read in the template and chosen format (terminal) and size (in points), and a batch job fired off to Gnuplot. The resulting PNG (or SVG, GIF, JPEG, or (?) TIFF) file would then be output to the PDF at the chosen position and size, like any other image.

Does PGG give me anything beyond this capability that I (or document creators) would find useful and exciting? Only a single static plot at a time is needed (no animation, no interactivity). The installation process seems to have some problems (described in the other ticket), so you may want to address those anyway, but if I don't really need to use PGG, problems with it won't hold me up. (Sorry!)

For running the "demo/" examples, I would have to caution users to examine a .dem file first and make sure it only produces one output plot, and any "pause" command is removed. I'm guessing that the demos look for any data files in the demo/ directory, so I'll have to test where and when paths are needed. That reminds me that I need to do some investigating about just what "default config file" it's sometimes looking for, and whether I need to have a Gnuplot user set one or more search paths.

I think just having an API to pass in the Gnuplot script (as a file/string) would be the simplest and allow using multiple ways of working with Gnuplot such as https://metacpan.org/pod/Gnuplot::Builder and PGG. You would need a way of also specifying the output file and terminal or just query Gnuplot (show term; show output) after the plot. By the way, though there is no official extension for Gnuplot scripts, there are a couple popular ones https://stackoverflow.com/questions/5497889/is-there-a-standard-file-extension-for-gnuplot-files.

What PGG does is build the script in the background and pass the PDL ndarray data. It currently does not support writing the script itself out to a file/string (as far as I know). You can get the script by passing in a couple options and capturing STDOUT which isn't ideal, but this is because the API is currently more centered on driving Gnuplot immediately after the plot commands rather than saving the script for later use. If that changes, I'll let you know --- it may be part of #95.

use PDL;
use PDL::Graphics::Gnuplot;
use Capture::Tiny qw(capture_stdout);
use feature qw(say);

my $x = zeroes(1e2)->xlinvals(0,6.28);
my $w = gpwin( 'dumb', { dump => 1, binary => 0 } );
my ($script, undef) = capture_stdout{ $w->plot( 'lines', $x, $x->sin ) };

say $script;

The next release (2.025, coming soon) will have a plot_generate method:

use PDL;
use PDL::Graphics::Gnuplot;
use feature qw(say);

my $x = zeroes(1e2)->xlinvals(0,6.28);
my $w = gpwin( 'dumb' );
my $script = $w->plot_generate( 'lines', $x, $x->sin );

say $script;

There are equivalents for multiplot, multiplot_next and end_multi. @PhilterPaper does this meet your needs, and if so can this issue be closed?

A lot of water has flowed under the bridge over the past year, and I haven't had much of a chance to explore PGG further. I will very soon have SVG vector image capability, as well as PNG images, so I expect that one way or the other I should be able to use PGG to do plotting in PDF::Builder. I don't know yet if plot_generate() will do the job, or I should just build a batch job and invoke GnuPlot at the system level to create an image file. When I get time to get back to this, I'll update this ticket to report what I ended up doing.

PDLPorters / PDL-Graphics-Gnuplot

Questions about general capabilities #94