PyFerret nodisplay (Cairo) graphics issues

[karlmsmith]

Reported by @AndrewWittenberg on 1 Oct 2014 23:09 UTC This ticket extracts the two remaining items from ticket #2167.

To run the example, untar cairo.tgz (which I'll attach to this ticket). Make any necessary environment changes to "mytest.csh" and then execute it. I ran this with the PyFerret v1.0.3 based on the 9/29/14 version of Ferret v6.925.

Things to notice:

1) Most important, the Cairo (nodisplay) graphics produce much bigger files than Qt (display). The inflation factor is 2x for PNG; 3-4x for PDF; and 20x (!) for PS. For PS, zooming in on a sloping line (e.g. the slant in the number "2") by a factor of 100-1000 shows that the output has been rasterized at high resolution.

2) For PNG output, the Cairo output is blurred compared to Qt.

3) FRAME/XPIXEL=1000 gives the same PNG image sizes in both Cairo and Qt. But Cairo and Qt appear to use different default sizes when FRAME is issued with no qualifiers:

% file {qt,cairo}/asp_0.77/plot.png
qt/asp_0.77/plot.png:    PNG image data, 997 x 768, 8-bit/color RGBA, non-interlaced
cairo/asp_0.77/plot.png: PNG image data, 948 x 730, 8-bit/color RGBA, non-interlaced

Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/2202

[karlmsmith]

Comment by @AndrewWittenberg on 2 Oct 2014 17:49 UTC After more testing, it seems that something in my frame_to_page.jnl is inflating Qt PS files by a factor of 2, but Cairo PS files by a factor of almost 200! Looking into this.

The PNG results are independent of my script though.

[karlmsmith]

Comment by @AndrewWittenberg on 2 Oct 2014 18:27 UTC The PS rasterization problem seems to be with the PDF->PS conversion. Possibly having to do with transparency.

[karlmsmith]

Comment by @AndrewWittenberg on 2 Oct 2014 21:27 UTC Even simpler -- save the following as mytest.jnl:

\can mode verify
! Usage: go mytest [display]
!                  [  $1   ]

def sym display $1"qt"

sp rm -rf ($display)
sp mkdir -p ($display)

set win/asp=.7 1

plot/nolab/noax/vs/line=1 {0,1},{0,1}
frame/file="($display)/line.ps"
frame/file="($display)/line.pdf"

sp sleep 1
sp pdftops ($display)/line{.pdf,_convert.ps}
sp ps2pdf ($display)/line_convert.{ps,pdf}

set mode/last verify

Then run from the command prompt:

% echo "go mytest qt; quit" | $FER_DIR/bin/pyferret
% echo "go mytest cairo; quit" | $FER_DIR/bin/pyferret -nodisplay

Here are the results:

% tree -s
.
|-- [       2048]  cairo
|   |-- [      36728]  line_convert.pdf
|   |-- [     496474]  line_convert.ps
|   |-- [       1797]  line.pdf
|   `-- [       2643]  line.ps
|-- [        389]  mytest.jnl
`-- [       2048]  qt
    |-- [       2427]  line_convert.pdf
    |-- [      11067]  line_convert.ps
    |-- [       1353]  line.pdf
    `-- [       6058]  line.ps

This just draws a single line, with no axes. But I can't figure out how to get pdftops to convert Cairo PDF graphics into a PS file without rasterizing. I'm guessing that it has something to do with lines like the following in the Cairo PDF, which are not in the Qt PDF:

      /S /Transparency

Why this difference between Cairo and Qt?

This is a headache -- since unless we can convert to PS, how are we to frame/rotate the graphics produced by Cairo?

[karlmsmith]

Comment by kevin-obrien on 23 Dec 2014 22:41 UTC

It is possible that removing the alpha channel from the output ps might prevent the rasterization of the converted ps file. Karl will experiment with this and report back soon.....

[karlmsmith]

Comment by @karlmsmith on 24 Dec 2014 20:24 UTC So the alpha channel (transparency) is fully supported for PDF v1.4 and later, but not for PostScript. The code does not use the alpha channel when drawing to a PostScript or PDF surface (although the latter would be acceptable). However, the workflow was changed quite some time ago so that everything is drawn to an in-memory SVG surface. This means the alpha channel is used for all drawing. Then when the image is saved to a PostScript file, the code has to deal with the colors with an alpha channel (even if all colors have an alpha of 1.0 - Correction: works fine if all are 100% opaque) - thus the rasterization of the the PostScript. The PDF should be fine, but the code to convert PDF to PostScript also has to deal with the alpha channel issue - thus it rasterizes the images.

One solution is to have some mode option indicating whether or not to use the alpha channel (allow opacity). The software would then use that to determine whether to use colors with an alpha channel. If you know you will want PostScript output (not rasterized) then you would have to set it to disallow opacity.

Another (easier) solution is to use the "batch" file - output directly to the PostScript file. In this case the code knows it is producing PostScript and will not use the alpha channel. This, or course, is a solution for a script-type run, not an interactive session. But I will check that this works as expected.

I will investigate if I can get a more recent version of Cairo to work in PyFerret without messing up Qt (which uses Cairo as well). I am hoping this might solve some of the quality issues with Cairo images.

[karlmsmith]

Comment by @karlmsmith on 24 Dec 2014 22:17 UTC Using the script test2202.jnl:

set win /xinch=4 /thick=0.5 /asp=1 1
let v = 2*3.14159*x[gx=0:1:.01]
poly/hl=-1:1/vl=-1:1/key/pal=red/thick=1/opac=50 sin(v), cos(v)

If I run:

pyferret -nodisplay -batch test2202_batch.ps -script test2202.jnl 

I get a PostScript file with a red circle and key (so the /opac=50 was ignored) and a file size for test2202_batch.ps that is 45K. Examining the contents shows it defintely is a vector image (no rasterization).

If instead I run pyferret and save using a frame /file=...ps :

pyferret -nodisplay
yes? go test2202.jnl
set win /xinch=4 /thick=0.5 /asp=1 1
let v = 2*3.14159*x[gx=0:1:.01]
poly/hl=-1:1/vl=-1:1/key/pal=red/thick=1/opac=50 sin(v), cos(v)
yes? frame /file=test2202_interactive.ps
yes? quit

Examining the image file test2202_interactive.ps, it is part vector and part raster. The file size is 194K. The circle and key are pink but fuzzy, indicating the opacity was used but it had to rasterize the image for PostScript.

Interestingly, if I leave off the /opac=50, either method produces a pure vectorized image with size 45K. So as long as all opacities are 100%, it appears to handle generation of pure-vector PostScript just fine from the SVG which the 1.0 alpha channel.

[karlmsmith]

Comment by @karlmsmith on 24 Dec 2014 23:03 UTC Using the opaque version of the above script (no /opac=50), and saving both the PDF and PS using frame /file, gives files which are vector images, test2202_originter.ps about 46K, test2202_originter.pdf about 13K. Using these files for input with the following programs:

ps2ps test2202_originter.ps test2202_ps2ps.ps 
pstops '0(1.25in,5.75in)' test2202_originter.ps test2202_pstops.ps
pstopdf test2202_originter.ps test2202_pstopdf.pdf
pdftops test2202_originter.pdf test2202_pdftops.ps

gives the following files with sizes

  13K Dec 24 14:35 test2202_originter.pdf
  46K Dec 24 14:35 test2202_originter.ps
 3.2M Dec 24 14:36 test2202_pdftops.ps
  11K Dec 24 14:37 test2202_ps2pdf.pdf
  29K Dec 24 14:44 test2202_ps2ps.ps
  48K Dec 24 14:47 test2202_pstops.ps

So the problem program is pdftops. As Andrew mentioned, it might be that the program sees the /transparency and immediately decides to rasterize, even though every color is opaque.

So it appears the PyFerret output needs to be opaque PostScript, which then is manipulated with pstops.

I am not remembering the reason for trying to work with the PDF output from PyFerret. Maybe it was to try to support transparency. If this is the case, PostScript cannot be used and another route needs to be found.

[karlmsmith]

Comment by steven.c.hankin on 26 Dec 2014 20:38 UTC Ignore the following comment without hesitation, as I realize that I am being a "butt-inski".

Given the opening sentence, "''So the alpha channel (transparency) is fully supported for PDF v1.4 and later, but not for PostScript''", is there any hope that by hunting around for better graphics conversion utilities we may find simple solutions that do not involve pyFerret changes? PDF 1.4 dates from 2001, so shouldn't full alpha channel support in PDF be routine today? Speculation: Is it possible that we could side-step the problems by avoiding the use of PostScript?

Again, please ignore (with my apologies) if these thoughts come out of left field.

[karlmsmith]

Comment by @karlmsmith on 30 Dec 2014 18:51 UTC If I change the code to allow the alpha channel for everything in PDF's, Cairo rasterizes the PDFs; even when using Cairo 1.12 and insisting on PDF v1.4. The PDF is still vectorized if I provide a opaque background for the PDF and then do the drawing with the alpha channel onto that opaque surface. This is what is happening when saving the image with frame /file=...pdf. In this way the /OPAC=50 is respected (pink circle on a white background) but the plot is still vectorized (crisp lines at 3200x).

Using Cairo 1.12 does not improve sharpness of the PNG's even when requesting the best anti-aliasing.

"Using Cairo 1.12" means building the static libraries and linking them in prior to other system libraries. But other things use the system Cairo library (1.8) so the system Cairo library shows up in the ldd list (list of full-path shared-object libraries that are used for an executable or another shared-object library). Functions used to insist on PDF v1.4 are not found in system library, so it must be using (at least some of) the v1.12 library.

So yes, if we can use PDFs, that is preferred and need to find the right tools to manipulate PDFs. I had heard that these images need to be pulled into LaTex. I appears PDFs can be included in tex documents: http://tex.stackexchange.com/questions/139063/import-pdf-image-into-latex-file But I do not know what Tex produces from that (dvipdf is just a wrapper for dvips followed by conversion to PDF). However, there is pdftex that works natively with PDFs (PS needs to be covnerted to PDF using esptopdf). The pdftex program can produce dvi, or go directly to pdf. The following might have some clues on LaTex including PDFs: http://tex.stackexchange.com/questions/201884/no-image-in-pdf-using-pdf-tex?rq=1 So maybe pdftex could be a solution.

Still need to figure out a solution for crisper PNGs. Maybe a behind-the-scenes oversize and reduction.

Could check Qt again. But when I checked before, the first thing Qt requires is a connection to a display to get display parameters. So at that time a no-display version using Qt was not possible.

[karlmsmith]

Comment by @karlmsmith on 30 Dec 2014 22:13 UTC Regarding (3): the default window size is in inches. So the PNG image size in pixels created from the default window size will depend on the DPI of the display. For no-display (Cairo), 96 DPI is used for the resolution since that is close to most display resolutions at this time. It appears your display is 101 DPI (probably 100 DPI) from the PNG size you provided.

I am investigating setting the DPI to 200 in Cairo, but saving images as if it was 100 DPI, as a way of sharpening the PNG images.

[karlmsmith]

Comment by @karlmsmith on 31 Dec 2014 00:28 UTC Messing with the DPI and saving as a smaller PNG has limited benefits.

Investigating using Qt without a display. The "start-up" that I had missed before is using QCoreApplication instead of QApplication for the event loop. I should be able to copy and modify (mostly remove) stuff from the PyQt code for the displayed viewer to make another graphics "engine" for PyFerret. This might not take too much time to try out to see if it works and, if so, how fast it is.

[karlmsmith]

Comment by @AndrewWittenberg on 2 Jan 2015 20:52 UTC Thanks Karl for your work on these issues.

Regarding comment:13 -- I'd suggest making 96 DPI the default pixel size, so that things remain consistent no matter how PyFerret is run. But then, one could also provide a way for the user to explicitly "optimize" the dpi for their screen, e.g. by providing new symbols DISPLAY_XPIXEL, DISPLAY_YPIXEL:

SET WINDOW/XPIXEL=($DISPLAY_XPIXEL)/YPIXEL=($DISPLAY_YPIXEL)

Regarding PostScript and transparency, I'm fine with an interim solution that just disallows transparency for PostScript altogether, at whatever stage in graphics-generation pipeline that this needs to come in, as long as we can rotate/frame the PostScript files after-the-fact (e.g. using the offline PostScript tools as we have before). This would permit us to move our existing PostScript-based workflow into PyFerret, since we're not currently using transparency for anything. Then as we eventually begin to use transparency, we can work on how to incorporate PDFs into our workflow.

An alternative which may be easier at this point (certainly for the user, and maybe also the developer), would be to just add rotation & page-framing options to the FRAME command itself -- as I described in #2167 (see the last item (5) of the original ticket, and also the update at comment:4:ticket:2167).

Thank you again,

Andrew

[karlmsmith]

Comment by @karlmsmith on 3 Feb 2015 18:10 UTC Results from working with Qt as the "no-display" provider of graphics:

(A) the PNG (rasert) images are, as expected, good - identical to those from displayed graphics.

(B) the time to make these PNG images is slower than from Cairo, but faster (to much faster) than displayed graphics (since all the X-Windows communications are removed - the displayed gets slower the further the display machine is from the machine running pyferret). So the speed ratios to that of Ferret/XGKS were something like Cairo: 1.5-2x and displayed Qt: 5-6x; the no-display Qt is about 3x.

(C) use of Qt's fonts and text rendering is not possible in the no-display mode, apparently due to the Qt on RedHat not being configured for FontConfig: http://qt-project.org/doc/qt-4.8/threads-modules.html#painting-in-threads "Note that on X11 systems without FontConfig support, Qt cannot render text outside of the GUI thread". At this time that is not a problem since all text in ferret is rendered by ferret (other than /ANNOTATE of the FRAME command). But for plans of possibly using Qt to do all the text, this will be a problem. Using Qt for text rendering in displayed mode (in the "GUI thread") works fine, as can be demonstrated by FRAME /ANNOTATE.

(D) I am not able to output of PS and PDF. These outputs are produced using a QPrinter, and initialization of the QPrinter requires the displayed app (QApplication instead of QCoreApplication). Not sure at this time if there is an alternative, but given issue (C), I have not dug deeper into this issue.

So either back to tweak Cairo to make better PNG images, or look into another graphics package.

Still need to check the default line width scaling factor to see if it needs adjusting to match default line widths displayed by Ferret.

[karlmsmith]

Comment by @AndrewWittenberg on 3 Feb 2015 22:52 UTC I think matplotlib can produce PDFs without a display, using "AGG" as the graphics engine:

http://www.astrobetter.com/plotting-to-a-file-in-python/

Alternatively, is there any way to fool Qt into thinking there's a display, using a virtual framebuffer as we used to do with Ferret?

[karlmsmith]

Comment by @karlmsmith on 4 Feb 2015 19:07 UTC (Copying my email into the Trac ticket)

The Anti-Grain Geometry (AGG) library might be a possible alternative, although I have not examined all the details. The wikipedia article describes it as "a high-quality 2D rendering library written in C++. It features anti-aliasing and sub-pixel resolution. It is not a graphics library, per se, but rather a framework to build a graphics library upon". The down side is that development on it appears to have stopped in 2006. Wikipedia mentions the sudden death of the author; not sure what the SourceForge site is. Not sure what matplotlib actually uses as its "Agg" backend given the "framework" comment. Tempting to try just using matplotlib to do the drawing - see what the quality is and the time required to do it.

I noticed the GD graphics library http://libgd.github.io/ but they are only raster images. The provider of Python bindings to libgd https://github.com/NOAA-ORR-ERD/py_gd Chris Barker comments about gd: "For the project at hand we needed fast and simple drawing -- 8-bit color, no anti-aliasing. We also wanted a nice simple API to work with. There are a number of newer drawing libs (AGG, Skia) that produce some pretty results, but are not as simple to use, and are focused on 32 bit fully anti-aliased drawing. If you want the prettiest rendering possible, I encourage you to check those out. If you want something fast and simple -- py_gd may be for you." So probably not GD for us.

I had also seen the Skia graphics library https://code.google.com/p/skia/ which is used by Chrome, Chrome OS, Android, Firefox, etc. This sounds like a good possibility. It does mention PDF, but not PostScript (XPS is XML Paper Specification), so that might be an issue. They do mention subpixel text. They do have the "Picture" canvas for recording and playing back to another canvas. Looking briefly at the API documentation, resembles parts of both Qt and Cairo. Not sure if it works without a display. It is C++ (like Qt) but appears the Python bindings (equivalent to PyQt) are not very mature and are rather incomplete. So this would be C/C++ work.

[karlmsmith]

Comment by @karlmsmith on 5 Feb 2015 00:36 UTC So attached are the Cairo PNG plots from PyFerret with and without anti-aliasing (except in the annotations at the top are always anti-aliased).

I switched to using 10in, 20in, 30in, and 100in for the memory (SVG) surface, and saving as 720 pixel width PNG since the point size of the SVG surface would be an integer multiple of the pixel size of the PNG surface. That seems to have greatly improved things in my opinion. (Of course these are 20% larger than the 600 pixel PNGs I had been looking at.) The annotations certainly look much better. The anti-aliasing is definitely thickening up the double drawn axes and labels.

sst_cairo_direct : generating the plot by writing directly to a 720 pixel width raster surface without anti-aliasing. So this should be comparable to a Ferret 720 pixel wide plot.

So appears to be an issue with changes in s

[karlmsmith]

Comment by @karlmsmith on 5 Feb 2015 17:57 UTC The documentation for the Cairo SVG surface http://cairographics.org/manual/cairo-SVG-Surfaces.html does say the size of the plot is in units of points (1/72"). I assumed that was strictly for defining the bounding box when generating an actual SVG image. Similarly for PS and PDF. For image surfaces, of course, the size is in units of pixels.

For the recording surface http://cairographics.org/manual/cairo-Recording-Surfaces.html (introduced in Cairo 1.10 - RHEL6 uses Cairo 1.8), the documentation says "A recording surface is a surface that records all drawing operations at the highest level of the surface backend interface, (that is, the level of paint, mask, stroke, fill, and show_text_glyphs)". Optionally an origin and size of a bounding box can be given, but not units are associated with these values. This, along with the above results, makes me think there are very real differences between the the recording surface and the SVG surface. I had assumed they were practically identical since prior to 1.8, the recommendation was to just use an SVG surface as a recording surface, as I have done.

So I think the solution is to statically link in a private copy of the latest Cairo, and switch to using the recording surface. By linking in a private copy, I mean using compiler flags to resolve the Cairo function calls to the internally incorporated 1.10 function definitions during the build process and then obscuring their existence to the "outside".

[karlmsmith]

Comment by @karlmsmith on 6 Apr 2015 23:59 UTC I did discover this bug report indicating the fuzzy PNG issue is a Cairo bug: https://bugs.freedesktop.org/show_bug.cgi?id=30282 This is suppose to be fixed for a recording surface in Cairo v1.12 (or later).

I have successfully built PyFerret with Cairo v1.14 (hiding all linked-in libraries to hide it from Qt) and have it using a recording surface if available (which it is in v1.14). When using pixel size for the recording surface and then playing back to the image (PNG) surface of various sizes gives very crisp PNG images. So it does appear the bug is fixed, and maybe a point-sized recording surface can be used instead.

I changed Cairo behaviour so that if the user gives a pixel size in set window, then the DPI is set using either the user-provided or the ferret default inch size (dynamic DPI; no longer a hard-coded DPI unless pixel size not given). This has helped figure out the truncation of PDF and PS images that results from using a pixel-sized recording surface. Using a combination of point size and pixel size so the implied DPI is 72 or less, the complete PDF or PS image is drawn. If the implied DPI is larger than 72, then the image is truncated. Like it is drawing then scaling. But if I request a larger or smaller plot than the window size, that works fine. Maybe the recording (and certainly the SVG) surface expect to be in points and somehow that messes with things when replaying to PDF or PS; don't know.

So changed back to using points for the recording surface. This goes back to giving complete PDF, PS. Because of the Cairo bug fix, the PNG's from the recording surface are still sharp despite the scaling. So this appears to have fixed the fuzzy PNG issue and still have good PDF and PS plots.

Also, somewhere along the way, the saving of raster images in the PS went away. An option to not use the alpha channel is no longer a needed. Not sure if this is another Cairo bug that got fixed or if some change change ensured that the alpha value was always 255 for opaque colors.

Does mean I need to build the latest Cairo and Pixman libraries wherever we build PyFerret (including GFDL). But they are fairly easy builds. They are statically linked in, so regular users are not burdened by this additional requirement.

[karlmsmith]

Attachment from @AndrewWittenberg on 1 Oct 2014 23:10 UTC The demo scripts. cairo_vs_qt.tar.gz

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:37 UTC direct draw to PNG surface

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:37 UTC 10 in canvas with antialiasing

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:37 UTC 10 in canvas without anti-aliasing

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:38 UTC 20 in canvas with anti-aliasing

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:38 UTC 20 in canvas without anti-aliasing (/thicken=2)

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:39 UTC 30 in canvas with anti-aliasing (/thicken=3)

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:39 UTC 30 in canvas without anti-aliasing (/thicken=3)

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:39 UTC 100 in canvas with anti-aliasing (/thicken=10)

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 00:40 UTC 100 in canvas without anti-aliasing (/thicken=10)

[karlmsmith]

Attachment from @karlmsmith on 5 Feb 2015 18:25 UTC direct draw to PNG surface with anti-aliasing

[karlmsmith]

Attachment from @karlmsmith on 7 Apr 2015 00:02 UTC PNG from pyferret with Cairo-1.14 and using Cairo to draw text

[karlmsmith]

Attachment from @karlmsmith on 7 Apr 2015 00:04 UTC PDF version of same contourtest.pdf