search

GenericMappingTools / gmt

The Generic Mapping Tools
https://www.generic-mapping-tools.org
Other
843 stars 352 forks source link

Transparency doesn't work with ghostscript 9.28rc1 #1416

Closed seisman closed 5 years ago

seisman commented 5 years ago

Test script:

gmt begin transparency
gmt basemap -R0/10/0/4 -Jx1c -Baf
echo 2 2 | gmt plot -Sc2c -Gred
echo 5 2 | gmt plot -Sc2c -Gred@50
echo 8 2 | gmt plot -Sc2c -Gred -t50
gmt end show
PaulWessel commented 5 years ago

Make the PS, then run psconvert -S to see what the options to gs are and see if that is among the new problems they are trying to fix with the SAFER stuff.

PaulWessel commented 5 years ago

Try to add -dNOSAFER and see if that changes anything,

seisman commented 5 years ago

Output of psconvert -S:

$ gmt psconvert -A -Tf -S transparency.ps                                                                                                  
gs -q -dNOPAUSE -dBATCH -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -sDEVICE=pdfwrite  -g3160x1510 -r720 -sOutputFile='transparency.pdf' './psconvert_41626d.eps'

I then run the command above with the -dNOSAFER flag, the transparency works.

PaulWessel commented 5 years ago

OK, I have asked the question of the gs developers if this is intentional or not.

seisman commented 5 years ago

Another small issue is, when I run gmt psconvert -A -Tj -S transparency.ps, it prints two gs commands, with the second one starting with "psconvert [ERROR]:":

$ gmt psconvert -A -P -Tj -S transparency.ps
gs -q -dNOPAUSE -dBATCH -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=1 -dTextAlphaBits=1 -sDEVICE=pdfwrite  -g1317x629 -r300 -sOutputFile='transparency_intermediate.pdf' './psconvert_44138d.eps'
psconvert [ERROR]: gs -q -dNOPAUSE -dBATCH -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=jpeg -dJPEGQ=90 -g1317x629 -r300 -sOutputFile='transparency.jpg' 'transparency_intermediate.pdf'
PaulWessel commented 5 years ago

Per discussion with the gs people, we should add -dSAFER to now have access to transparency. Could you please try that and verify it works?

seisman commented 5 years ago

Do you mean -dSAFER or -dNOSAFER?

PaulWessel commented 5 years ago

Sorry meant -dNOSAFER.

seisman commented 5 years ago

Just tried this script and it works. I don't have gs9.27 anymore, you may also try if it works for gs 9.27.

gmt begin transparency pdf,jpg,png,bmp,eps,ppm,ps,tif A,C-dNOSAFER
gmt basemap -R0/10/0/4 -Jx1c -Baf
echo 2 2 | gmt plot -Sc2c -Gred
echo 5 2 | gmt plot -Sc2c -Gred@50
echo 8 2 | gmt plot -Sc2c -Gred -t50
gmt end show
PaulWessel commented 5 years ago

Yes, works fine. I will add the -dNOSAFER to psconvert now so we are ready for 9.28.

seisman commented 5 years ago

Just found some other possible bugs:

Script:

gmt begin transparency pdf,jpg E500
gmt basemap -R0/10/0/4 -Jx1c -Baf
echo 2 2 | gmt plot -Sc2c -Gred
echo 5 2 | gmt plot -Sc2c -Gred@50
echo 8 2 | gmt plot -Sc2c -Gred -t50
gmt end

Error message:

Error: /ioerror in --showpage--
Operand stack:
   1   true
Execution stack:
   %interp_exit   .runexec2   --nostringval--   showpage   --nostringval--   2   %stopped_push   --nostringval--   showpage   showpage   false   1   %stopped_push   1974   1   3   %oparray_pop   1973   1   3   %oparray_pop   1961   1   3   %oparray_pop   1962   1   3   %oparray_pop   showpage   showpage   2   1   1   showpage   %for_pos_int_continue   1965   1   7   %oparray_pop   showpage   showpage   1824   0   9   %oparray_pop   showpage   showpage
Dictionary stack:
   --dict:746/1123(ro)(G)--   --dict:1/20(G)--   --dict:80/200(L)--   --dict:80/200(L)--   --dict:135/256(ro)(G)--   --dict:315/325(ro)(G)--   --dict:33/64(L)--   --dict:6/9(L)--   --dict:6/20(L)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript RELEASE CANDIDATE 1 9.28: Unrecoverable error, exit code 1
psconvert [ERROR]: System call [gs -q -dNOPAUSE -dBATCH -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=jpeg -dJPEGQ=90 -g227549x227549 -r500 -sOutputFile='transparency.jpg' '/Users/seisman/.gmt/sessions/gmt6.62471/gmt_0_intermediate.pdf'] returned error 256.
end [ERROR]: Failed to call psconvert
end [ERROR]: process_figures returned error 78

If I change E500 to A,E500, it works again.

PaulWessel commented 5 years ago

Hm, and with -dNOSAFER?

seisman commented 5 years ago

Same error with gmt begin transparency pdf,jpg E500,C-dNOSAFER.

seisman commented 5 years ago

It seems -dNOSAFER works for most figure formats, except PNG (uppercase, not png).

With gmt begin transparency PNG A,C-dNOSAFER, the output png file is:

image

With gmt begin transparency PNG A, the output png figure is "correct" (but without transparency):

image

PaulWessel commented 5 years ago

I will be busy with mostly non-GMT stuff today. Could you make the plot even simpler (use -B0 for no ticks and annotations) and post th PS, then use -S to extract the commands, then make a shell script that just runs the gs commands on that PS file and produce the various outputs? I can then post that to gs later tonight. I usually hand-edit the pS to eliminate as much of our macros as possible first.

PaulWessel commented 5 years ago

Wait until the new PR has been committed. I ran all tests locally and no change(9.27).

seisman commented 5 years ago

Below is a minimum script to reproduce the first issue. The classic and modern commands are almost equivalent. 

# classic
gmt psbasemap -R0/10/0/5 -Jx1c -B0 -P > bug1c.ps
gmt psconvert -Tj -E500 -S bug1c.ps

# modern
gmt begin bug1m jpg E500,S
gmt basemap -R0/10/0/5 -Jx1c -B0
gmt end

The error messages are:

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=jpeg -dJPEGQ=90 -g4132x5848 -r500 -sOutputFile='bug1c.jpg' './psconvert_75500d.eps'

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=jpeg -dJPEGQ=90 -g227549x227549 -r500 -sOutputFile='bug1m.jpg' '/Users/seisman/.gmt/sessions/gmt6.75498/psconvert_75505d.eps'
Error: /ioerror in --showpage--
Operand stack:
   1   true
Execution stack:
   %interp_exit   .runexec2   --nostringval--   showpage   --nostringval--   2   %stopped_push   --nostringval--   showpage   showpage   false   1   %stopped_push   1974   1   3   %oparray_pop   1973   1   3   %oparray_pop   1961   1   3   %oparray_pop   1817   1   3   %oparray_pop   --nostringval--   %errorexec_pop   .runexec2   --nostringval--   showpage   --nostringval--   2   %stopped_push   --nostringval--   1824   0   4   %oparray_pop   showpage   showpage
Dictionary stack:
   --dict:745/1123(ro)(G)--   --dict:0/20(G)--   --dict:75/200(L)--   --dict:158/250(L)--
Current allocation mode is local
Last OS error: Invalid argument
Current file position is 20550
GPL Ghostscript RELEASE CANDIDATE 1 9.28: Unrecoverable error, exit code 1
psconvert [ERROR]: System call [gs -q -dNOPAUSE -dBATCH -dNOSAFER -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=jpeg -dJPEGQ=90 -g227549x227549 -r500 -sOutputFile='bug1m.jpg' '/Users/seisman/.gmt/sessions/gmt6.75498/psconvert_75505d.eps'] returned error 256.
end [ERROR]: Failed to call psconvert
end [ERROR]: process_figures returned error 78
seisman commented 5 years ago

Below is a minimum script to reproduce the PNG issue:

gmt psbasemap -R0/10/0/5 -Jx1c -B0 -B+gblue@50 > test.ps
gmt psconvert -TG -S test.ps

The test.ps is attached here test.ps.zip.

The GS commands are:

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=1 -dTextAlphaBits=1 -sDEVICE=pdfwrite  -g2480x3509 -r300 -sOutputFile='test_intermediate.pdf' './psconvert_76744d.eps'
gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=pngalpha  -g2480x3509 -r300 -sOutputFile='test.png' 'test_intermediate.pdf'
PaulWessel commented 5 years ago

Did you build 9.28rc1 from their tarball? I would like to give it a try as well.

seisman commented 5 years ago

I modified the ghostscript recipe of homebrew and installed 9.28rc1 by running brew reinstall ghostscript.

PaulWessel commented 5 years ago

Hi @seisman, the first issue bug1c.ps is trying to rasterize on a gigantic canvas (-g227549x227549 -r500); that cannot work I think and is more of a memory issue than gs bug. Can you check that the paper size is not crazy large in this case? Why would it select that huge canvas?

seisman commented 5 years ago

The same command works for PDF output format:

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -sDEVICE=pdfwrite  -g227549x227549 -r500 -sOutputFile='bug1m.pdf' '/Users/seisman/.gmt/sessions/gmt6.15910/psconvert_15913d.eps'

But for png, bmp, ppm, and tif formats, the script hangs.

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=png16m  -g227549x227549 -r500 -sOutputFile='bug1m.png' '/Users/seisman/.gmt/sessions/gmt6.16452/psconvert_16455d.eps'
joa-quim commented 5 years ago

I think it’s still the huge size. For PDF it doesn’t have to rasterize so the conversion is probably simpler. For the others that huge size comes in and it hangs trying to rasterize the image.

Sent from my iDedo

No dia 21/08/2019, às 20:41, Dongdong Tian notifications@github.com escreveu:

The same command works for PDF output format:

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -sDEVICE=pdfwrite -g227549x227549 -r500 -sOutputFile='bug1m.pdf' '/Users/seisman/.gmt/sessions/gmt6.15910/psconvert_15913d.eps' For png, bmp, ppm, and tif formats, the script hangs.

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=png16m -g227549x227549 -r500 -sOutputFile='bug1m.png' '/Users/seisman/.gmt/sessions/gmt6.16452/psconvert_16455d.eps' — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

seisman commented 5 years ago

Strictly speaking, the script doesn't hang. It keeps writing bytes to the disk. After killing the script, the file size of the output figure is as large as 4 GB.

PaulWessel commented 5 years ago

I dont understand? Why is it trying to render at that size? It is supposed to determine the bounding box first, then crop and do the actual rendering. I will see if I can reproduce but busy again with Chair prep stuff.

PaulWessel commented 5 years ago

Reproduced with 9.27 so I can debug this.

seisman commented 5 years ago 
gmt begin bug1m jpg E500,S
gmt basemap -R0/10/0/5 -Jx1c -B0
gmt end

There is no -A passed to psconvert, so I believe the canvas size is 10mx10m. Is -g227549x227549 equivalent to 10mx10m?

PaulWessel commented 5 years ago

It is 32767 points times 500.

PaulWessel commented 5 years ago

Seems out bug is that we are allowing psconvert to run without A when we know it is 10x10 meters... The user cannot turn off A unless asking for PostScript I think.

seisman commented 5 years ago

I agree. -A should be mandatory in modern mode non-PS plots.

PaulWessel commented 5 years ago

In light of the A fix, etc., could you please check that the test.ps crash still happens and can you try to use test.ps directly in those gs commands instead of the hidden and unavailable EPS file? I want to make things as simple to reproduce for the gs people as possible.

seisman commented 5 years ago

The first issue was fixed in #1429, but the PNG issue still happens.

The gs commands are:

gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=1 -dTextAlphaBits=1 -sDEVICE=pdfwrite  -g2480x3509 -r300 -sOutputFile='test_intermediate.pdf' './test.ps'
gs -q -dNOPAUSE -dBATCH -dNOSAFER -dPDFSETTINGS=/prepress -dDownsampleColorImages=false -dDownsampleGrayImages=false -dDownsampleMonoImages=false -dUseFlateCompression=true -dEmbedAllFonts=true -dSubsetFonts=true -dMonoImageFilter=/FlateEncode -dAutoFilterGrayImages=false -dGrayImageFilter=/FlateEncode -dAutoFilterColorImages=false -dColorImageFilter=/FlateEncode -dSCANCONVERTERTYPE=2 -dMaxBitmap=2147483647 -dUseFastColor=true -dGraphicsAlphaBits=2 -dTextAlphaBits=4 -sDEVICE=pngalpha  -g2480x3509 -r300 -sOutputFile='test.png' 'test_intermediate.pdf'
PaulWessel commented 5 years ago

Hi @seisman can you post the resulting PNG you get with 9.28rc1 for this? I can then email them the PS, the before and after PNGs and the commands.

seisman commented 5 years ago

Please see the attached PNG: test.png.zip

PaulWessel commented 5 years ago

Just so I understand: Compared to 9.27 the color seems more transparent and the right border is clipped because the whole image is stretched in the x-direction, and we see this because the vertical black border is much thicker than the horizontal border. Anything else?

seisman commented 5 years ago

I think that's all.

seisman commented 5 years ago

The bug has been fixed in ghostscript 9.28.

SeismoFelix commented 2 months ago

I have GMT version 6.4.0, with ghostscript 10.02.0. I run the transparency test and it did not work. I tried to use the transparency feature in this command:

gmt plot TXT/stations_used_coordinates.txt $region $projection -Si0.5c -W0.5p -GPALEVIOLETRED -t50 -V

and it is not working.

Screenshot 2024-07-06 at 3 59 29 AM

Do you have any suggestions? I would appreciate any.

Thanks.

seisman commented 2 months ago

Upgrade to GMT 6.5