GenericMappingTools / pygmt

A Python interface for the Generic Mapping Tools.
https://www.pygmt.org
BSD 3-Clause "New" or "Revised" License
758 stars 220 forks source link

DCW file not found when trying to generate the projection plots from the examples folder #316

Closed phloose closed 5 years ago

phloose commented 5 years ago

Description of the problem

When trying to plot the projections from the examples folder inside a jupyter notebook after compiling GMT from source i am not able to generate a single plot. It always shows the error that the Digital Chart of the World for GMT file is not found. Initially i wasn't aware that dcw-gmt.nc file was maybe needed and i didn't point the cmake file to any location. Then i downloaded it from here and tried to add it to ~/.gmt and to usr/local/share but without success. I tried to recompile with the specific setting in the cmake file but also without success. What am i doing wrong? Do i need an environment variable pointing to that location like this error from the gmt julia wrapper, even though he is running gmt on windows? I tried to set an environment variable GMT_SHAREDIR but this also didn't work.

Full code that generated the error cassini.py from examples/projections

"""
Cassini Cylindrical
============================

``Clon0/lat0/width``: ``lon0`` and ``lat0`` specifies the projection center.
"""
import pygmt

fig = pygmt.Figure()
# Use the ISO code for Madagascar (MG) and pad it by 2 degrees (+R2)
fig.coast(projection="C47/-19/8i", region="MG+R2", frame="afg", land="gray", borders=1)
fig.show()

Full error message

---------------------------------------------------------------------------
GMTCLibError                              Traceback (most recent call last)
 in 
      9 fig = pygmt.Figure()
     10 # Use the ISO code for Madagascar (MG) and pad it by 2 degrees (+R2)
---> 11 fig.coast(projection="C47/-19/8i", region="MG+R2", frame="afg", land="gray", borders=1)
     12 fig.show()

~/Dokumente/opensource/pygmt/pygmt/helpers/decorators.py in new_module(*args, **kwargs)
    185                 if alias in kwargs:
    186                     kwargs[arg] = kwargs.pop(alias)
--> 187             return module_func(*args, **kwargs)
    188 
    189         new_module.aliases = aliases

~/Dokumente/opensource/pygmt/pygmt/helpers/decorators.py in new_module(*args, **kwargs)
    280                         )
    281             # Execute the original function and return its output
--> 282             return module_func(*args, **kwargs)
    283 
    284         return new_module

~/Dokumente/opensource/pygmt/pygmt/base_plotting.py in coast(self, **kwargs)
    124         kwargs = self._preprocess(**kwargs)
    125         with Session() as lib:
--> 126             lib.call_module("coast", build_arg_string(kwargs))
    127 
    128     @fmt_docstring

~/Dokumente/opensource/pygmt/pygmt/clib/session.py in call_module(self, module, args)
    488             raise GMTCLibError(
    489                 "Module '{}' failed with status code {}:\n{}".format(
--> 490                     module, status, self._error_message
    491                 )
    492             )

GMTCLibError: Module 'coast' failed with status code 73:
pscoast [ERROR]: Unable to find or open the Digital Chart of the World for GMT
pscoast [ERROR]: Perhaps you did not install this file in DIR_DCW, the shared dir, or the user dir?
pscoast [ERROR]: Use your package manager to install package dcw-gmt.
pscoast [ERROR]: Alternatively, get the latest dcw-gmt-.tar.gz or dcw-gmt-.zip from the ftp://ftp.soest.hawaii.edu/gmt.
pscoast [ERROR]: Internal Error = GMT_MAP_BAD_LAT_MIN
pscoast [ERROR]: Internal Error = i

System information

output of conda list
$ conda list
# packages in environment at /home/mo3klo4r/miniconda3:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
asn1crypto                0.24.0                   py37_0  
atomicwrites              1.3.0                      py_0    conda-forge
attrs                     19.1.0                     py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
bleach                    3.1.0                      py_0    conda-forge
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2019.5.15                     0  
certifi                   2019.6.16                py37_0  
cffi                      1.12.2           py37h2e261b9_1  
chardet                   3.0.4                    py37_1  
conda                     4.7.5                    py37_0  
conda-package-handling    1.3.11                   py37_0  
cryptography              2.6.1            py37h1ba5d50_0  
cycler                    0.10.0                     py_1    conda-forge
dbus                      1.13.6               he372182_0    conda-forge
decorator                 4.4.0                      py_0    conda-forge
defusedxml                0.5.0                      py_1    conda-forge
entrypoints               0.3                   py37_1000    conda-forge
expat                     2.2.5             hf484d3e_1002    conda-forge
flake8                    3.7.7                    py37_0    conda-forge
fontconfig                2.13.1            he4413a7_1000    conda-forge
freetype                  2.10.0               he983fc9_0    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
glib                      2.58.3            hf63aee3_1001    conda-forge
gst-plugins-base          1.14.4            hdf3bae2_1001    conda-forge
gstreamer                 1.14.4            h66beb1c_1001    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
idna                      2.8                      py37_0  
ipykernel                 5.1.1            py37h24bf2e0_0    conda-forge
ipython                   7.5.0            py37h24bf2e0_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.4.2                      py_0    conda-forge
jedi                      0.13.3                   py37_0    conda-forge
jinja2                    2.10.1                     py_0    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
jsonschema                3.0.1                    py37_0    conda-forge
jupyter                   1.0.0                      py_2    conda-forge
jupyter_client            5.2.4                      py_3    conda-forge
jupyter_console           6.0.0                      py_0    conda-forge
jupyter_core              4.4.0                      py_0    conda-forge
kiwisolver                1.1.0            py37hc9558a2_0    conda-forge
libarchive                3.3.3                h5d8350f_5  
libblas                   3.8.0                9_openblas    conda-forge
libcblas                  3.8.0                9_openblas    conda-forge
libedit                   3.1.20181209         hc058e9b_0  
libffi                    3.2.1                hd88cf55_4  
libgcc-ng                 8.2.0                hdf63c60_1  
libgfortran-ng            7.3.0                hdf63c60_0  
libiconv                  1.15              h516909a_1005    conda-forge
liblapack                 3.8.0                9_openblas    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libsodium                 1.0.16            h14c3975_1001    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1  
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.9                h13577e0_0    conda-forge
lz4-c                     1.8.1.2              h14c3975_0  
lzo                       2.10                 h49e0be7_2  
markupsafe                1.1.1            py37h14c3975_0    conda-forge
matplotlib                3.0.3                    py37_1    conda-forge
matplotlib-base           3.0.3            py37h5f35d83_1    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
mistune                   0.8.4           py37h14c3975_1000    conda-forge
more-itertools            4.3.0                 py37_1000    conda-forge
nbconvert                 5.5.0                      py_0    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
ncurses                   6.1                  he6710b0_1  
notebook                  5.7.8                    py37_0    conda-forge
numpy                     1.16.3           py37he5ce36f_0    conda-forge
openblas                  0.3.6                h6e990d7_2    conda-forge
openssl                   1.1.1c               h7b6447c_1  
pandoc                    2.7.2                         0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.4.0                      py_0    conda-forge
pcre                      8.41              hf484d3e_1003    conda-forge
pexpect                   4.7.0                    py37_0    conda-forge
pickleshare               0.7.5                 py37_1000    conda-forge
pip                       19.0.3                   py37_0  
pluggy                    0.11.0                     py_0    conda-forge
prometheus_client         0.6.0                      py_0    conda-forge
prompt_toolkit            2.0.9                      py_0    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
ptyprocess                0.6.0                   py_1001    conda-forge
py                        1.8.0                      py_0    conda-forge
pycodestyle               2.5.0                      py_0    conda-forge
pycosat                   0.6.3            py37h14c3975_0  
pycparser                 2.19                     py37_0  
pycval                    0.1.0                     dev_0    
pyflakes                  2.1.1                      py_0    conda-forge
pygments                  2.4.0                      py_0    conda-forge
pyopenssl                 19.0.0                   py37_0  
pyparsing                 2.4.0                      py_0    conda-forge
pyqt                      5.9.2            py37hcca6a23_0    conda-forge
pyrsistent                0.15.2           py37h516909a_0    conda-forge
pysocks                   1.6.8                    py37_0  
pytest                    4.5.0                    py37_0    conda-forge
python                    3.7.3                h0371630_0  
python-dateutil           2.8.0                      py_0    conda-forge
python-libarchive-c       2.8                     py37_11  
pyzmq                     18.0.1           py37hc4ba49a_1    conda-forge
qt                        5.9.7                h52cfd70_1    conda-forge
qtconsole                 4.4.4                      py_0    conda-forge
readline                  7.0                  h7b6447c_5  
requests                  2.21.0                   py37_0  
ruamel_yaml               0.15.46          py37h14c3975_0  
send2trash                1.5.0                      py_0    conda-forge
setuptools                41.0.0                   py37_0  
sip                       4.19.8          py37hf484d3e_1000    conda-forge
six                       1.12.0                   py37_0  
sqlite                    3.27.2               h7b6447c_0  
tasks                     0.1.0                     dev_0    
terminado                 0.8.2                    py37_0    conda-forge
testpath                  0.4.2                   py_1001    conda-forge
tk                        8.6.9             h84994c4_1001    conda-forge
tornado                   6.0.2            py37h516909a_0    conda-forge
tqdm                      4.32.1                     py_0  
traitlets                 4.3.2                 py37_1000    conda-forge
urllib3                   1.24.1                   py37_0  
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.33.1                   py37_0  
widgetsnbextension        3.4.2                 py37_1000    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xz                        5.2.4                h14c3975_4  
yaml                      0.1.7                had09818_2  
zeromq                    4.3.1             hf484d3e_1000    conda-forge
zlib                      1.2.11               h7b6447c_3  
zstd                      1.3.7                h0b5b093_0  
welcome[bot] commented 5 years ago

👋 Thanks for opening your first issue here! Please make sure you filled out the template with as much detail as possible. You might also want to take a look at our contributing guidelines and code of conduct.

seisman commented 5 years ago

Please check if the following command works:

gmt coast -JC47/-19/8i -RMG+R2 -Bafg -Ggray -W1 -pdf map
phloose commented 5 years ago

Results in the same error:

$ gmt coast -JC47/-19/8i -RMG+R2 -Bafg -Ggray -W1 -pdf map
pscoast [ERROR]: Unable to find or open the Digital Chart of the World for GMT
pscoast [ERROR]: Perhaps you did not install this file in DIR_DCW, the shared dir, or the user dir?
pscoast [ERROR]: Use your package manager to install package dcw-gmt.
pscoast [ERROR]: Alternatively, get the latest dcw-gmt-<version>.tar.gz or dcw-gmt-<version>.zip from the ftp://ftp.soest.hawaii.edu/gmt.
pscoast [ERROR]: Internal Error = GMT_MAP_BAD_LAT_MIN
pscoast [ERROR]: Internal Error = i
pscoast (GMT_pscoast): South is outside -90 to +90 degree range
seisman commented 5 years ago

Please run it again with an extra -Vd option and paste the full debug logs.

gmt coast -JC47/-19/8i -RMG+R2 -Bafg -Ggray -W1 -Vd -pdf map
phloose commented 5 years ago

Here you go:

$ gmt coast -JC47/-19/8i -RMG+R2 -Bafg -Ggray -W1 -Vd -pdf map
gmt [DEBUG]: Obtained the ppid from parent: 5491
gmt [DEBUG]: Enter: gmtinit_new_GMT_ctrl
gmt [DEBUG]: GMT->session.SHAREDIR = /usr/local/share/gmt
gmt [DEBUG]: GMT->session.HOMEDIR = /home/mo3klo4r
gmt [DEBUG]: GMT->session.USERDIR = /home/mo3klo4r/.gmt [created]
gmt [DEBUG]: GMT->session.CACHEDIR = /home/mo3klo4r/.gmt/cache [created]
gmt [DEBUG]: GMT: 0. Will try to find subdir=postscriptlight stem = PSL_custom_fonts suffix=.txt
gmt [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
gmt [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR /home/mo3klo4r/.gmt
gmt [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir /home/mo3klo4r/.gmt/postscriptlight
gmt [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir /usr/local/share/gmt/postscriptlight
gmt [DEBUG]: GMT: 5. gmt_getsharepath failed
gmt [DEBUG]: Map distance calculation will be Cartesian
gmt [DEBUG]: Exit:  gmtinit_new_GMT_ctrl
gmt [DEBUG]: Enter: New_PSL_Ctrl
gmt [DEBUG]: Exit:  New_PSL_Ctrl
gmt [DEBUG]: Enter: gmt_manage_workflow
gmt [DEBUG]: Exit : gmt_manage_workflow
gmt [DEBUG]: Enter: PSL_beginsession
gmt [DEBUG]: Exit : PSL_beginsession
gmt [DEBUG]: Enter: PSL_setdefaults
gmt [DEBUG]: Exit : PSL_setdefaults
gmt [DEBUG]: Enter: gmtlib_io_init
gmt [DEBUG]: Exit : gmtlib_io_init
gmt [DEBUG]: Enter: gmt_hash_init
gmt [DEBUG]: Exit:  gmt_hash_init
gmt [DEBUG]: Enter: gmt_hash_init
gmt [DEBUG]: Exit:  gmt_hash_init
gmt [DEBUG]: The PROJ_GEODESIC set to Vincenty
gmt [DEBUG]: Enter: gmt_getdefaults
gmt [DEBUG]: Look for file /home/mo3klo4r/gmt.conf
gmt [DEBUG]: Look for file /home/mo3klo4r/.gmt/gmt.conf
gmt [DEBUG]: Look for file /home/mo3klo4r/.gmt/server/gmt.conf
gmt [DEBUG]: Look for file /home/mo3klo4r/.gmt/cache/gmt.conf
gmt [DEBUG]: Could not find file gmt.conf
gmt [DEBUG]: Exit:  gmt_getdefaults
gmt [DEBUG]: Enter: gmtlib_plot_C_format
gmt [DEBUG]: Exit:  gmtlib_plot_C_format
gmt [DEBUG]: Enter: gmtinit_get_history
gmt [DEBUG]: GMT_Create_Session initialized GMT structure
gmt [DEBUG]: Shared Library # 0 (core). Path = libgmt.so
gmt [DEBUG]: Loading core GMT shared library: libgmt.so
gmt [DEBUG]: Loading GMT plugins from: /usr/local/lib/gmt/plugins
gmt [DEBUG]: Shared Library # 1 (supplements). Path = /usr/local/lib/gmt/plugins/supplements.so
begin [INFORMATION]: Creating a workflow directory /home/mo3klo4r/.gmt/sessions/gmt6.5491
begin [DEBUG]: The PROJ_GEODESIC set to Vincenty
begin [DEBUG]: Look for file /home/mo3klo4r/gmt.conf
begin [DEBUG]: Look for file /home/mo3klo4r/.gmt/gmt.conf
begin [DEBUG]: Look for file /home/mo3klo4r/.gmt/server/gmt.conf
begin [DEBUG]: Look for file /home/mo3klo4r/.gmt/cache/gmt.conf
begin [DEBUG]: Could not find file gmt.conf
begin [DEBUG]: Set session name to be map pdf
begin [DEBUG]: Begin Workflow.  Session ID = 5491. Directory /home/mo3klo4r/.gmt/sessions/gmt6.5491 created.
begin [DEBUG]: GMT now running in modern mode [Session ID = 5491]
gmt [DEBUG]: Use PS filename /home/mo3klo4r/.gmt/sessions/gmt6.5491/gmt_0.ps-
gmt [DEBUG]: get_current_panel: No current panel selected so not in subplot mode
pscoast [DEBUG]: History: Process -JC47/-19/8i
pscoast [DEBUG]: History: Process -RMG+R2
pscoast [DEBUG]: Map distance calculation will be using great circle approximation with authalic auxiliary latitudes and authalic (R_2) radius = 6371007.1809 m, in meter.
pscoast [DEBUG]: Look for file MG+R2 in /home/mo3klo4r/.gmt
pscoast [DEBUG]: Look for file MG+R2 in /home/mo3klo4r/.gmt/cache
pscoast [DEBUG]: Look for file MG+R2 in /home/mo3klo4r/.gmt/server
pscoast [DEBUG]: Look for file MG+R2 in /home/mo3klo4r/.gmt/server/srtm1
pscoast [DEBUG]: Look for file MG+R2 in /home/mo3klo4r/.gmt/server/srtm3
pscoast [DEBUG]: GMT: 0. Will try to find subdir=dcw stem = dcw-countries suffix=.txt
pscoast [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
pscoast [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR /home/mo3klo4r/.gmt
pscoast [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir /home/mo3klo4r/.gmt/dcw
pscoast [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir /usr/local/share/gmt/dcw
pscoast [DEBUG]: GMT: 5. gmt_getsharepath failed
pscoast [ERROR]: Unable to find or open the Digital Chart of the World for GMT
pscoast [ERROR]: Perhaps you did not install this file in DIR_DCW, the shared dir, or the user dir?
pscoast [ERROR]: Use your package manager to install package dcw-gmt.
pscoast [ERROR]: Alternatively, get the latest dcw-gmt-<version>.tar.gz or dcw-gmt-<version>.zip from the ftp://ftp.soest.hawaii.edu/gmt.
pscoast [DEBUG]: Map distance calculation will be using great circle approximation with authalic auxiliary latitudes and authalic (R_2) radius = 6371007.1809 m, in meter.
pscoast [ERROR]: Internal Error = GMT_MAP_BAD_LAT_MIN
pscoast [ERROR]: Internal Error = i
pscoast (GMT_pscoast): South is outside -90 to +90 degree range
pscoast (Free_Ctrl): tried to free unallocated memory
pscoast (Free_Ctrl): tried to free unallocated memory
pscoast [DEBUG]: GMT now running in modern mode [Session ID = 5491]
pscoast (gmtlib_free_tmp_arrays): tried to free unallocated memory
end [DEBUG]: End Workflow.  Session ID = 5491. Directory /home/mo3klo4r/.gmt/sessions/gmt6.5491 removed.
end [DEBUG]: No figure file /home/mo3klo4r/.gmt/sessions/gmt6.5491/gmt.figures - nothing to do
end [DEBUG]: Got session name as map and default graphics formats as pdf
end [INFORMATION]: Process GMT figure queue: 1 figures found
end [INFORMATION]: Processing GMT figure #0 [map pdf ]
end [INFORMATION]: Destroying the current workflow directory /home/mo3klo4r/.gmt/sessions/gmt6.5491
end [DEBUG]: Delete gmt.conf
end [DEBUG]: Delete gmt.frame
end [DEBUG]: Delete gmt.session
end (gmtlib_free_tmp_arrays): tried to free unallocated memory
pscoast (gmtlib_free_tmp_arrays): tried to free unallocated memory
gmt [DEBUG]: Entering GMT_Destroy_Session
seisman commented 5 years ago

You can try to put all the dcw files (one .nc file and two .txt files) under the directory /home/mo3klo4r/.gmt/dcw and see if it works.

phloose commented 5 years ago

Ok now it works, but i get a different error concering GSHHG:

pscoast [ERROR]: GSHHG version 2.2.0 or newer is needed to use coastlines with GMT.
    Get and install GSHHG from ftp://ftp.soest.hawaii.edu/gshhg/.
pscoast [ERROR]: Could not find file [GSHHG high resolution shorelines]
pscoast [ERROR]: Could not find file [GSHHG high resolution political boundaries]
pscoast [ERROR]: No GSHHG databases available - must abort

But as this is not part of this issue i would consider it as solved.

Thank you very much for your fast help!!

Though i have to mention, that it is not very clear how to install or download dcw-gmt or gshhg from the build instructions, especially that i have to name a folder dcw (whether in the .../share or .gmt folder) and not only put in there the .nc file but ALSO the two .txt files.

seisman commented 5 years ago

The reason is the same, GMT cannot find your GSHHGG data.

When building GMT, it's recommended to at least use following configurations:

set (CMAKE_INSTALL_PREFIX /opt/gmt)
set (GSHHG_ROOT /path/to/gshhg)
set (DCW_ROOT /path/to/dcw)
set (COPY_GSHHG true)
set (COPY_DCW true)

If COPY_GSHHG and COPY_DCW are set to TRUE, it will automatically copy the GSHHG and DCW data to the right place (usually, GMT/share/coast and GMT/share/dcw).

phloose commented 5 years ago

Yeah i assumed that. I fixed the GSHHG issue by placing a gmt.conf into the .gmt folder which points to the gshhg directory.

The thing is, that it is not mentioned that DCW or GSHHG is necessarily needed to run GMT successfully on the build instructions page. Only that i have to place the path to it in the respective settings in the cmake file. For my feeling it should be mentioned in a small sentence that one needs DCW and GSHHG before building to not have a hassle afterwards and a link where to get them.

I used gmt some time ago but on windows, where everything was bundled together, so no need to build it from source.

Thanks again for your help!

leouieda commented 5 years ago

Hi @xibalba01 those datasets are the kind of dependency that isn't "required" but you can't really do much without them. But they are listed as requirements in the building instructions: https://github.com/GenericMappingTools/gmt/blob/master/BUILDING.md#build-and-runtime-dependencies

Thanks for giving this a try! Did you manage to get everything working?

phloose commented 5 years ago

@leouieda yes i got it up and running! I looked up the link you provided and maybe i just skimmed to fast over that dependency list on the first try and overlooked GSHHG and DCW... I mean it is mentioned that one needs that support data, but not why, i.e. that no single plot out of the box is working :)

I also stumbled over the respective path settings in the configuration step and was confused about that settings so i left it out. Maybe a reminder that the support data is really needed and a link to the dependency list (or a download link) would help here. Just a suggestion from a lazy reader :)

leouieda commented 5 years ago

I mean it is mentioned that one needs that support data, but not why, i.e. that no single plot out of the box is working :)

Yeah, that is a good point. We're hoping that once GMT 6 is actually released we can let the conda package do all the dependency management. Closing this for now, OK?