pyproj4 / pyproj

Python interface to PROJ (cartographic projections and coordinate transformations library)
https://pyproj4.github.io/pyproj
MIT License
1.06k stars 214 forks source link

Supplying ellipsoid parameters prevents datum from being included in CRS #1061

Closed wjbenfold closed 2 years ago

wjbenfold commented 2 years ago

Code Sample, a copy-pastable example if possible

from pyproj.crs import CRS

crss = [
    CRS.from_user_input("+proj=merc +datum=WGS84 +ellps=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs"),
    CRS.from_user_input("+proj=merc +a=6378137.0 +rf=298.257223563 +datum=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs"),
]

for crs in crss:
    print(crs)
    print(crs.datum)
    print()
+proj=merc +datum=WGS84 +ellps=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs
World Geodetic System 1984

+proj=merc +a=6378137.0 +rf=298.257223563 +datum=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs
unknown

Problem description

Currently, passing the datum along with ellipsoid parameters (even accurate ones) when creating a CRS causes the datum to be silently ignored by pyproj. The datum is still visible when printing the CRS, but specifically interrogating the datum shows that it is not attached (and indeed transformations using the CRS that has been created will act as if the datum is not present).

Expected Output

I would be happy for the two examples above to be interpreted as equivalent, or for the user to be warned that the datum is being ignored (along with the datum not being visible when the CRS is printed). This would make it easier to work out what has gone wrong when this instance is encountered. There could alternatively be an error raised if this definition is considered invalid because a datum specifies the ellipse, or the ellipse could be modified using the parameters given (in the same way that providing parameters after an ellipse specification works).

Environment Information

$pyproj -v
pyproj info:
    pyproj: 3.3.0
      PROJ: 8.2.0
  data dir: .../.conda/envs/iris-dev/share/proj
user_data_dir: .../.local/share/proj
PROJ DATA (recommended version): 1.8
PROJ Database: 1.2
EPSG Database: v10.038 [2021-10-21]
ESRI Database: ArcMap 12.8 [2021-05-06]
IGNF Database: 3.1.0 [2019-05-24]

System:
    python: 3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51)  [GCC 9.4.0]
executable: .../.conda/envs/iris-dev/bin/python
   machine: Linux-3.10.0-1160.62.1.el7.x86_64-x86_64-with-glibc2.10

Python deps:
   certifi: 2021.10.08
       pip: 21.3.1
setuptools: 59.1.1
    Cython: None

Installation method

Conda

Conda environment information (if you installed with conda):


Environment (conda list):

``` $conda list # packages in environment at /home/h02/wbenfold/.conda/envs/iris-dev: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge alabaster 0.7.12 pyhd3eb1b0_0 antlr-python-runtime 4.7.2 py38h578d9bd_1003 conda-forge appdirs 1.4.4 pyhd3eb1b0_0 argcomplete 1.12.3 pyhd3eb1b0_0 asv 0.4.2 py38h709712a_3 conda-forge atk-1.0 2.36.0 ha1a6a79_0 babel 2.9.1 pyhd3eb1b0_0 backcall 0.2.0 pyhd3eb1b0_0 bokeh 2.1.1 py38_0 brotli 1.0.9 he6710b0_2 brotlipy 0.7.0 py38h27cfd23_1003 bzip2 1.0.8 h7b6447c_0 c-ares 1.18.1 h7f8727e_0 ca-certificates 2021.10.26 h06a4308_2 cairo 1.16.0 hf32fb01_1 cartopy 0.20.1 py38h677edd8_3 conda-forge certifi 2021.10.8 py38h06a4308_2 cf-units 3.0.1 py38h6c62de6_1 conda-forge cffi 1.15.0 py38hd667e15_1 cfgv 3.3.1 pyhd8ed1ab_0 conda-forge cftime 1.5.1.1 py38h6c62de6_1 conda-forge chardet 4.0.0 py38h06a4308_1003 charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge click 8.0.3 pyhd3eb1b0_0 cloudpickle 2.0.0 pyhd3eb1b0_0 colorama 0.4.4 pyhd3eb1b0_0 colorlog 4.8.0 py38h06a4308_1 cryptography 36.0.0 py38h9ce1e76_0 curl 7.80.0 h7f8727e_0 cycler 0.11.0 pyhd3eb1b0_0 cytoolz 0.11.0 py38h7b6447c_0 dask 2021.11.1 pyhd8ed1ab_0 conda-forge dask-core 2021.11.1 pyhd8ed1ab_0 conda-forge dbus 1.13.18 hb2f20db_0 debugpy 1.5.1 py38h295c915_0 decorator 5.1.1 pyhd3eb1b0_0 deepdiff 5.6.0 pyhd8ed1ab_0 conda-forge distlib 0.3.2 pyhd3eb1b0_0 distributed 2021.11.1 py38h578d9bd_0 conda-forge docutils 0.17.1 py38h06a4308_1 editdistance-s 1.0.0 py38h1fd1430_2 conda-forge entrypoints 0.3 py38_0 esmf 8.2.0 mpi_mpich_h4975321_100 conda-forge esmpy 8.2.0 mpi_mpich_py38h9147699_101 conda-forge expat 2.4.1 h2531618_2 filelock 3.4.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hd3eb1b0_0 font-ttf-inconsolata 2.001 hcb22688_0 font-ttf-source-code-pro 2.030 hd3eb1b0_0 font-ttf-ubuntu 0.83 h8b1ccd4_0 fontconfig 2.13.1 h6c09931_0 fonts-anaconda 1 h8fa9717_0 fonts-conda-ecosystem 1 hd3eb1b0_0 fonttools 4.25.0 pyhd3eb1b0_0 freetype 2.11.0 h70c0345_0 fribidi 1.0.10 h7b6447c_0 fsspec 2022.1.0 pyhd3eb1b0_0 g-ir-build-tools 1.70.0 py38h69e1f0e_1 conda-forge g-ir-host-tools 1.70.0 h601b988_1 conda-forge gdk-pixbuf 2.42.6 h8cc273a_5 geos 3.10.0 h9c3ff4c_0 conda-forge gettext 0.21.0 hf68c758_0 glib 2.70.2 h780b84a_1 conda-forge glib-tools 2.70.2 h780b84a_1 conda-forge gobject-introspection 1.70.0 py38h382169c_1 conda-forge graphite2 1.3.14 h23475e2_0 graphviz 2.49.3 h85b4f2f_0 conda-forge gst-plugins-base 1.14.0 h8213a91_2 gstreamer 1.14.0 h28cd5cc_2 gtk2 2.24.33 h539f30e_1 conda-forge gts 0.7.6 hb67d8dd_2 harfbuzz 3.1.2 h6b1f951_0 hdf4 4.2.15 h10796ff_3 conda-forge hdf5 1.12.1 mpi_mpich_h9c45103_3 conda-forge heapdict 1.0.1 pyhd3eb1b0_0 icu 58.2 he6710b0_3 identify 2.3.7 pyhd8ed1ab_0 conda-forge idna 3.3 pyhd3eb1b0_0 imagehash 4.2.1 pyhd8ed1ab_0 conda-forge imagesize 1.3.0 pyhd3eb1b0_0 importlib-metadata 4.8.2 py38h06a4308_0 importlib_metadata 4.8.2 hd3eb1b0_0 ipykernel 6.4.1 py38h06a4308_1 ipython 7.31.1 py38h06a4308_0 ipython_genutils 0.2.0 pyhd3eb1b0_1 iris-sample-data 2.4.0 pyhd8ed1ab_0 conda-forge jbig 2.1 hdba287a_0 jedi 0.18.1 py38h06a4308_0 jinja2 3.0.2 pyhd3eb1b0_0 jpeg 9d h7f8727e_0 jsonpickle 2.0.0 pyhd3eb1b0_0 jupyter_client 7.1.2 pyhd3eb1b0_0 jupyter_core 4.9.1 py38h06a4308_0 kiwisolver 1.3.1 py38h2531618_0 krb5 1.19.2 hac12032_0 lcms2 2.12 h3be6417_0 ld_impl_linux-64 2.35.1 h7274673_9 lerc 3.0 h295c915_0 libblas 3.9.0 13_linux64_openblas conda-forge libcblas 3.9.0 13_linux64_openblas conda-forge libcurl 7.80.0 h0b77cf5_0 libdeflate 1.8 h7f8727e_5 libedit 3.1.20210910 h7f8727e_0 libev 4.33 h7f8727e_1 libffi 3.4.2 h295c915_2 libgcc-ng 11.2.0 h1d223b6_11 conda-forge libgd 2.3.3 h695aa2c_0 libgfortran-ng 11.2.0 h69a702a_11 conda-forge libgfortran5 11.2.0 h5c6108e_11 conda-forge libgirepository 1.70.0 hb520f89_1 conda-forge libglib 2.70.2 h174f98d_1 conda-forge libgomp 11.2.0 h1d223b6_11 conda-forge libiconv 1.16 h516909a_0 conda-forge liblapack 3.9.0 13_linux64_openblas conda-forge libllvm11 11.1.0 h3826bc1_0 libmo_unpack 3.1.2 hf484d3e_0 conda-agnostic libnetcdf 4.8.1 mpi_mpich_h319fa22_1 conda-forge libnghttp2 1.46.0 hce63b2e_0 libnsl 2.0.0 h7f98852_0 conda-forge libopenblas 0.3.18 pthreads_h8fe5266_0 conda-forge libpng 1.6.37 hbc83047_0 librsvg 2.52.5 hc3c00ef_1 conda-forge libsodium 1.0.18 h7b6447c_0 libssh2 1.9.0 h1ba5d50_1 libstdcxx-ng 11.2.0 he4da1e4_11 conda-forge libtiff 4.3.0 h6f004c6_2 conda-forge libtool 2.4.6 h7b6447c_1005 libuuid 1.0.3 h7f8727e_2 libwebp-base 1.2.0 h27cfd23_0 libxcb 1.14 h7b6447c_0 libxml2 2.9.12 h03d6c58_0 libzip 1.8.0 h4de3113_1 conda-forge libzlib 1.2.11 h36c2ea0_1013 conda-forge llvmlite 0.37.0 py38h295c915_1 locket 0.2.1 py38h06a4308_1 lz4-c 1.9.3 h295c915_1 markupsafe 2.0.1 py38h27cfd23_0 matplotlib 3.5.0 py38h578d9bd_0 conda-forge matplotlib-base 3.5.0 py38h3ed280b_0 matplotlib-inline 0.1.2 pyhd3eb1b0_2 mo_pack 0.2.0 py38h6c62de6_1006 conda-forge mpi 1.0 mpich mpi4py 3.1.3 py38he865349_0 conda-forge mpich 3.4.3 h846660c_100 conda-forge msgpack-python 1.0.2 py38hff7bd54_1 munkres 1.1.4 py_0 nc-time-axis 1.4.0 pyhd8ed1ab_0 conda-forge ncurses 6.3 h7f8727e_2 nest-asyncio 1.5.1 pyhd3eb1b0_0 netcdf-fortran 4.5.4 mpi_mpich_h1364a43_0 conda-forge netcdf4 1.5.8 nompi_py38h2823cc8_101 conda-forge ninja 1.10.2 py38hd09550d_3 nodeenv 1.6.0 pyhd8ed1ab_0 conda-forge nose 1.3.7 py_1006 conda-forge nox 2021.10.1 pyhd8ed1ab_0 conda-forge numba 0.54.1 py38h51133e4_0 conda-main numpy 1.20.3 py38h9894fe3_1 conda-forge olefile 0.46 pyhd3eb1b0_0 openssl 1.1.1m h7f8727e_0 ordered-set 4.0.2 py_0 conda-forge packaging 21.3 pyhd3eb1b0_0 pandas 1.3.4 py38h43a58ef_1 conda-forge pango 1.48.10 h54213e6_2 conda-forge parso 0.8.3 pyhd3eb1b0_0 partd 1.2.0 pyhd3eb1b0_0 pcre 8.45 h295c915_0 pexpect 4.8.0 pyhd3eb1b0_3 pickleshare 0.7.5 pyhd3eb1b0_1003 pillow 6.2.2 py38h9776b28_0 conda-forge pip 21.3.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h7f8727e_1 pkg-config 0.29.2 h1bed415_8 pockets 0.9.1 py_0 conda-forge pre-commit 2.15.0 py38h578d9bd_1 conda-forge proj 8.2.0 h277dcde_0 conda-forge prompt-toolkit 3.0.20 pyhd3eb1b0_0 psutil 5.8.0 py38h27cfd23_1 ptyprocess 0.7.0 pyhd3eb1b0_2 py 1.10.0 pyhd3eb1b0_0 pycparser 2.21 pyhd3eb1b0_0 pygments 2.10.0 pyhd3eb1b0_0 pyke 1.1.1 pyhd8ed1ab_1004 conda-forge pyopenssl 21.0.0 pyhd3eb1b0_1 pyparsing 3.0.4 pyhd3eb1b0_0 pyproj 3.3.0 py38hdd21e9b_0 conda-forge pyqt 5.9.2 py38h05f1152_4 pyshp 2.1.3 pyhd3eb1b0_0 pysocks 1.7.1 py38h06a4308_0 python 3.8.12 hb7a2778_2_cpython conda-forge python-dateutil 2.8.2 pyhd3eb1b0_0 python-graphviz 0.16 pyhd3eb1b0_1 conda-main python-stratify 0.2.post0 py38h6c62de6_1 conda-forge python-xxhash 2.0.2 py38h497a2fe_1 conda-forge python_abi 3.8 2_cp38 conda-forge pytz 2021.3 pyhd3eb1b0_0 pywavelets 1.1.1 py38h7b6447c_2 pyyaml 6.0 py38h7f8727e_1 pyzmq 22.3.0 py38h295c915_2 qt 5.9.7 h5867ecd_1 readline 8.1.2 h7f8727e_1 requests 2.26.0 pyhd8ed1ab_0 conda-forge scipy 1.7.2 py38h56a6a73_0 conda-forge scitools-iris 3.3.dev0 dev_0 setuptools 59.1.1 py38h578d9bd_0 conda-forge shapely 1.8.0 py38hf7953bd_2 conda-forge sip 4.19.13 py38he6710b0_0 six 1.16.0 pyhd3eb1b0_0 snowballstemmer 2.2.0 pyhd3eb1b0_0 sortedcontainers 2.4.0 pyhd3eb1b0_0 sphinx 4.3.0 pyh6c4a22f_0 conda-forge sphinx-copybutton 0.4.0 pyhd8ed1ab_0 conda-forge sphinx-gallery 0.10.1 pyhd8ed1ab_0 conda-forge sphinx-panels 0.6.0 pyhd8ed1ab_0 conda-forge sphinx_rtd_theme 1.0.0 pyhd8ed1ab_0 conda-forge sphinxcontrib-applehelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-devhelp 1.0.2 pyhd3eb1b0_0 sphinxcontrib-htmlhelp 2.0.0 pyhd3eb1b0_0 sphinxcontrib-jsmath 1.0.1 pyhd3eb1b0_0 sphinxcontrib-napoleon 0.7 py_0 conda-forge sphinxcontrib-qthelp 1.0.3 pyhd3eb1b0_0 sphinxcontrib-serializinghtml 1.1.5 pyhd3eb1b0_0 sqlite 3.37.0 hc218d9a_0 tbb 2021.5.0 hd09550d_0 tblib 1.7.0 pyhd3eb1b0_0 tk 8.6.11 h1ccaba5_0 toml 0.10.2 pyhd3eb1b0_0 toolz 0.11.2 pyhd3eb1b0_0 tornado 6.1 py38h27cfd23_0 traitlets 5.1.1 pyhd3eb1b0_0 typing_extensions 3.10.0.2 pyh06a4308_0 udunits2 2.2.27.27 hc3e0081_3 conda-forge urllib3 1.26.7 pyhd3eb1b0_0 virtualenv 20.4.6 py38h06a4308_1 wcwidth 0.2.5 pyhd3eb1b0_0 wheel 0.37.1 pyhd3eb1b0_0 xxhash 0.8.0 h7f8727e_3 xz 5.2.5 h7b6447c_0 yaml 0.2.5 h7b6447c_0 zeromq 4.3.4 h2531618_0 zict 2.0.0 pyhd3eb1b0_0 zipp 3.7.0 pyhd3eb1b0_0 zlib 1.2.11 h36c2ea0_1013 conda-forge zstd 1.5.0 ha4553b6_1 ```


Details about conda and system ( conda info ):

``` $conda info active environment : iris-dev active env location : .../.conda/envs/iris-dev shell level : 1 user config file : .../.condarc populated config files : /etc/conda/condarc .../.condarc conda version : 4.11.0 conda-build version : not installed python version : 3.9.7.final.0 virtual packages : __linux=3.10.0=0 __glibc=2.17=0 __unix=0=0 __archspec=1=x86_64 base environment : /opt/conda (read only) conda av data dir : /opt/conda/etc/conda conda av metadata url : None channel URLs : ... package cache :.../.conda/pkgs envs directories : .../.conda/envs /opt/scitools/conda/environments /opt/conda/envs platform : linux-64 user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.7 Linux/3.10.0-1160.62.1.el7.x86_64 rhel/7.9 glibc/2.17 UID:GID : 12262:1000 netrc file : .../.netrc offline mode : False ```
wjbenfold commented 2 years ago

See also, https://github.com/SciTools/cartopy/issues/2035

snowman2 commented 2 years ago

Looking at this from the perspective of creating a Datum for a CRS may help: https://pyproj4.github.io/pyproj/stable/build_crs.html

When you specify +datum=WGS84 it could be a datum or an ensemble of datums. Each datum implicitly has a prime meridian & ellipsoid already defined:

>>> from pyproj.crs import Datum
>>> Datum.from_string("WGS84")
ENSEMBLE["World Geodetic System 1984 ensemble",
    MEMBER["World Geodetic System 1984 (Transit)",
        ID["EPSG",1166]],
    MEMBER["World Geodetic System 1984 (G730)",
        ID["EPSG",1152]],
    MEMBER["World Geodetic System 1984 (G873)",
        ID["EPSG",1153]],
    MEMBER["World Geodetic System 1984 (G1150)",
        ID["EPSG",1154]],
    MEMBER["World Geodetic System 1984 (G1674)",
        ID["EPSG",1155]],
    MEMBER["World Geodetic System 1984 (G1762)",
        ID["EPSG",1156]],
    MEMBER["World Geodetic System 1984 (G2139)",
        ID["EPSG",1309]],
    ELLIPSOID["WGS 84",6378137,298.257223563,
        LENGTHUNIT["metre",1],
        ID["EPSG",7030]],
    ENSEMBLEACCURACY[2.0],
    ID["EPSG",6326]]
>>> Datum.from_string("WGS84").name
'World Geodetic System 1984 ensemble'
>>> Datum.from_epsg(1309).ellipsoid
ELLIPSOID["WGS 84",6378137,298.257223563,
    LENGTHUNIT["metre",1],
    ID["EPSG",7030]]
>>> Datum.from_epsg(1309).prime_meridian
PRIMEM["Greenwich",0,
    ANGLEUNIT["degree",0.0174532925199433],
    ID["EPSG",8901]]

When you are specifying the +datum as well as ellipsoid parameters, it has to pick either the datum or the ellipsoid parameters to create the datum as it cannot override the existing datum presets. In this scenario, it is using the ellipsoid parameters:

>>> from pyproj.crs.datum import CustomDatum, CustomEllipsoid
>>> ell = CustomEllipsoid(semi_major_axis=6378137, inverse_flattening=298.257223563)
>>> ell
ELLIPSOID["undefined",6378137,298.257223563,
    LENGTHUNIT["metre",1,
        ID["EPSG",9001]]]
>>> datum = CustomDatum(ellipsoid=ell)
>>> datum
DATUM["undefined",
    ELLIPSOID["undefined",6378137,298.257223563,
        LENGTHUNIT["metre",1,
            ID["EPSG",9001]]]]

However, it has figured out the WGS84 ellipse in your scenario:

>>> from pyproj import CRS
>>> cc = CRS("+proj=merc +a=6378137.0 +rf=298.257223563 +datum=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs")
>>> cc.datum
DATUM["unknown",
    ELLIPSOID["WGS 84",6378137,298.257223563,
        LENGTHUNIT["metre",1,
            ID["EPSG",9001]]]]
snowman2 commented 2 years ago

The PROJ string parsing logic is handled by PROJ (https://github.com/OSGeo/PROJ/). I recommend raising an issue upstream if you have further questions or think there is an issue that needs to be addressed.

wjbenfold commented 2 years ago

Thanks @snowman2, I'll take it up with PROJ. I think there's an issue to be addressed in the way that information is silently dropped without indication that the provided string was invalid, but I've not been working with these strings for very long so would be interested to know if you think differently?

snowman2 commented 2 years ago

I think there's an issue to be addressed in the way that information is silently dropped without indication that the provided string was invalid

I think that would something to discuss with the PROJ devs (https://github.com/OSGeo/PROJ/). They have more background on the history and how that might impact other applications using PROJ.

snowman2 commented 2 years ago

This is also a helpful reference (note how the +datum= is dropped):

>>> from pyproj import CRS
>>> cc = CRS("+proj=merc +a=6378137.0 +rf=298.257223563 +datum=WGS84 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +units=m +no_defs +type=crs")
>>> cc.to_proj4()
UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems
  return self._crs.to_proj4(version=version)
'+proj=merc +lon_0=0 +k=1 +x_0=0 +y_0=0 +ellps=WGS84 +units=m +no_defs +type=crs'