corteva / rioxarray

geospatial xarray extension powered by rasterio
https://corteva.github.io/rioxarray
Other
511 stars 81 forks source link

Distortion in output geotiffs for non-contiguous blocks of geospatial data #543

Closed phlamirande closed 2 years ago

phlamirande commented 2 years ago

Problem description

Attached data : clusters.csv I have a geospatial dataset, a single csv containing x, y and clusters data (about 147 000 cells). The cell size is 10m and the CRS is epsg:26911. The dataset comprises two distinct yet nearby areas : image I want to convert this csv data to geotiff by using rioxarray.to_raster function.

Expected Output

I would expect the raster to look like this image

I rather get a raster where the cells are slightly offset from where they should be. The cell size is now 10.48xxxxx m when I was expecting a 10 m. See below on the image, the black transparent layer is the problematic one created with rioxarray while the ''correct'' colored one is the output created with geocube. image

I found that this behavior happens only when you have two or more distrinct / non-contiguous spatial blocks in your data. Otherwise, if the data is in a single block (cell that touches), the output is fine.

Code


import xarray
import rioxarray as rio
import time
import pandas as pd

def test_csv_to_raster(input_csv, output_directory, columns, x_field, y_field, dstCRS):

    start_time = time.time()

    df = pd.read_csv(input_csv)
    df.rename(columns={x_field: 'x', y_field: 'y'}, inplace = True)
    df.set_index(['x', 'y'], inplace=True)

    xr = xarray.Dataset.from_dataframe(df)
    xr = xr.rio.set_spatial_dims('x', 'y')
    xr = xr.rio.write_crs(dstCRS)

    xr = xr.transpose('y', 'x')
    xr = xr.fillna(-9999)

    if (os.path.exists(output_directory) != True):
        os.makedirs(output_directory)

    print("Rasterizing...")

    for col in columns:
        print("Column : " + col)
        output_path = os.path.join(output_directory, col + ".tif")

        xr[col] = xr[col].astype(dtype='int16')
        xr[col] = xr[col].where(xr[col] != -9999)

        xr[col].rio.to_raster(raster_path=output_path, nodata=-9999)
        print("Done.  See file " + output_path)

    end_time = time.time()
    print("Completed. Time elapsed (s) : ")
    print(end_time - start_time)
    return

cl_start = 5
cl_end = 30
crs='epsg:26911'
input_csv = 'clusters.csv'
columns = [f'labels_n{i}' for i in range(cl_start, cl_end+1)]
output_directory = 'my_rasters/'

test_csv_to_raster(input_csv, output_directory, columns, 'x', 'y', crs)

Environment Information

I use QGIS for visualisation of the geotiffs.

image

Installation method

conda

Conda environment information (if you installed with conda):


Environment (conda list):

``` $ # packages in environment at C:\Users\Utilisateur\.conda\envs\geods37: # # Name Version Build Channel _py-xgboost-mutex 2.0 cpu_0 abseil-cpp 20210324.2 hd77b12b_0 affine 2.3.0 py_0 conda-forge alembic 1.7.5 pyhd3eb1b0_1 annoy 1.17.0 pypi_0 pypi appdirs 1.4.4 pyh9f0ad1d_0 conda-forge argcomplete 1.12.3 pyhd8ed1ab_2 conda-forge argon2-cffi 21.1.0 py37hcc03f2d_0 conda-forge arrow-cpp 6.0.0 py37h88b90bb_3_cpu conda-forge asciitree 0.3.3 py_2 async_generator 1.10 py_0 conda-forge atomicwrites 1.4.0 py_0 attrs 21.2.0 pyhd8ed1ab_0 conda-forge autopage 0.4.0 pyhd8ed1ab_0 conda-forge aws-c-auth 0.6.4 hd061354_3 conda-forge aws-c-cal 0.5.12 h395cb70_2 conda-forge aws-c-common 0.6.11 h8ffe710_0 conda-forge aws-c-compression 0.2.14 h82db3a7_2 conda-forge aws-c-event-stream 0.2.7 he02faa2_24 conda-forge aws-c-http 0.6.6 h77bbe4a_1 conda-forge aws-c-io 0.10.9 h3a8db39_3 conda-forge aws-c-mqtt 0.7.8 h2f34d13_2 conda-forge aws-c-s3 0.1.27 habe44f2_1 conda-forge aws-checksums 0.1.12 h82db3a7_1 conda-forge aws-crt-cpp 0.17.1 haf3871f_3 conda-forge aws-sdk-cpp 1.9.120 h37b9141_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.11.1 py37haa95532_0 blas 1.0 mkl bleach 4.1.0 pyhd8ed1ab_0 conda-forge bokeh 2.4.0 py37h03978a9_0 conda-forge boost-cpp 1.74.0 h5b4e17d_4 conda-forge boto3 1.18.59 pyhd8ed1ab_0 conda-forge botocore 1.21.59 pyhd8ed1ab_0 conda-forge bottleneck 1.3.4 py37h080aedc_0 branca 0.4.2 pyhd8ed1ab_0 conda-forge brotlipy 0.7.0 py37hcc03f2d_1001 conda-forge bs4 4.11.1 hd3eb1b0_0 bzip2 1.0.8 h8ffe710_4 conda-forge c-ares 1.18.1 h2bbff1b_0 ca-certificates 2022.4.26 haa95532_0 cachetools 4.2.4 pyhd8ed1ab_0 conda-forge cairo 1.16.0 hb19e0ff_1008 conda-forge catboost 0.26.1 py37h03978a9_0 conda-forge certifi 2022.5.18.1 py37haa95532_0 cffi 1.14.6 py37hd8e9650_1 conda-forge cfitsio 3.470 h0af3d06_7 conda-forge cftime 1.5.1 py37hec80d1f_0 conda-forge chardet 4.0.0 py37h03978a9_1 conda-forge charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cliff 3.9.0 pyhd8ed1ab_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge cloudpickle 2.0.0 pyhd8ed1ab_0 conda-forge cmaes 0.8.2 pyh44b312d_0 conda-forge cmd2 2.2.0 py37h03978a9_1 conda-forge colorama 0.4.4 pyh9f0ad1d_0 conda-forge colorcet 2.0.6 pyhd3eb1b0_0 colorlog 5.0.1 py37haa95532_1 cryptography 3.4.8 py37h20c650d_0 conda-forge curl 7.79.1 h789b8ee_1 conda-forge cv2-tools 2.4.0 pypi_0 pypi cycler 0.10.0 py_2 conda-forge cytoolz 0.11.0 py37hcc03f2d_3 conda-forge dask 2021.9.1 pyhd8ed1ab_0 conda-forge dask-core 2021.9.1 pyhd8ed1ab_0 conda-forge datacube 1.8.6 pyhd8ed1ab_0 conda-forge datashader 0.13.0 pyhd3eb1b0_1 datashape 0.5.2 pypi_0 pypi debugpy 1.4.1 py37hf2a7229_0 conda-forge decorator 5.1.0 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge distributed 2021.9.1 py37h03978a9_0 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge esda 2.4.1 pyhd8ed1ab_0 conda-forge et_xmlfile 1.1.0 py37haa95532_0 expat 2.4.1 h39d44d4_0 conda-forge fake-geo-images 0.1.3 pypi_0 pypi fasteners 0.16.3 pyhd3eb1b0_0 fiona 1.8.20 py37h9dd7a2e_1 conda-forge folium 0.12.0 pyhd8ed1ab_1 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.13.1 h1989441_1005 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.10.4 h546665d_1 conda-forge freexl 1.0.6 ha8e266a_0 conda-forge fsspec 2021.10.0 pyhd8ed1ab_0 conda-forge gdal 3.3.2 py37h1349d40_4 conda-forge geocube 0.0.18 pyhd8ed1ab_0 conda-forge geopandas 0.10.1 pyhd8ed1ab_0 conda-forge geopandas-base 0.10.1 pyha770c72_0 conda-forge geos 3.9.1 h39d44d4_2 conda-forge geotiff 1.7.0 h350af67_3 conda-forge gettext 0.19.8.1 ha2e2712_1008 conda-forge gflags 2.2.2 ha925a31_0 glog 0.5.0 hd77b12b_0 greenlet 1.1.2 py37hf2a7229_0 conda-forge grpc-cpp 1.41.1 ha2fd154_2 conda-forge hdbscan 0.8.27 py37hda49f71_0 conda-forge hdf4 4.2.15 h0e5069d_3 conda-forge hdf5 1.12.1 nompi_h2a0e4a3_101 conda-forge heapdict 1.0.1 py_0 conda-forge holoviews 1.14.6 pypi_0 pypi icu 68.1 h0e60522_0 conda-forge idna 3.1 pyhd3deb0d_0 conda-forge imagehash 4.2.1 pypi_0 pypi imageio 2.19.3 pypi_0 pypi imbalanced-learn 0.8.1 pyhd8ed1ab_0 conda-forge importlib-metadata 4.8.1 py37h03978a9_0 conda-forge importlib_metadata 4.8.1 hd8ed1ab_0 conda-forge importlib_resources 5.2.0 pyhd3eb1b0_1 iniconfig 1.1.1 pyhd3eb1b0_0 intel-openmp 2021.3.0 h57928b3_3372 conda-forge ipykernel 6.4.1 py37h4038f58_0 conda-forge ipython 7.28.0 py37h4038f58_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge jbig 2.1 h8d14728_2003 conda-forge jedi 0.18.0 py37h03978a9_2 conda-forge jinja2 3.0.2 pyhd8ed1ab_0 conda-forge jmespath 0.10.0 pyh9f0ad1d_0 conda-forge joblib 1.1.0 pyhd8ed1ab_0 conda-forge jpeg 9d h8ffe710_0 conda-forge jsonschema 4.1.0 pyhd8ed1ab_0 conda-forge jupyter 1.0.0 py37h03978a9_6 conda-forge jupyter_client 7.0.6 pyhd8ed1ab_0 conda-forge jupyter_console 6.4.0 pyhd8ed1ab_0 conda-forge jupyter_core 4.8.1 py37h03978a9_0 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_widgets 1.0.2 pyhd8ed1ab_0 conda-forge kealib 1.4.14 h8995ca9_3 conda-forge kiwisolver 1.3.2 py37h8c56517_0 conda-forge krb5 1.19.2 hbae68bd_2 conda-forge lark-parser 0.12.0 pyhd8ed1ab_0 conda-forge lcms2 2.12 h2a16943_0 conda-forge lerc 2.2.1 h0e60522_0 conda-forge libblas 3.9.0 11_win64_mkl conda-forge libbrotlicommon 1.0.9 h8ffe710_6 conda-forge libbrotlidec 1.0.9 h8ffe710_6 conda-forge libbrotlienc 1.0.9 h8ffe710_6 conda-forge libcblas 3.9.0 11_win64_mkl conda-forge libclang 11.1.0 default_h5c34c98_1 conda-forge libcurl 7.79.1 h789b8ee_1 conda-forge libdeflate 1.7 h8ffe710_5 conda-forge libffi 3.4.2 h0e60522_4 conda-forge libgdal 3.3.2 h08edb8f_4 conda-forge libglib 2.68.4 h3be07f2_1 conda-forge libiconv 1.16 he774522_0 conda-forge libkml 1.3.0 h9859afa_1014 conda-forge liblapack 3.9.0 11_win64_mkl conda-forge libnetcdf 4.8.1 nompi_h1cc8e9d_101 conda-forge libpng 1.6.37 h1d00b33_2 conda-forge libpq 13.3 hfcc5ef8_1 conda-forge libprotobuf 3.18.1 h7755175_0 conda-forge libpysal 4.5.1 pyhd8ed1ab_0 conda-forge librttopo 1.1.0 hb340de5_6 conda-forge libsodium 1.0.18 h8d14728_1 conda-forge libspatialindex 1.9.3 h39d44d4_4 conda-forge libspatialite 5.0.1 h6a90f17_9 conda-forge libssh2 1.10.0 h680486a_2 conda-forge libthrift 0.15.0 he1d8c1a_0 libtiff 4.3.0 h0c97f57_1 conda-forge libutf8proc 2.6.1 h2bbff1b_0 libwebp-base 1.2.1 h8ffe710_0 conda-forge libxgboost 1.5.0 h0e60522_0 conda-forge libxml2 2.9.12 hf5bbc77_0 conda-forge libzip 1.8.0 hfed4ece_1 conda-forge libzlib 1.2.11 h8ffe710_1013 conda-forge lightgbm 3.2.1 py37hf2a7229_0 conda-forge llvmlite 0.38.0 py37h23ce68f_0 locket 0.2.0 py_2 conda-forge lz4-c 1.9.3 h8ffe710_1 conda-forge m2w64-gcc-libgfortran 5.3.0 6 conda-forge m2w64-gcc-libs 5.3.0 7 conda-forge m2w64-gcc-libs-core 5.3.0 7 conda-forge m2w64-gmp 6.1.0 2 conda-forge m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge mako 1.1.4 pyhd3eb1b0_0 mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markdown 3.3.4 pypi_0 pypi markupsafe 2.0.1 py37hcc03f2d_0 conda-forge matplotlib-base 3.4.3 py37h4a79c79_1 conda-forge matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge mistune 0.8.4 py37hcc03f2d_1004 conda-forge mkl 2021.3.0 hb70f87d_564 conda-forge mkl-service 2.4.0 py37h2bbff1b_0 mlxtend 0.19.0 pypi_0 pypi morecantile 3.1.2 pypi_0 pypi msgpack-python 1.0.2 py37h8c56517_1 conda-forge msys2-conda-epoch 20160418 1 conda-forge multipledispatch 0.6.0 pypi_0 pypi munch 2.5.0 py_0 conda-forge nbclient 0.5.4 pyhd8ed1ab_0 conda-forge nbconvert 6.2.0 py37h03978a9_0 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge netcdf4 1.5.7 nompi_py37h8860187_103 conda-forge networkx 2.3 py_0 conda-forge notebook 6.4.4 pyha770c72_0 conda-forge numba 0.55.1 py37hf11a4ad_0 numcodecs 0.8.0 py37hd77b12b_0 numexpr 2.7.3 py37hb80d3ca_1 numpy 1.21.2 py37h940b05c_0 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge opencv-python 4.5.3.56 pypi_0 pypi openjpeg 2.4.0 hb211442_1 conda-forge openpyxl 3.0.9 pyhd3eb1b0_0 openssl 1.1.1o h2bbff1b_0 optuna 2.10.0 pyhd8ed1ab_0 conda-forge packaging 21.0 pyhd8ed1ab_0 conda-forge pacmap 0.6.3 pypi_0 pypi pandas 1.3.5 py37h6214cd6_0 pandoc 2.14.2 h8ffe710_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge panel 0.12.4 pypi_0 pypi param 1.12.0 pyhd3eb1b0_0 parquet-cpp 1.5.1 h34088ae_4 parso 0.8.2 pyhd8ed1ab_0 conda-forge partd 1.2.0 pyhd8ed1ab_0 conda-forge pathlib 1.0.1 py37h03978a9_4 conda-forge patsy 0.5.2 pyhd8ed1ab_0 conda-forge pbr 5.6.0 pyhd3eb1b0_0 pcre 8.45 h0e60522_0 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.3.2 py37hd7d9ad0_0 conda-forge pip 21.3 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h8ffe710_0 conda-forge plotly 5.3.1 py_0 plotly pluggy 1.0.0 py37haa95532_1 poppler 21.09.0 h24fffdf_3 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 13.3 h1c22c4f_1 conda-forge prettytable 2.4.0 pyhd8ed1ab_0 conda-forge proj 8.1.1 h1cfcee9_2 conda-forge prometheus_client 0.11.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.20 pyha770c72_0 conda-forge prompt_toolkit 3.0.20 hd8ed1ab_0 conda-forge psutil 5.8.0 py37hcc03f2d_1 conda-forge psycopg2 2.9.1 py37hd8e9650_0 conda-forge py 1.11.0 pyhd3eb1b0_0 py-xgboost 1.5.0 py37h03978a9_0 conda-forge pyarrow 6.0.0 py37ha6b0ef9_3_cpu conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pyct 0.4.8 pypi_0 pypi pyct-core 0.4.6 py_0 conda-forge pydantic 1.9.1 pypi_0 pypi pygments 2.10.0 pyhd8ed1ab_0 conda-forge pynndescent 0.5.7 pyh6c4a22f_0 conda-forge pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyperclip 1.8.2 pyhd8ed1ab_2 conda-forge pyproj 3.2.1 py37hd4387d0_2 conda-forge pyqt 5.12.3 py37h03978a9_7 conda-forge pyqt-impl 5.12.3 py37hf2a7229_7 conda-forge pyqt5-sip 4.19.18 py37hf2a7229_7 conda-forge pyqtchart 5.12 py37hf2a7229_7 conda-forge pyqtwebengine 5.12.1 py37hf2a7229_7 conda-forge pyreadline 2.1 py37_1 pyrsistent 0.17.3 py37hcc03f2d_2 conda-forge pysocks 1.7.1 py37h03978a9_3 conda-forge pytest 7.1.1 py37haa95532_0 python 3.7.10 h7840368_103_cpython conda-forge python-constraint 1.4.0 pypi_0 pypi python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-snappy 0.6.0 py37hd77b12b_3 python_abi 3.7 2_cp37m conda-forge pytz 2021.3 pyhd8ed1ab_0 conda-forge pyviz-comms 2.1.0 pypi_0 pypi pywavelets 1.3.0 pypi_0 pypi pywin32 301 py37hcc03f2d_0 conda-forge pywinpty 1.1.4 py37h7f67f24_0 conda-forge pyyaml 5.4.1 py37hcc03f2d_1 conda-forge pyzmq 22.3.0 py37hcce574b_0 conda-forge qt 5.12.9 h5909a2a_4 conda-forge qtconsole 5.1.1 pyhd8ed1ab_0 conda-forge qtpy 1.11.2 pyhd8ed1ab_0 conda-forge rasterio 1.2.9 py37ha212cda_2 conda-forge re2 2021.11.01 h0e60522_0 conda-forge requests 2.26.0 pyhd8ed1ab_0 conda-forge retrying 1.3.3 py37_2 rio-cogeo 3.2.0 pypi_0 pypi rioxarray 0.7.1 pyhd8ed1ab_0 conda-forge rtree 0.9.7 py37h13cc57e_2 conda-forge s3transfer 0.5.0 pyhd8ed1ab_0 conda-forge scikit-image 0.18.3 pypi_0 pypi scikit-learn 1.0 py37ha78be43_1 conda-forge scipy 1.7.1 py37hb6553fb_0 conda-forge seaborn 0.11.2 hd8ed1ab_0 conda-forge seaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 61.2.0 py37haa95532_0 shap 0.40.0 py37h9386db6_0 conda-forge shapely 1.7.1 py37hc520ffa_5 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge slicer 0.0.7 pyhd3eb1b0_0 snappy 1.1.9 h6c2663c_0 snuggs 1.4.7 py_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.3.1 pyhd3eb1b0_0 spatialpandas 0.4.3 pyhd8ed1ab_0 conda-forge sqlalchemy 1.4.25 py37hcc03f2d_0 conda-forge sqlite 3.36.0 h8ffe710_2 conda-forge statsmodels 0.13.2 py37hec80d1f_0 conda-forge stevedore 3.5.0 py37h03978a9_1 conda-forge susi 1.2.2 pyhd8ed1ab_0 conda-forge tbb 2021.3.0 h2d74725_0 conda-forge tblib 1.7.0 pyhd8ed1ab_0 conda-forge tenacity 8.0.1 pyhd8ed1ab_0 conda-forge terminado 0.12.1 py37h03978a9_0 conda-forge testpath 0.5.0 pyhd8ed1ab_0 conda-forge threadpoolctl 3.0.0 pyh8a188c0_0 conda-forge tifffile 2021.11.2 pypi_0 pypi tiledb 2.3.4 h78dabda_0 conda-forge tk 8.6.11 h8ffe710_1 conda-forge tomli 1.2.2 pyhd3eb1b0_0 toolz 0.11.1 py_0 conda-forge tornado 6.1 py37hcc03f2d_1 conda-forge tqdm 4.64.0 py37haa95532_0 traitlets 5.1.0 pyhd8ed1ab_0 conda-forge typing_extensions 3.10.0.2 pyha770c72_0 conda-forge ucrt 10.0.20348.0 h57928b3_0 conda-forge umap-learn 0.5.1 py37h03978a9_1 conda-forge urllib3 1.26.7 pyhd8ed1ab_0 conda-forge vc 14.2 hb210afc_5 conda-forge vs2015_runtime 14.29.30037 h902a5da_5 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.37.0 pyhd8ed1ab_1 conda-forge widgetsnbextension 3.5.1 py37h03978a9_4 conda-forge win_inet_pton 1.1.0 py37h03978a9_2 conda-forge wincertstore 0.2 py37haa95532_2 winpty 0.4.3 4 conda-forge xarray 0.19.0 pyhd8ed1ab_1 conda-forge xarray-spatial 0.2.7 pyhd8ed1ab_0 conda-forge xerces-c 3.2.3 h0e60522_2 conda-forge xgboost 1.5.0 py37h03978a9_0 conda-forge xlrd 2.0.1 pyhd3eb1b0_0 xyzservices 2021.9.1 pyhd8ed1ab_0 conda-forge xz 5.2.5 h62dcd97_1 conda-forge yaml 0.2.5 he774522_0 conda-forge zarr 2.8.1 pyhd3eb1b0_0 zeromq 4.3.4 h0e60522_1 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.6.0 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h8ffe710_1013 conda-forge zstd 1.5.0 h6255e5f_0 conda-forge ```


Details about conda and system ( conda info ):

``` $ conda info active environment : geods37 active env location : C:\Users\Utilisateur\.conda\envs\geods37 shell level : 2 user config file : C:\Users\Utilisateur\.condarc populated config files : C:\Users\Utilisateur\.condarc conda version : 4.10.0 conda-build version : 3.20.5 python version : 3.8.5.final.0 virtual packages : __cuda=11.5=0 __win=0=0 __archspec=1=x86_64 base environment : C:\ProgramData\Anaconda3 (read only) conda av data dir : C:\ProgramData\Anaconda3\etc\conda conda av metadata url : https://repo.anaconda.com/pkgs/main channel URLs : https://repo.anaconda.com/pkgs/main/win-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/win-64 https://repo.anaconda.com/pkgs/r/noarch https://repo.anaconda.com/pkgs/msys2/win-64 https://repo.anaconda.com/pkgs/msys2/noarch package cache : C:\ProgramData\Anaconda3\pkgs C:\Users\Utilisateur\.conda\pkgs C:\Users\Utilisateur\AppData\Local\conda\conda\pkgs envs directories : C:\Users\Utilisateur\.conda\envs C:\ProgramData\Anaconda3\envs C:\Users\Utilisateur\AppData\Local\conda\conda\envs platform : win-64 user-agent : conda/4.10.0 requests/2.24.0 CPython/3.8.5 Windows/10 Windows/10.0.19041 administrator : False netrc file : None offline mode : False ```

Thanks a lot, P-H

snowman2 commented 2 years ago

I looked at your data, and it appears that your points don't properly represent the grid.

If I do:

numpy.unique(xr.y.diff(dim="y"))

I get:

array([ 10., 710.])

To solve your problem, I believe you need to rasterize your points to a grid. geocube can help: https://corteva.github.io/geocube/stable/getting_started.html

For example:

gdf = geopandas.GeoDataFrame(
    df[columns],
    geometry=geopandas.points_from_xy(
        df.x, df.y, crs="epsg:26911"
    )
)
xr = make_geocube(
    gdf,
    resolution=10,
    fill=-9999,
)

for col in columns:
    print("Column : " + col)
    output_path = os.path.join(output_directory, col + ".tif")
    xr[col].rio.to_raster(raster_path=output_path, dtype="int16")
phlamirande commented 2 years ago

Thanks @snowman2,

My group and I usually work with contiguous blocks of data, but it happens that sometimes we have to deal with non-contiguous data such as what I am showing here.

When applying the make_geocube() command, the output geotiffs are correct. So I understand that rioxarray requires that the x-y points are evenly spaced ? Is that right?

However, it seems that the execution time increases by lot (from 2-3s to 90s). Is there a workaround solution that would take less computation time ?

snowman2 commented 2 years ago

So I understand that rioxarray requires that the x-y points are evenly spaced ? Is that right?

That is correct.

Is there a workaround solution that would take less computation time ?

You would need to add in the missing rows filled in with nodata.

snowman2 commented 2 years ago

I am closing as I believe this has been resolved. If you have further questions, feel free to ask them.