georust / gdal

Rust bindings for GDAL
https://crates.io/crates/gdal
MIT License
369 stars 94 forks source link

Incomplete block pixel iterator #570

Open lnicola opened 1 month ago

lnicola commented 1 month ago

This bit me in #556 and probably before: it you read a block at the right or bottom edges of a band, it might be incomplete. In #556 I assumed we'd get a block of smaller dimensions, but GDAL does something different: you get a full block, but only the top-left data is valid (and the extra borders are ignored when writing).

This is hinted at in the GDAL docs:

Note that the final block may be partial ... data beyond the edge of the underlying raster band in these edge blocks is of an undetermined value.

We could offer a pixels (cells. etc.) iterator that only returns the in-bounds values, or we could go full-in on ndarray (see also #569).

jdroenner commented 1 month ago

I think we should avoid implementing our own Grid type with iterators and so on. It is to complex and beyond the scope of the crate (my opinion). The Buffer struct was intended to be a return type for reading where the data (Vec) and some metadata is stored. After reading, the user should convert it to ndarray or use a custom struct (as we do in Geo Engine).

I know that writing is more complex. If the data is not in C order it needs to be transformed because GDAL uses C order and i think there is no way around it. We could think of a trait for writing that has a &[T] and provides x and y size instead of using the Buffer struct to avoid a copy if you have data already in the perfect shape. I never used GDALRasterIOEx with spacing so there may be more options if the data does not have the correct shape.

lnicola commented 1 month ago

It is to complex and beyond the scope of the crate (my opinion).

Just this API, I swear, just this API and we're done :laughing:. But seriously, we should at least store the actual block size in the buffer so it's easy to access.

I never used GDALRasterIOEx with spacing so there may be more options if the data does not have the correct shape.

I think it supports setting cell (e.g. for pixels with padding) and line strides, plus an initial offset by pointing it to the middle of the buffer.