the-lay / tiler

N-dimensional NumPy array tiling and merging with overlapping, padding and tapering
https://the-lay.github.io/tiler/
MIT License
65 stars 10 forks source link

Add option to load data from file into tile #2

Closed tdmorello closed 3 years ago

tdmorello commented 3 years ago

Thank you for creating this package! I was looking for something just like this.

I work with very large image files that are usually not fully loaded into memory. This modification adds the option of giving the get_tile method a callable, which can be any image reader function that takes the tile corner coordinates and width and height. (I gave two examples in the docstring)

I hope you find this useful!

the-lay commented 3 years ago

Thank you for the pull request, that's a great idea! For completeness, I think iterate() should also accept Callable. This should be relatively easy to add, since underneath it also uses get_tile.

reader_func = lambda X, Y, W, H: reader.read(XYWH=[X, Y, W, H])
for tile_id, tile in tiler(reader_func):
    print(f'Tile {tile_id} out of {len(tiler)} tiles.')
for tile_id, tile in tiler.iterate(reader_func):
    print(f'Tile {tile_id} out of {len(tiler)} tiles.')

Would you want to continue with that? Otherwise I can, but later on the weekend.

P.S. The build failed not because you did something wrong. I've fixed it, should not happen anymore :)

tdmorello commented 3 years ago

Excellent! The typing definitions for __call__ and iterate have been updated.

the-lay commented 3 years ago

Sorry for the delay with merging this. On a second thought, your proposed update would work only for 2D data. I've taken the liberty to make it support variable length callables, making it n-dimensional again. I've updated the docs and added unit tests. The only thing I'm not happy about is typing, it seems typing does not yet fully support variable length Callable. I've put ellipsis there but added a note in docs that the input must be integers.

Let me know if you see any problems with this!

tdmorello commented 3 years ago

Looks great to me. I changed the docstring example to work with the update. There, the function is declared differently between bioformats and openslide solely to demonstrate two ways of doing the same thing. The example is lengthier now, but feel free to modify it.

the-lay commented 3 years ago

That's even better. Thank you for your work!

tdmorello commented 3 years ago

Thank you! I'm so glad I found this package -- it has saved me an immense amount of time.