developmentseed / titiler-image

TiTiler extension to work with non-geo images
MIT License
18 stars 2 forks source link

Reader caching resulting in segmantation fault from reader #13

Closed nadavFux closed 7 months ago

nadavFux commented 7 months ago

Hi there! I have being working on a titiler-image fork where for performance reasons I have opted to replace the "with ImageReader(src_path) as dst" in "titiler-image/titiler/image /factory.py" as a function with a basic @functools.lru_cache annotation therefore forfeiting the "with" part. When stress testing the version I found even minor stress caused after a few seconds a random "segmantation fault". I know this matter doesnt concern the existing "titiler-image" it more about the rio-tile.io module or other underlying packages but I would really appreciate some advice on the matter:

Also - is there a plan to add https://github.com/developmentseed/titiler-image/issues/12 ?

I would greatly appreciate any help! Please and thank you

nadavFux commented 7 months ago

Also, and this may not be related, is there a different inside way to cache file handles(or to expand caching?) I have gone through https://developmentseed.org/titiler/advanced/performance_tuning/ and am looking to cache read operations to my s3/reading neaby tiles

I've seen this https://gdal.org/user/virtual_file_systems.html#vsicached-file-caching and am wondering if there could be of use

I'm kinda new to all of this so any help would be much appreciate @vincentsarago

please and thank you!