search

girder / large_image

Python modules to work with large multiresolution images.
http://girder.github.io/large_image/
Apache License 2.0
195 stars 43 forks source link

Handle massive json annotations #1073

Open manthey opened 1 year ago

manthey commented 1 year ago

Currently, we parse annotations by loading the entire annotation into memory. Usinb an iterative approach for larger files (e.g., via https://github.com/ICRAR/ijson) would be mord scalable.

manthey commented 1 year ago

A quick but limited solution to this would be to pick a bigger size than Girder's default of 16MB and read those using the current technique.