search

Quasars / orange-spectroscopy

Other
51 stars 58 forks source link

[FIX] Lazy visual images for less memory use #725

Closed markotoplak closed 1 month ago

markotoplak commented 1 month ago

@clsandt reported running out of memory when performing PCA on Opus data with large visual images.

That was caused by .attributes dictionary on a table that contained actual visual images, and that dict was deepcopied for each table transformation. Thus, memory got filled up quickly.

This PR does not save image contents into that dict. VisualImage now accesses file contents only when someone (=HyperSpectra) want's to display the images.

markotoplak commented 1 month ago

The disadvantage of this PR is that after saving as the .tab file, .tab.metadata will not contain images anymore.

The alternative fix would be to read images into the memory, buy reimplement __deepcopy__ on the file so that it links but does not copy the images. The disadvantage of this solution is that images are still kept in memory, albeit just once. I'll probably go for this one.