search

nion-software / nionswift

Nion Swift is open source scientific image processing software integrating hardware control, data acquisition, visualization, processing, and analysis using Python. Nion Swift is easily extended using Python. It runs on Windows, Linux, and macOS.
http://nion.com/swift
GNU General Public License v3.0
45 stars 33 forks source link

Computations producing large data use a suitable large format storage #378

Open Brow71189 opened 4 years ago

Brow71189 commented 4 years ago

The issue is that the size of the data is not known at the time the data item is created. So there needs to be some mechanism of updating the storage format when the data size is first determined. This can be when the data is set or when space is reserved.

Also needs test cases.

https://github.com/nion-software/nionswift/blob/db37dfebc7146497c4b7ad2bb547bf28f0f9cb82/nion/swift/model/DocumentModel.py#L2395

cmeyer commented 4 years ago

Doesn't new_data_item do this already?

Brow71189 commented 4 years ago

new_data_item does that only when you pass in data with more than 2 dimensions. Computations create an empty data_item first and populate it later (see the link in my first comment). In this case, the created data_item is NOT of large_format. Since it is impossible to know what kind of data will be put into the data_item, we either have to have the option to specify this explicitly, or make "large_format" the default. Would the latter option have any major disadvantages?