nion-software / nionswift

Nion Swift is open source scientific image processing software integrating hardware control, data acquisition, visualization, processing, and analysis using Python. Nion Swift is easily extended using Python. It runs on Windows, Linux, and macOS.
http://nion.com/swift
GNU General Public License v3.0
43 stars 33 forks source link

Writing data files should occur in a thread #1066

Open cmeyer opened 2 weeks ago

cmeyer commented 2 weeks ago

In order to avoid hanging the user interface when writing large data, writing to disk should be performed in a thread.

In the code today, writing occurs via the Persistence.py write_external_data method, which then calls FileStorageSystem.py write_external_data which then calls FileStorageSystem.py update_data.

At one of these points, the write could put the write in a queue that is serviced by a thread.

The thread should handle data access while the write is pending.

The thread should also handle writing during shutdown (should probably complete the write).

Multiple threads could run simultaneously to write the data.

### Tasks
- [ ] Create thread to service writing requests
- [ ] Handle overlapping data and metadata writing requests
- [ ] Handle overlapping reading and writing requests
- [ ] Handle shutdown occurring during writing (finish writing)