search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

Speed up K2/K3 acquisition by faking returned sum array #8161

Closed leginonbot closed 7 months ago

leginonbot commented 7 months ago

Author Name: Anchi Cheng (@anchi2c) Original Redmine Issue: 8161, https://emg.nysbc.org/redmine/issues/8161 Original Date: 2019-10-14 Original Assignee: Anchi Cheng

The very large float32 array takes too long to transfer. This becomes more critical as K3 process and save in 10-15 seconds while transferring its super-res image takes 8+ seconds on the typical 1G network to leginon workstation.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2019-10-15T15:49:49Z

To use this, set "Save 8x8 stats image to speed up" when edit the preset. It does this only if saving frame is also used.

Tested it on K2 super-res. 10 second exposure and 200 ms frame time. The time for full format acquisition, including acquiring, publishing (save sum image and metadata), and display is 23 seconds. 8x8 format gives 20 seconds.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2019-11-13T00:51:36Z

Bill tested on K3 super-res. Did not improve the speed much.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2020-07-16T20:05:52Z

Recent test in combination of changes in skipping normalization in Leginon corrector client gives 6 seconds improvement on K3 super-res movies.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2020-07-16T20:46:53Z

Outstanding!! Well done.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2020-08-14T21:00:27Z

This feature is now available for non-early return as well and gui is added in camera settings. dmsem.cfg EARLY_RETURN_FORMAT is now ignored.

leginonbot commented 7 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2020-10-30T02:09:47Z

Usage:

Activate "Save 8x8 stats image to speed up" in Edit Preset of PresetsManager.