Running initialization on the same recording and on the same computer takes much longer now than it used to take before. Maybe it is loading every file before getting a single plane from it, and loading the next file to get the frames from that same plane? And then repeating that for every plane?
This is from the current main version:
[2023-10-10 23:25:58][00] Loading job directory for Dali2 in /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/
[2023-10-10 23:25:58][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/registered_data
[2023-10-10 23:25:58][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/summary
[2023-10-10 23:25:58][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/iters
[2023-10-10 23:25:58][01] Loading default params
[2023-10-10 23:25:58][02] Updating param fs
[2023-10-10 23:25:58][02] Updating param planes
[2023-10-10 23:25:58][02] Updating param tau
[2023-10-10 23:25:58][01] Updated main params file
[2023-10-10 23:26:05][01] Saved a copy of params at /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/summary
[2023-10-10 23:26:05][01] Updated main params file
[2023-10-10 23:26:05][00] Launching initial pass
[2023-10-10 23:26:05][00] Saving summary to /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/summary/summary.npy
[2023-10-10 23:27:33][01] Loaded movie with 800 frames and shape 14, 868, 876
[2023-10-10 23:27:39][03] Enforcing positivity in mean image
[2023-10-10 23:27:44][01] No crosstalk estimation or subtraction
[2023-10-10 23:27:44][01] Estimating fusing shifts
[2023-10-10 23:27:47][01] Using best fuse shift of 0
[2023-10-10 23:27:47][01] Building ops file
[2023-10-10 23:28:56][01] Saved init mov to /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali2/summary/init_mov.npy
[2023-10-10 23:28:56][01] Aligning planes
[2023-10-10 23:29:03][02] Fixing plane alignment outliers
[2023-10-10 23:29:03][01] Fusing and padding init mov
[2023-10-10 23:29:25][01] Computing reference images
[2023-10-10 23:35:31][02] Computed reference for plane 0
[2023-10-10 23:41:37][02] Computed reference for plane 1
[2023-10-10 23:47:43][02] Computed reference for plane 2
[2023-10-10 23:53:49][02] Computed reference for plane 3
[2023-10-10 23:59:55][02] Computed reference for plane 4
[2023-10-11 00:06:03][02] Computed reference for plane 5
compared to a previous run from an older version:
[2023-07-28 02:07:39][00] Loading job directory for Dali in /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/
[2023-07-28 02:07:39][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/registered_data
[2023-07-28 02:07:39][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/summary
[2023-07-28 02:07:39][02] Created dir /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/iters
[2023-07-28 02:07:39][01] Loading default params
[2023-07-28 02:07:39][02] Updating param fs
[2023-07-28 02:07:39][02] Updating param planes
[2023-07-28 02:07:39][02] Updating param tau
[2023-07-28 02:07:39][01] Updated main params file
[2023-07-28 02:07:50][01] Saved a copy of params at /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/summary
[2023-07-28 02:07:50][01] Updated main params file
[2023-07-28 02:07:50][00] Launching initial pass
[2023-07-28 02:07:50][00] Saving summary to /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/summary/summary.npy
[2023-07-28 02:09:11][01] Loaded movie with 800 frames and shape 14, 868, 876
[2023-07-28 02:09:17][03] Enforcing positivity in mean image
[2023-07-28 02:09:24][01] No crosstalk estimation or subtraction
[2023-07-28 02:09:24][01] Building ops file
[2023-07-28 02:10:35][01] Saved init mov to /home/freiwald/Data/analysis_2pRAM/Dali/20230620d/154951tUTC_Max15_depth400um_fov2628x2600um_res3p00x3p00umpx_fr04p482Hz_pow299p9mW/s3d-Dali/summary/init_mov.npy
[2023-07-28 02:10:35][01] Aligning planes and calculating reference images
[2023-07-28 02:10:35][01] Computing reference images
[2023-07-28 02:12:07][02] Computed reference for plane 0
[2023-07-28 02:13:26][02] Computed reference for plane 1
[2023-07-28 02:14:45][02] Computed reference for plane 2
[2023-07-28 02:16:05][02] Computed reference for plane 3
[2023-07-28 02:17:24][02] Computed reference for plane 4
[2023-07-28 02:18:44][02] Computed reference for plane 5
[2023-07-28 02:20:00][02] Computed reference for plane 6
[2023-07-28 02:21:17][02] Computed reference for plane 7
[2023-07-28 02:22:33][02] Computed reference for plane 8
[2023-07-28 02:23:48][02] Computed reference for plane 9
[2023-07-28 02:25:04][02] Computed reference for plane 10
[2023-07-28 02:26:19][02] Computed reference for plane 11
[2023-07-28 02:27:35][02] Computed reference for plane 12
[2023-07-28 02:28:51][02] Computed reference for plane 13
alihaydaroglu
commented
4 months ago
Running initialization on the same recording and on the same computer takes much longer now than it used to take before. Maybe it is loading every file before getting a single plane from it, and loading the next file to get the frames from that same plane? And then repeating that for every plane?
This is from the current main version:
compared to a previous run from an older version: