Automated analysis

[AlexanderNiedrig]

Initial analysis and determination of the dynamic localization error, as well as the calculation of D_min are automated for multiple coverslips in Dmin_autocalculator. Consecutive analysis and saving of filtered statistics is automated in trackAnalysis_automizer.py. .py files with classes of the TrackAnalysis and trackStatistics notebook were added as well as a version of the trajectory_statistics file that doesn't output its graphs.

[JohannaRahm]

When executing the file, I ran into the following error (this is why the current comments mainly focus on obvious required changes and style):

Error config

#works for both, determinD and batchAnalysis. To determin D_min min_detectable_D should be 0, for batch analysis replace it with the D_min you get out of determinD
[SOFTWARE]
# choose software, either ThunderSTORM or rapidSTORM
software = ThunderSTORM

[INPUT_DIRS]
# path to the coverslips
dir1 = D:\Chemie_phd\SPTAnalyser\code_reviews\test_data\test_data_automated\CS2
dir2 = D:\Chemie_phd\SPTAnalyser\code_reviews\test_data\test_data_automated\CS5

[MASK_WORDS]
#words to filter in your directory, e.g. BACKGROUND
mask = background

[BACKGROUND]
#whether to use background correction. If true give directories.
background = false

[BACKGROUND_DIRS]
#directories to the background, make sure  they are in the same order as the INPUT_DIRS. Only necessary when background is set to true.
dir01 = 

[CAMERA]
# camera parameters: pixel size in nm, camera integration time in s,background size in pixel (256*256=65536)
pixel_size = 159
camera_integration_time = 0.02
background_size = 65536

[DIFFUSION_TYPE_PARAM]
#parameters for the analysis of the diffusion types. min_detectable_D in micrometer^2s^-1, min_track_length in frames
number_of_points = 4
MSD_fit_area = 0.6
degree_of_freedom = 4
min_detectable_D = 0
min_track_length = 20

[ID_TYPE]
#Whether to analyze by segments or whole tracks (see manual) valid options: seg id or track id
id_type = seg id

[FILTER_PARAMETERS]
#parameters to filter the statistics by
min_trajectory_len = 
max_trajectory_len = 
min_D =0
max_D =
filter_for_immobile = true
filter_for_confined = true
filter_for_free = true
filter_for_noType = true

[SAVE_DIR]
#where to create the trackAnalysis folder as well as save the filtered data
sav = D:\Chemie_phd\SPTAnalyser\code_reviews\test_data\test_data_automated

We require two separate scripts for automatically determining Dmin and trackAnalysis. Atm, the ini files “trackAnalysis_automizer_config.ini” and “Dmin_autocalculator_config.ini” have the same description which states that both processes could be operated within one script. Please split these processes! Furthermore, setting Dmin=0 would atm trigger Dmin calculation and a value of Dmin=0 (which might be interesting) would not be possible, which should be in trackAnalysis_automizer.

LOG file is expected to be named “cell_sizes.LOG” → just search for a log file in the rois directory

Please give variables and functions lower case names, add spaces between “#” and comment, use “”” instead of ‘’’ as docstring symbol, add description to function names (partly only the function variables are described, but not the function itself), and correct typos (they are all over the place).

[JohannaRahm]

Thanks for fixing the error message above (min_dD = 0 → min_dD = str(0))!

Here further points:

Dmin_calculator

• rename "DLP_per_cell.csv" to "D_min_per_cell.csv"
• add units to row names in "D_min.csv"
• rename column in "DLP_per_cell.csv" to sigma_dyn + unit
• remove trackAnalysis folder after calculation
• have a meaningful output of the command line (suppress warnings, add progress bar)

trackAnalysis_automizer

• rename to trackAnalysisStatistics_automizer.py
• only create background folder (./trackAnalysis/backgrounds) if background are defined
• have a meaningful output of the command line (suppress warnings, add progress bar)

[AlexanderNiedrig]

Thanks for the updates!

* A few typos / style corrections are marked in the code.

* Furthermore, the LOG file name was only made flexible for Dmin, please also add this to trackAnalysisStatistics.

* The progressbar in Dmin and trackAnalysisStatistics behaves not properly. It displays 20 cells, but does not count upwards and just depicts random trajectories underneath (see screenshot). It would be great if the progress bar simply shows the amount of cells, no need for trajectories.

Point 3 is because it uses existing SPTAnalyser code written for Jupyter Notebooks that is buried somewhere deep in the package. I'd have to dig that out and create new versions of it at which point I might as well give up using existing code.

[JohannaRahm]

The progress bar is called in cell.py and coverslip.py. You could copy these files to notebookspy and make changes to the few lines that are using tqdm. This is more of a workaround, like the copies of the notebooks in that folder, but okay due to your deadline.