search

rg314 / pytraction

Bayesian Traction Force Microscopy
BSD 3-Clause "New" or "Revised" License
3 stars 1 forks source link

MemoryError - Can't process 2048x2048 images #50

Open l-kaplan opened 3 years ago

l-kaplan commented 3 years ago

If I try to process my full uncropped 2048x2048 images I get a memory error saying I ran out of RAM. Cropped images do work though.

Automatically selected window size of [32.]

|-----> || The Open Source P article | | Open || I mage | | PIV || V elocimetry Toolbox | | <-----|| www.openpiv.net version 1.0 |

('Algorithm : ', 'WiDIM')

Parameters

(' ', 'Size of image', ' | ', [2048, 2048]) (' ', 'total number of iterations', ' | ', 1) (' ', 'overlap ratio', ' | ', 0.5) (' ', 'coarse factor', ' | ', 0) (' ', 'time step', ' | ', 1.0) (' ', 'validation method', ' | ', 'mean_velocity') (' ', 'number of validation iterations', ' | ', 3) (' ', 'subpixel_method', ' | ', 'gaussian') (' ', 'Nrow', ' | ', array([127], dtype=int32)) (' ', 'Ncol', ' | ', array([127], dtype=int32)) (' ', 'Window sizes', ' | ', array([32], dtype=int32))

| STARTING |

('ITERATION # ', 0) ..[DONE] (' --residual : ', 0.9999999812503433) Starting validation.. ('Validation, iteration number ', 0)

('Validation, iteration number ', 1)

('Validation, iteration number ', 2)

..[DONE]

////////////////////////////////////////////////////////////////// end of iterative process.. Re-arranging vector fields.. ...[DONE]

('[DONE] ..after ', 226.073166847229, 'seconds ')

MemoryError Traceback (most recent call last) /tmp/ipykernel_350/2562665077.py in 41 # p.map(tf,range(1,17)) 42 ---> 43 log1 = tf(0) 44 print("--- %s seconds ---" % (time.time() - start_time)) 45

/tmp/ipykernel_350/2562665077.py in tf(i) 27 img, ref, roi = traction_config.load_data(img_path, ref_path, roi_path) # we can load the dat using the load_data method 28 ---> 29 log1 = process_stack(img, ref, tractionconfig, roi = roi, verbose=0) 30 log1.save("out/pos"+ j + "_.h5") 31

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/pytraction/core.py in process_stack(img_stack, ref_stack, config, bead_channel, cell_channel, roi, frame, crop, verbose, custom_noise) 313 314 # compute traction map --> 315 traction_map, f_n_m, L_optimal = calculate_traction_map(pos, vec, beta, 316 config.config['tfm']['meshsize'], 317 config.config['tfm']['s'],

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/pytraction/process.py in calculate_traction_map(pos, vec, beta, meshsize, s, pix_per_mu, E) 48 49 # get lambda from baysian bad boi ---> 50 L, evidencep, evidence_one = optimal_lambda(beta, fuu, Ftux, Ftuy, 1, s, meshsize, i_max, j_max, X, 1) 51 52 # do the TFM with bays lambda

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/pytraction/optimal_lambda.py in optimal_lambda(beta, fuu, Ftux, Ftuy, E, s, cluster_size, i_max, j_max, X, sequence) 54 55 target = partial(minus_logevidence, beta=beta, C_a=C_a, BX_a=BX_a, X=X, fuu=fuu, constant=constant, Ftux=Ftux,Ftuy=Ftuy,E=E,s=s,cluster_size=cluster_size,i_max=i_max, j_max=j_max) ---> 56 alpha_opt = optimize.fminbound(target, alpha1, alpha2, disp=3) 57 58 evidence_one = -target(alpha_opt)

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/scipy/optimize/optimize.py in fminbound(func, x1, x2, args, xtol, maxfun, full_output, disp) 1907 'disp': disp} 1908 -> 1909 res = _minimize_scalar_bounded(func, (x1, x2), args, **options) 1910 if full_output: 1911 return res['x'], res['fun'], res['status'], res['nfev']

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/scipy/optimize/optimize.py in _minimize_scalar_bounded(func, bounds, args, xatol, maxiter, disp, *unknown_options) 1956 rat = e = 0.0 1957 x = xf -> 1958 fx = func(x, args) 1959 num = 1 1960 fmin_data = (1, xf, fx)

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/pytraction/optimal_lambda.py in minus_logevidence(alpha, beta, C_a, BX_a, X, fuu, constant, Ftux, Ftuy, E, s, cluster_size, i_max, j_max) 22 A = alphacsr_matrix(C_a) + BX_a 23 ---> 24 L = sparse_cholesky(csr_matrix(A)).toarray() 25 logdetA = 2np.sum(np.log(np.diag(L))) 26

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/scipy/sparse/compressed.py in toarray(self, order, out) 1029 if out is None and order is None: 1030 order = self._swap('cf')[0] -> 1031 out = self._process_toarray_args(order, out) 1032 if not (out.flags.c_contiguous or out.flags.f_contiguous): 1033 raise ValueError('Output array must be C or F contiguous')

~/miniconda3/envs/pytraction/lib/python3.8/site-packages/scipy/sparse/base.py in _process_toarray_args(self, order, out) 1200 return out 1201 else: -> 1202 return np.zeros(self.shape, dtype=self.dtype, order=order) 1203 1204

MemoryError: Unable to allocate 48.6 GiB for an array with shape (80800, 80800) and data type float64`

rg314 commented 3 years ago

@l-kaplan thank you for raising this, please could you send me an example image over? I think a larger window size would work in the config but then you would lose resolution. If you could send a screenshot of the image in question and also the raw image via email that would be great :)