intergarte tested cell_membrane_polarisation_detector()

[jjb-hub]

optimize spike removal and test function (if inducing error make sure is consistent - i.e. +/- 0.5mV from non-spiking raw trace)

[jjb-hub]

@debapratimj this would now be the highest priority:

to have it running for APP files:

1. list of input R for PRE, APP and WASH e.g. 'inputR_PRE' 'inputR_APP' ect
2. list of RMP (V without ) for PRE, APP and WASH e.g. 'rmp_APP' 'rmp_WASH' ect

i would imagine a function that I will add to the expandDF() that will add these colums as it builds,

let me know an ETA - otherwise, I can also build

[debapratimj]

Started work

Current To dos:

1. Use steady state to calculate V_membrane during PRE APP and WASH,
2. Fix current error

ETA: 2 days so deliverable: 4.1.23

[debapratimj]

Debugging this error where the voltage- trace doesnt show up properly and leads to a Boolean error.

Time: ~12.1.23

IndexError Traceback (most recent call last) /Users/owner/Desktop/IGOR_phd/debug.ipynb Cell 2 line 1 10 cell_types = np.unique(raw_df_application['cell_id'][:4]) 14 folder_file, cell_id, data_type, drug, conc_uM, replication_no, application_order, drug_in, drug_out, I_set, R_series, R_tip, cell_type, cell_subtype, sex, offset, weight, P_age, Bistable, pAD = RawDFRow_unpacker(raw_df_application , 0) ---> 16 cell_membrane_polarisation_detector(cell_ID= cell_id, folder_file= folder_file, I_set= I_set , drug=drug , drug_in= drug_in , drug_out= drug_out, application_order=application_order)

File ~/Desktop/IGOR_phd/module/action_potential_functions.py:1453, in cell_membrane_polarisation_detector(cell_ID, folder_file, I_set, drug, drug_in, drug_out, I_setting, application_order, pAD_locs) 1449 v_array_drug = np.array(v_array_drug) 1450 v_drug_diff = np.diff(v_array_drug) -> 1453 v_drug_filtered = v_array_drug[np.concatenate( ( v_drug_diff - np.mean(v_drug_diff) <= 2*np.std(v_drug_diff) , [True] ) )] 1456 v_drug_cleaned, v_drug_peaks = array_peak_cleaner(v_array_drug) 1458 fig, axs = plt.subplots(1,1)

IndexError: boolean index did not match indexed array along dimension 0; dimension is 0 but corresponding boolean dimension is 1

[debapratimj]

IndexError solved, other main issue was in array_cleaner() also solved, need now one session to put things together, ETA 1 day max

[debapratimj]

recent push on issue-20-duplicate has all the necessary function that has been tested on the subset feature_df. Functions added are :

1. mean_inputR_APP_calculator(V_df, I_df, drug_in, drug_out) : Returns : input_R_PRE, input_R_APP, input_R_WASH which are individually lists (dtype is list) of input resistance for each APP condition

2. mean_RMP_APP_calculator(V_df, I_df, drug_in, drug_out) : returns : mean_RMP_PRE, mean_RMP_APP, mean_RMP_WASH which are again lists for mean rest. pot. for each APP condition in the absence of injected current

3. APP_splitter(v_df, drug_in, drug_out) returns list_PRE, list_APP, list_WASH which are each lists of voltage values in each condition

[jjb-hub]

hello

1. which subset feature df - it would be for a single cell_type ?
2. you did not add a spike remover function? or it was already there ? (better to include all added functions so i can also use the generic/modular ones)
3. Please tell me when i should go onto the branch 'issue-20-duplicate' and check it ?
4. delete any unnecessary branches

[debapratimj]

1. subset is the entire AP file type
2. I did add: spike_remover(array) returns array_cleaaned
3. as functions tested on APP dataset, you can check now
4. will delete other branches by 19.1.24

[jjb-hub]

Generally the docs need to be more specific.

APP_splitter:

1. is not called in mean_inputR_APP_calculator or mean_RMP_APP_calculator - which was why i suggested it be written
2. need to include in docs (''' ''') whether it takes the drug_in and drug_out sweep as part of the PRE or APP or WASH and be consistent with use elsewhere

mean_inputR_APP_calculator:

1. this should return a list for each condition the len of sweeps in condition of the mean RMP of each sweep I do not understand teh docs here 'Calculates RMP or the Resting Membrane Potential based on the Vmean{condition} and I_injected is zero in this case'

mean_inputR_APP_calculator:

1. I do not understand what Vmean{condition} is ? the mean of a sweep ? of all sweeps? I assume a conditon is PRE APP or WASH?
2. 'Returns : input_R_PRE, input_R_APP, input_R_WASH : each a list of input resistances for the condition.' is this one per sweep in which case it is not a mean is that correct?

could you please explain the use case for this: replace_nan_with_mean()

[debapratimj]

APP_splitter ():

1. It will be called in the next push.
2. Docs also will be updated.

mean_inputR_APP_Calculator :

The docs imply simply that we calculate the RMP per sweep per condtion. So for PRE for instance we put in a np.array of shape say L x S and we get out means of RMP per sweep --> S datapoints in a list. Let me know if this is clear, can update doc description but wanted to be as detailed as possible.

mean_RMP_APP_Calculator :

1. Mean of each sweep is always returned. condition is PRE/APP/WASH: that is Vmean{CONDITION} is either V_mean_PRE / V_mean_APP or V_mean_APP as the convention suggests

2. It is a mean per sweep. Meaning lets consider the voltage array for PRE. Suppose it is of shape L x S : length is L and num of sweeps S. Then the list that is returned for PRE condition is a list with S floats one per sweep. I do not understand how it is not a mean.

Use case: replace_nan_with_mean():

Often either when the neuron spikes or there might be anomalies in the data, the value crosses significantly over the mean by more than say 2 stds. Then we replace those values with nan. Now if we delete or drop nans, then then lengths of different sweeps / recordings would be different. Instead the nans are replaced with the mean to preserve the lengths

From whatsapp chat:

Question: Why use V_df when in code we immediately convert it to a np,array? Why dont we instead use V_array?

A: Firstly, the variable name legacy V_array, V_df is misleading. I stuck with it as I was keeping tracking of the types and writing code according but V_array really is a List, V_df is correct however, it really is a df. We do not use V_array is it is a 1d array or list and we don't apriori just from the length of this what should be the appropriate shape of the 2d array which is length of each rec x num of sweeps. It can be fed in and reshaped as the len of each recordings are usually the same, but it uses one reshape and could be a point of wrong shaping in case the length of the recording was actually different. V_df seems to be the obvious choice, and i would propose perhaps to return a np.array from igor_exporter() than a df.

ETA for proposed changes: 19-20th Jan

[jjb-hub]

V_array is not a list and has a shape the same as df - inputs have been changed to V_array (once all functions handle arrays yes we can move the iror exporter to only output V_array) this is the V_array available as input (see getters.py line 194) this unpacks everything and will be cleaned once we know what we need (as stated above)

[jjb-hub]

mean_inputR_APP_Calculator: If i understand correctly this takes the mean of both V and I for a sweep and divides it - this is not how you calculate input R, R = I injected /delta V

[jjb-hub]

to resolve this I will need to make steady_state_value() more generic can you please confirm its inputs and outputs why is the steady state V called asym_current? what does it mean ? I need this function to fix mean_inputR_APP_Calculator

[jjb-hub]

please do not work on branch issue-20-duplicate ! @debapratimj

I have rewritten mean_inputR_APP_Calculator (involved modularising steady_state, spike_remover and APP_splitter) I will test it tonight and if it runs will push so you can look at what needed to be done/ if doesn't run will continue tomorrow

If it runs I will then finish mean_RMP_APP_Calculator tomorrow.

[debapratimj]

V_array is not a list and has a shape the same as df - inputs have been changed to V_array (once all functions handle arrays yes we can move the iror exporter to only output V_array) this is the V_array available as input (see getters.py line 194) this unpacks everything and will be cleaned once we know what we need (as stated above)

Look here:

What I was referring to is what is returned from igor_exporter(): It returns a V_array and a V_df. In the image you sent, you actually use what I said, ie. taking V_df and converting it to np. array. So we actually are talking about the same thing

[jjb-hub]

the screenshot you posted is not the code that is being used (i do not know from where it comes) - getters.py is where functions are applied (see screenshot) - until other things are built it will stay like this

[debapratimj]

Adding new function for spike_remover that cleans spikes in case of regular spiking :

CODE:

def spike_remover_mod(array):

'''
Cleans a voltage array by removing spikes
Input: array: voltage  np.array : shape : length x num_sweeps 
(Note that when using igor exporter, this is actually df_V and NOT V_array although V_array when reshaped quickly gives the same result)
Returns: array_cleaned: voltage np.array  with peaks removed, shape : length x num_sweeps  
'''

array_cleaned  = array.copy() 

if array.shape[0] <= 1 :
    print('Voltage array has 1 or fewer points')
    return array_cleaned
else:
    pass

array_diff_abs = np.abs(np.diff(array, axis = 0))
array_diff_abs = np.vstack([array_diff_abs, np.mean(array_diff_abs, axis  = 0 ).reshape(1,-1) ])

array_cleaned[array > np.mean(array , axis = 0 ) + 2*np.std(array, axis =  0 )] = np.nan

array_cleaned = replace_nan_with_mean(array_cleaned)

return array_cleaned  

Code Usage:

folder_name = 'JJB210427/t8' #'JJB210406/t11'

path_V, path_I = make_path(folder_name) V_array, V_df = igor_exporter(path_V)
V_array = np.array(V_df)

V_array_cleaned = spike_remover_mod(V_array) V_array_cleaned = spike_remover_mod(V_array_cleaned) # can use twice for slightly better performance

USE CASES : tested on : folder_file: 'JJB210406/t11' Example Traces:

USE CASES : tested on : folder_file: 'JJB210427/t8' Example Traces:

1. Fewer Spikes:
2. Regular/Tonic Spiking:

[debapratimj]

As we don't need to use np.diff() in spike_remover, those lines can be deleted:

def spike_remover_mod(array):

'''
Cleans a voltage array by removing spikes
Input: array: voltage  np.array : shape : length x num_sweeps 
(Note that when using igor exporter, this is actually df_V and NOT V_array although V_array when reshaped quickly gives the same result)
Returns: array_cleaned: voltage np.array  with peaks removed, shape : length x num_sweeps  
'''

array_cleaned  = array.copy() 

if array.shape[0] <= 1 :
    print('Voltage array has 1 or fewer points')
    return array_cleaned
else:
    pass

array_cleaned[array > np.mean(array , axis = 0 ) + 2*np.std(array, axis =  0 )] = np.nan

array_cleaned = replace_nan_with_mean(array_cleaned)

return array_cleaned  

[debapratimj]

issue-20-cell-membrane-polarisation-detector was checked and is a redundant branch and was deleted. if issue-20-duplicate solves the open issue then pull request to merge it was jasmine branch and then close issue + delete issues-20-duplicate ?

[jjb-hub]

rewritten