NeuralAnalysis / PyalData

Repository for the Python implementation of the TrialData analysis library.
GNU General Public License v3.0
7 stars 9 forks source link

Deep copy in `copy_td` #127

Open bagibence opened 2 years ago

bagibence commented 2 years ago

I don't think df.copy() does what we want in the @copy_td decorator because it doesn't make an actual copy of array fields. We will probably have to make recursive deep copy with something like pd.DataFrame(data=copy.deepcopy(df.values), index=df.index, columns=df.columns). Unfortunately, it will be definitely slower and memory intensive, so we might have to think about those consequences later.

import copy
dfa = df.copy(deep=True)
dfb = deepcopy(df)
dfc = pd.DataFrame(data=copy.deepcopy(df.values), index=df.index, columns=df.columns)

dfa.PMd_spikes[0][0, 0] = -1 # changes the value in df
dfb.PMd_spikes[1][0, 0] = -1 # changes the value in df
dfc.PMd_spikes[2][0, 0] = -1 # leaves the value in df intact
raeedcho commented 2 years ago

I agree that we should change @copy_td to do a full deep copy, just so there aren't any surprises later on. Regarding the memory usage and slowness, I suppose we could re-evaluate places where we use the copy_td decorator to ensure that we really need to use it, but it seems like it should be used most of the time, if we're going for non-destructive manipulations as intended.