Closed ccruizm closed 2 years ago
Could you please show which arguments you are providing to cell2location.models.Cell2location
and how you setup anndata?
Hello @vitkl,
I used the following code:
mod = cell2location.models.Cell2location(
adata_vis, cell_state_df=inf_aver,
N_cells_per_location=30,
detection_alpha=200
)
For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue")
. The output of scvi.data.view_anndata_setup(adata_vis)
looks like
:
Hope this helps to find the source of the issue. Thanks!
Hi Cristian
Unfortunately this doesn't help. I think the error means that there's an issue with copying some part of the anndata object. Could you please try deepcopy.deepcopy(adata) on your anndata object?
On Sun, 19 Sep 2021, 09:43 Cristian, @.***> wrote:
Hello @vitkl https://github.com/vitkl,
I used the following code:
mod = cell2location.models.Cell2location( adata_vis, cell_state_df=inf_aver, N_cells_per_location=30, detection_alpha=200 )
For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue"). The output of scvi.data.view_anndata_setup(adata_vis) looks like [image: Screenshot 2021-09-19 at 10 42 41] https://user-images.githubusercontent.com/37718031/133921143-a6ae8c8a-4144-4ec9-9d37-d08906c5f2e6.png :
Hope this helps to find the source of the issue. Thanks!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BayraktarLab/cell2location/issues/66#issuecomment-922437897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMFTV2ZI7XUIZMXBFEQW3TUCWPDHANCNFSM5ECXUKYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
And figure out how to change the object to make the copying work.
The issue could also be in pandas version. Is your version different to the tutorial?
Are you using a fresh conda environment as described in readme?
On Sun, 19 Sep 2021, 12:26 Vitalii Kleshchevnikov, @.***> wrote:
Hi Cristian
Unfortunately this doesn't help. I think the error means that there's an issue with copying some part of the anndata object. Could you please try deepcopy.deepcopy(adata) on your anndata object?
On Sun, 19 Sep 2021, 09:43 Cristian, @.***> wrote:
Hello @vitkl https://github.com/vitkl,
I used the following code:
mod = cell2location.models.Cell2location( adata_vis, cell_state_df=inf_aver, N_cells_per_location=30, detection_alpha=200 )
For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue"). The output of scvi.data.view_anndata_setup(adata_vis) looks like [image: Screenshot 2021-09-19 at 10 42 41] https://user-images.githubusercontent.com/37718031/133921143-a6ae8c8a-4144-4ec9-9d37-d08906c5f2e6.png :
Hope this helps to find the source of the issue. Thanks!
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BayraktarLab/cell2location/issues/66#issuecomment-922437897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMFTV2ZI7XUIZMXBFEQW3TUCWPDHANCNFSM5ECXUKYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
@ccruizm did you manage to solve the issue?
I think it is an issue with copying anndata objects and compatible pandas versions - rather than an issue with the cell2location package.
Hi! Is there any news about this issue?
I got a fresh conda environment installed on a cluster and I have check that copy.deepcopy work on AnnDatas and basic dictionaries but I am still getting this error in the line: --> 117 self.initparams = self._get_init_params(deepcopy(locals())) Could it be an issue with the dictionary structure of locals().
Thanks in advance!
Hi @adamgayoso
I am wondering if you have any idea about the error reported in this issue.
Hi @vitkl there should be no deepcopy in
self.init_params_ = self._get_init_params(deepcopy(locals()))
I believe this arose with the other issue that came up a few weeks ago about encoder params I believe
Hi @marioacera @ccruizm
I don't experience this issue and it also doesn't come up in the automatic tests. As Adam suggested, I removed the use of deepcopy
in that line. Could you please try installing the latest cell2location from the specific branch with that edit pip install git+https://github.com/BayraktarLab/cell2location.git@deepcopy_issue
? I will add that change to cell2location if it works for you.
As a potential solution, I would suggest deleting all unnecessary data from the anndata object, leaving .obs['sample']
, .uns['spatial']
, .obsm['spatial']
slots.
Good day!
I have tried import copy
copy.deepcopy(adata_vis)
and get a similar error
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/scratch/216023/ipykernel_118883/1954573082.py in <module>
1 import copy
----> 2 copy.deepcopy(adata_vis)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
170 y = x
171 else:
--> 172 y = _reconstruct(x, memo, *rv)
173
174 # If is its own copy, don't memoize.
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
268 if state is not None:
269 if deep:
--> 270 state = deepcopy(state, memo)
271 if hasattr(y, '__setstate__'):
272 y.__setstate__(state)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
170 y = x
171 else:
--> 172 y = _reconstruct(x, memo, *rv)
173
174 # If is its own copy, don't memoize.
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
268 if state is not None:
269 if deep:
--> 270 state = deepcopy(state, memo)
271 if hasattr(y, '__setstate__'):
272 y.__setstate__(state)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
151 copier = getattr(x, "__deepcopy__", None)
152 if copier is not None:
--> 153 y = copier(memo)
154 else:
155 reductor = dispatch_table.get(cls)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/views.py in __deepcopy__(self, memo)
55 # TODO: This makes `deepcopy(obj)` return `obj._view_args.parent._adata_ref`, fix it
56 def __deepcopy__(self, memo):
---> 57 parent, attrname, keys = self._view_args
58 return deepcopy(getattr(parent._adata_ref, attrname))
59
TypeError: cannot unpack non-iterable NoneType object
adata_vis
I have installed the branch you recommended but still get the same error
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/scvi/model/base/_base_model.py:149: UserWarning: Make sure the registered X field in anndata contains unnormalized count data.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/scratch/216023/ipykernel_118883/1557678947.py in <module>
1 # create and train the model
----> 2 mod = cell2location.models.Cell2location(
3 adata_vis, cell_state_df=inf_aver,
4 # the expected average cell abundance: tissue-dependent
5 # hyper-prior which can be estimated from paired histology:
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py in __init__(self, adata, cell_state_df, model_class, detection_mean_per_sample, detection_mean_correction, **model_kwargs)
115 )
116 self._model_summary_string = f'cell2location model with the following params: \nn_factors: {self.n_factors_} \nn_batch: {self.summary_stats["n_batch"]} '
--> 117 self.init_params_ = self._get_init_params(deepcopy(locals()))
118
119 def train(
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
170 y = x
171 else:
--> 172 y = _reconstruct(x, memo, *rv)
173
174 # If is its own copy, don't memoize.
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
268 if state is not None:
269 if deep:
--> 270 state = deepcopy(state, memo)
271 if hasattr(y, '__setstate__'):
272 y.__setstate__(state)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
170 y = x
171 else:
--> 172 y = _reconstruct(x, memo, *rv)
173
174 # If is its own copy, don't memoize.
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
268 if state is not None:
269 if deep:
--> 270 state = deepcopy(state, memo)
271 if hasattr(y, '__setstate__'):
272 y.__setstate__(state)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
170 y = x
171 else:
--> 172 y = _reconstruct(x, memo, *rv)
173
174 # If is its own copy, don't memoize.
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
268 if state is not None:
269 if deep:
--> 270 state = deepcopy(state, memo)
271 if hasattr(y, '__setstate__'):
272 y.__setstate__(state)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
144 copier = _deepcopy_dispatch.get(cls)
145 if copier is not None:
--> 146 y = copier(x, memo)
147 else:
148 if issubclass(cls, type):
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
228 memo[id(x)] = y
229 for key, value in x.items():
--> 230 y[deepcopy(key, memo)] = deepcopy(value, memo)
231 return y
232 d[dict] = _deepcopy_dict
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
151 copier = getattr(x, "__deepcopy__", None)
152 if copier is not None:
--> 153 y = copier(memo)
154 else:
155 reductor = dispatch_table.get(cls)
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/views.py in __deepcopy__(self, memo)
55 # TODO: This makes `deepcopy(obj)` return `obj._view_args.parent._adata_ref`, fix it
56 def __deepcopy__(self, memo):
---> 57 parent, attrname, keys = self._view_args
58 return deepcopy(getattr(parent._adata_ref, attrname))
59
TypeError: cannot unpack non-iterable NoneType object
Even after deleting almost all fields in the anndata object
AnnData object with n_obs × n_vars = 3449 × 16267
obs: 'in_tissue', '_scvi_batch', '_scvi_labels', '_scvi_local_l_mean', '_scvi_local_l_var', '_indices'
var: 'SYMBOL'
uns: 'spatial', '_scvi'
obsm: 'spatial'
varm: 'PCs'
obsp: 'connectivities', 'distances'
What else should I try or am I doing something wrong?
Thanks in advance
Hi @ccruizm
This confirms the issue with deepcopy and some of the data formats in your anndata.
Use of deepcopy in that part of the code is not essential but I would like to confirm that removing it solves your issue. Could I ask you to make sure that you install the version from the deepcopy_issue
branch and make use it is loaded?
Cell2location might not be loading correctly (installed into the wrong conda environment?) because after installing the code from that branch you should no longer see this:
/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py in __init__(self, adata, cell_state_df, model_class, detection_mean_per_sample, detection_mean_correction, **model_kwargs)
115 )
116 self._model_summary_string = f'cell2location model with the following params: \nn_factors: {self.n_factors_} \nn_batch: {self.summary_stats["n_batch"]} '
--> 117 self.init_params_ = self._get_init_params(deepcopy(locals()))
118
119 def train(
because line 117 looks like this:
self.init_params_ = self._get_init_params(deepcopy(locals()))
Hello @vitkl
You were right, indeed the installation wasn't done properly. I reinstalled it and now it works even without deleting any data from the anndata object! thank you so much for your help!
Merged https://github.com/BayraktarLab/cell2location/pull/85 with a fix for this issue
Good day!
I am trying to run cell2location using the integrated pipeline with scvi. When I reached the step to train the model on the ST data (
cell2location.models.Cell2location
), I get the error below:What do you think the problem might be?
Thanks in advance!