BayraktarLab / cell2location

Comprehensive mapping of tissue cell architecture via integrated single cell and spatial transcriptomics (cell2location model)
https://cell2location.readthedocs.io/en/latest/
Apache License 2.0
317 stars 58 forks source link

Error training model #66

Closed ccruizm closed 2 years ago

ccruizm commented 3 years ago

Good day!

I am trying to run cell2location using the integrated pipeline with scvi. When I reached the step to train the model on the ST data (cell2location.models.Cell2location), I get the error below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/scratch/11488722/ipykernel_67679/1717331508.py in <module>
      1 # create and train the model
----> 2 mod = cell2location.models.Cell2location(
      3     adata_vis, cell_state_df=inf_aver,
      4     # the expected average cell abundance: tissue-dependent
      5     # hyper-prior which can be estimated from paired histology:

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py in __init__(self, adata, cell_state_df, model_class, detection_mean_per_sample, detection_mean_correction, **model_kwargs)
    115         )
    116         self._model_summary_string = f'cell2location model with the following params: \nn_factors: {self.n_factors_} \nn_batch: {self.summary_stats["n_batch"]} '
--> 117         self.init_params_ = self._get_init_params(deepcopy(locals()))
    118 
    119     def train(

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    151             copier = getattr(x, "__deepcopy__", None)
    152             if copier is not None:
--> 153                 y = copier(memo)
    154             else:
    155                 reductor = dispatch_table.get(cls)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/views.py in __deepcopy__(self, memo)
     55     # TODO: This makes `deepcopy(obj)` return `obj._view_args.parent._adata_ref`, fix it
     56     def __deepcopy__(self, memo):
---> 57         parent, attrname, keys = self._view_args
     58         return deepcopy(getattr(parent._adata_ref, attrname))
     59 

TypeError: cannot unpack non-iterable NoneType object

What do you think the problem might be?

Thanks in advance!

vitkl commented 3 years ago

Could you please show which arguments you are providing to cell2location.models.Cell2location and how you setup anndata?

ccruizm commented 3 years ago

Hello @vitkl,

I used the following code:

mod = cell2location.models.Cell2location(
    adata_vis, cell_state_df=inf_aver,
    N_cells_per_location=30,
    detection_alpha=200
)

For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue"). The output of scvi.data.view_anndata_setup(adata_vis) looks like

Screenshot 2021-09-19 at 10 42 41

:

Hope this helps to find the source of the issue. Thanks!

vitkl commented 3 years ago

Hi Cristian

Unfortunately this doesn't help. I think the error means that there's an issue with copying some part of the anndata object. Could you please try deepcopy.deepcopy(adata) on your anndata object?

On Sun, 19 Sep 2021, 09:43 Cristian, @.***> wrote:

Hello @vitkl https://github.com/vitkl,

I used the following code:

mod = cell2location.models.Cell2location( adata_vis, cell_state_df=inf_aver, N_cells_per_location=30, detection_alpha=200 )

For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue"). The output of scvi.data.view_anndata_setup(adata_vis) looks like [image: Screenshot 2021-09-19 at 10 42 41] https://user-images.githubusercontent.com/37718031/133921143-a6ae8c8a-4144-4ec9-9d37-d08906c5f2e6.png :

Hope this helps to find the source of the issue. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BayraktarLab/cell2location/issues/66#issuecomment-922437897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMFTV2ZI7XUIZMXBFEQW3TUCWPDHANCNFSM5ECXUKYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

vitkl commented 3 years ago

And figure out how to change the object to make the copying work.

The issue could also be in pandas version. Is your version different to the tutorial?

Are you using a fresh conda environment as described in readme?

On Sun, 19 Sep 2021, 12:26 Vitalii Kleshchevnikov, @.***> wrote:

Hi Cristian

Unfortunately this doesn't help. I think the error means that there's an issue with copying some part of the anndata object. Could you please try deepcopy.deepcopy(adata) on your anndata object?

On Sun, 19 Sep 2021, 09:43 Cristian, @.***> wrote:

Hello @vitkl https://github.com/vitkl,

I used the following code:

mod = cell2location.models.Cell2location( adata_vis, cell_state_df=inf_aver, N_cells_per_location=30, detection_alpha=200 )

For setting up anndata I used scvi.data.setup_anndata(adata=adata_vis, batch_key="in_tissue"). The output of scvi.data.view_anndata_setup(adata_vis) looks like [image: Screenshot 2021-09-19 at 10 42 41] https://user-images.githubusercontent.com/37718031/133921143-a6ae8c8a-4144-4ec9-9d37-d08906c5f2e6.png :

Hope this helps to find the source of the issue. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/BayraktarLab/cell2location/issues/66#issuecomment-922437897, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFMFTV2ZI7XUIZMXBFEQW3TUCWPDHANCNFSM5ECXUKYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

vitkl commented 3 years ago

@ccruizm did you manage to solve the issue?

I think it is an issue with copying anndata objects and compatible pandas versions - rather than an issue with the cell2location package.

marioacera commented 3 years ago

Hi! Is there any news about this issue?

I got a fresh conda environment installed on a cluster and I have check that copy.deepcopy work on AnnDatas and basic dictionaries but I am still getting this error in the line: --> 117 self.initparams = self._get_init_params(deepcopy(locals())) Could it be an issue with the dictionary structure of locals().

Thanks in advance!

vitkl commented 3 years ago

Hi @adamgayoso

I am wondering if you have any idea about the error reported in this issue.

adamgayoso commented 3 years ago

Hi @vitkl there should be no deepcopy in

self.init_params_ = self._get_init_params(deepcopy(locals()))

I believe this arose with the other issue that came up a few weeks ago about encoder params I believe

vitkl commented 3 years ago

Hi @marioacera @ccruizm

I don't experience this issue and it also doesn't come up in the automatic tests. As Adam suggested, I removed the use of deepcopy in that line. Could you please try installing the latest cell2location from the specific branch with that edit pip install git+https://github.com/BayraktarLab/cell2location.git@deepcopy_issue? I will add that change to cell2location if it works for you.

vitkl commented 3 years ago

As a potential solution, I would suggest deleting all unnecessary data from the anndata object, leaving .obs['sample'], .uns['spatial'], .obsm['spatial'] slots.

ccruizm commented 2 years ago

Good day!

I have tried import copy copy.deepcopy(adata_vis) and get a similar error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/scratch/216023/ipykernel_118883/1954573082.py in <module>
      1 import copy
----> 2 copy.deepcopy(adata_vis)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    151             copier = getattr(x, "__deepcopy__", None)
    152             if copier is not None:
--> 153                 y = copier(memo)
    154             else:
    155                 reductor = dispatch_table.get(cls)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/views.py in __deepcopy__(self, memo)
     55     # TODO: This makes `deepcopy(obj)` return `obj._view_args.parent._adata_ref`, fix it
     56     def __deepcopy__(self, memo):
---> 57         parent, attrname, keys = self._view_args
     58         return deepcopy(getattr(parent._adata_ref, attrname))
     59 

TypeError: cannot unpack non-iterable NoneType object
adata_vis

I have installed the branch you recommended but still get the same error

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/scvi/model/base/_base_model.py:149: UserWarning: Make sure the registered X field in anndata contains unnormalized count data.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/scratch/216023/ipykernel_118883/1557678947.py in <module>
      1 # create and train the model
----> 2 mod = cell2location.models.Cell2location(
      3     adata_vis, cell_state_df=inf_aver,
      4     # the expected average cell abundance: tissue-dependent
      5     # hyper-prior which can be estimated from paired histology:

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py in __init__(self, adata, cell_state_df, model_class, detection_mean_per_sample, detection_mean_correction, **model_kwargs)
    115         )
    116         self._model_summary_string = f'cell2location model with the following params: \nn_factors: {self.n_factors_} \nn_batch: {self.summary_stats["n_batch"]} '
--> 117         self.init_params_ = self._get_init_params(deepcopy(locals()))
    118 
    119     def train(

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    170                     y = x
    171                 else:
--> 172                     y = _reconstruct(x, memo, *rv)
    173 
    174     # If is its own copy, don't memoize.

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
    268     if state is not None:
    269         if deep:
--> 270             state = deepcopy(state, memo)
    271         if hasattr(y, '__setstate__'):
    272             y.__setstate__(state)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    144     copier = _deepcopy_dispatch.get(cls)
    145     if copier is not None:
--> 146         y = copier(x, memo)
    147     else:
    148         if issubclass(cls, type):

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in _deepcopy_dict(x, memo, deepcopy)
    228     memo[id(x)] = y
    229     for key, value in x.items():
--> 230         y[deepcopy(key, memo)] = deepcopy(value, memo)
    231     return y
    232 d[dict] = _deepcopy_dict

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/copy.py in deepcopy(x, memo, _nil)
    151             copier = getattr(x, "__deepcopy__", None)
    152             if copier is not None:
--> 153                 y = copier(memo)
    154             else:
    155                 reductor = dispatch_table.get(cls)

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/anndata/_core/views.py in __deepcopy__(self, memo)
     55     # TODO: This makes `deepcopy(obj)` return `obj._view_args.parent._adata_ref`, fix it
     56     def __deepcopy__(self, memo):
---> 57         parent, attrname, keys = self._view_args
     58         return deepcopy(getattr(parent._adata_ref, attrname))
     59 

TypeError: cannot unpack non-iterable NoneType object

Even after deleting almost all fields in the anndata object

AnnData object with n_obs × n_vars = 3449 × 16267
    obs: 'in_tissue', '_scvi_batch', '_scvi_labels', '_scvi_local_l_mean', '_scvi_local_l_var', '_indices'
    var: 'SYMBOL'
    uns: 'spatial', '_scvi'
    obsm: 'spatial'
    varm: 'PCs'
    obsp: 'connectivities', 'distances'

What else should I try or am I doing something wrong?

Thanks in advance

vitkl commented 2 years ago

Hi @ccruizm

This confirms the issue with deepcopy and some of the data formats in your anndata.

Use of deepcopy in that part of the code is not essential but I would like to confirm that removing it solves your issue. Could I ask you to make sure that you install the version from the deepcopy_issue branch and make use it is loaded? Cell2location might not be loading correctly (installed into the wrong conda environment?) because after installing the code from that branch you should no longer see this:

/hpc/pmc_stunnenberg/cruiz/miniconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py in __init__(self, adata, cell_state_df, model_class, detection_mean_per_sample, detection_mean_correction, **model_kwargs)
    115         )
    116         self._model_summary_string = f'cell2location model with the following params: \nn_factors: {self.n_factors_} \nn_batch: {self.summary_stats["n_batch"]} '
--> 117         self.init_params_ = self._get_init_params(deepcopy(locals()))
    118 
    119     def train(

because line 117 looks like this:

self.init_params_ = self._get_init_params(deepcopy(locals()))
ccruizm commented 2 years ago

Hello @vitkl

You were right, indeed the installation wasn't done properly. I reinstalled it and now it works even without deleting any data from the anndata object! thank you so much for your help!

vitkl commented 2 years ago

Merged https://github.com/BayraktarLab/cell2location/pull/85 with a fix for this issue