Thank you for making this wonderful work available. I am currently running the spSeudoMap_example.ipynb file, but I encountered an error. Could you please assist me in resolving it? Thanks a lot!
GPU is available and will be used
Shape of the provided spatial data is (2702, 32285)
normalizing counts per cell
finished (0:00:00)
Using TensorFlow backend.
Shape of the provided single-cell data is (13309, 12819)
normalizing counts per cell
finished (0:00:00)
ranking genes
finished: added to `.uns['rank_genes_groups']`
'names', sorted np.recarray to be indexed by group ids
'scores', sorted np.recarray to be indexed by group ids
'logfoldchanges', sorted np.recarray to be indexed by group ids
'pvals', sorted np.recarray to be indexed by group ids
'pvals_adj', sorted np.recarray to be indexed by group ids (0:00:10)
Total number of cell type marker genes is 524
Dataframe summarizing counts of pseudobulk transcriptomes and log fold change
sp_count sc_count sp_ratio sc_ratio logfc
mt-Nd3 498123.0 616.0 6.642646e-03 5.731767e-06 10.178562
Hba-a1 122435.0 175.0 1.632714e-03 1.628343e-06 9.969652
Hba-a2 100737.0 148.0 1.343363e-03 1.377113e-06 9.929987
Msx1 652.0 1.0 8.694650e-06 9.304817e-09 9.867933
Hbb-bt 55835.0 94.0 7.445794e-04 8.746528e-07 9.733500
... ... ... ... ... ...
Hs6st3 3.0 123132.0 4.000606e-08 1.145721e-03 -14.805677
4930419G24Rik 1.0 44670.0 1.333535e-08 4.156462e-04 -14.927814
Gm37240 1.0 51744.0 1.333535e-08 4.814685e-04 -15.139898
Sorbs2os 1.0 65776.0 1.333535e-08 6.120336e-04 -15.486068
Xist 1.0 67598.0 1.333535e-08 6.289870e-04 -15.525488
[11939 rows x 5 columns]
Total number of genes for training: 655
computing score 'pseudotype'
finished: added
'pseudotype', score of gene set (adata.obs).
300 total control genes are used. (0:00:00)
Genes used for calculation of pseudotype fraction
['mt-Nd3', 'Hba-a1', 'Hba-a2', 'Msx1', 'Hbb-bt', 'Hbb-bs', 'mt-Atp6', 'Cldn5', 'Ly6a', 'mt-Cytb', 'mt-Co2', 'Sparc', 'mt-Nd1', 'Myoc', 'Vtn', 'mt-Co1', 'Pmch', 'Otp', 'Mlc1', 'Ttr']
Size of spatial, single-cell, pseudospot, and cell fraction data: (2702, 655) (13309, 655) (13510, 655) (13510, 24)
Epoch 1/10
27/27 [==============================] - 1s 2ms/step - loss: 1.9174 - mae: 0.0557
Epoch 2/10
27/27 [==============================] - 0s 2ms/step - loss: 1.2118 - mae: 0.0478
Epoch 3/10
27/27 [==============================] - 0s 2ms/step - loss: 0.7561 - mae: 0.0397
Epoch 4/10
27/27 [==============================] - 0s 2ms/step - loss: 0.4277 - mae: 0.0283
Epoch 5/10
27/27 [==============================] - 0s 2ms/step - loss: 0.2626 - mae: 0.0188
Epoch 6/10
27/27 [==============================] - 0s 2ms/step - loss: 0.2002 - mae: 0.0142
Epoch 7/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1728 - mae: 0.0123
Epoch 8/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1558 - mae: 0.0113
Epoch 9/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1428 - mae: 0.0105
Epoch 10/10
27/27 [==============================] - 0s 2ms/step - loss: 0.1338 - mae: 0.0099
initial_train_done
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Input In [18], in <cell line: 3>()
1 # from spSeudoMap.pred_cellf_spSeudoMap import pred_cellf_spSeudoMap
2 from spSeudoMap.pred_cellf_spSeudoMap import pred_cellf_spSeudoMap
----> 3 adata_spatial_cellf = pred_cellf_spSeudoMap(adata_sp=adata_spatial, adata_sc=adata_sc_sub, count_from_raw=False,
4 gpu=True, celltype=celltype, num_markers=num_markers,
5 mixture_mode='pseudotype', seed_num=0,
6 mk_ratio_fix=True, mk_ratio=mk_ratio,
7 pseudo_frac_m=pseudo_frac_m, pseudo_frac_std=pseudo_frac_std,
8 nmix=nmix, npseudo=npseudo, alpha=0.6, alpha_lr=5, emb_dim=64,
9 batch_size=512, n_iterations=3000, init_train_epoch=10,
10 outdir=out_dir, return_format='anndata')
File ~/CMRI Dropbox/Chunlei liu/project_integration_benchmark/Chunlei/benchmark_method/diagonal integration/python/spSeudoMap/spSeudoMap/spSeudoMap/pred_cellf_spSeudoMap.py:243, in pred_cellf_spSeudoMap(adata_sp, adata_sc, count_from_raw, gpu, celltype, num_markers, mixture_mode, seed_num, mk_ratio_fix, mk_ratio, pseudo_num_genes, pseudo_frac_m, pseudo_frac_std, num_top_genes, nmix, npseudo, alpha, alpha_lr, emb_dim, batch_size, n_iterations, init_train_epoch, outdir, return_format)
238 print('Size of spatial, single-cell, pseudospot, and cell fraction data:',
239 mat_sp_s.shape, mat_sc_s.shape, sc_mix_s.shape, lab_mix.shape)
242 # Train the domain adaptation model
--> 243 embs, clssmodel = da_cellfraction.train(sc_mix_s, lab_mix, mat_sp_s,
244 emb_dim = emb_dim, batch_size = batch_size,
245 n_iterations = n_iterations,
246 enable_dann = True,
247 alpha = alpha, alpha_lr = alpha_lr,
248 initial_train = True,
249 initial_train_epochs = init_train_epoch)
250 # Prediction of cell fraction in each spot
251 pred_sp = pd.DataFrame(clssmodel.predict(mat_sp_s))
File ~/CMRI Dropbox/Chunlei liu/project_integration_benchmark/Chunlei/benchmark_method/diagonal integration/python/spSeudoMap/spSeudoMap/spSeudoMap/da_cellfraction.py:164, in train(Xs, ys, Xt, yt, emb_dim, batch_size, enable_dann, n_iterations, alpha, alpha_lr, initial_train, initial_train_epochs)
162 print(y_adversarial_2.shape)
163 print("!!!!")
--> 164 domain_classification_model.train_on_batch(X_adv, [y_adversarial_2])
165 # retrain the domain classification model with inverted labels for pseudospot + spots
167 k = 0
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:1722, in Model.train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics, return_dict)
1718 iterator = data_adapter.single_batch_iterator(self.distribute_strategy, x,
1719 y, sample_weight,
1720 class_weight)
1721 self.train_function = self.make_train_function()
-> 1722 logs = self.train_function(iterator)
1724 if reset_metrics:
1725 self.reset_metrics()
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:828, in Function.__call__(self, *args, **kwds)
826 tracing_count = self.experimental_get_tracing_count()
827 with trace.Trace(self._name) as tm:
--> 828 result = self._call(*args, **kwds)
829 compiler = "xla" if self._experimental_compile else "nonXla"
830 new_tracing_count = self.experimental_get_tracing_count()
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:871, in Function._call(self, *args, **kwds)
868 try:
869 # This is the first call of __call__, so we have to initialize.
870 initializers = []
--> 871 self._initialize(args, kwds, add_initializers_to=initializers)
872 finally:
873 # At this point we know that the initialization is complete (or less
874 # interestingly an exception was raised) so we no longer need a lock.
875 self._lock.release()
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:725, in Function._initialize(self, args, kwds, add_initializers_to)
722 self._lifted_initializer_graph = lifted_initializer_graph
723 self._graph_deleter = FunctionDeleter(self._lifted_initializer_graph)
724 self._concrete_stateful_fn = (
--> 725 self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
726 *args, **kwds))
728 def invalid_creator_scope(*unused_args, **unused_kwds):
729 """Disables variable creation."""
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:2969, in Function._get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
2967 args, kwargs = None, None
2968 with self._lock:
-> 2969 graph_function, _ = self._maybe_define_function(args, kwargs)
2970 return graph_function
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:3361, in Function._maybe_define_function(self, args, kwargs)
3357 return self._define_function_with_shape_relaxation(
3358 args, kwargs, flat_args, filtered_flat_args, cache_key_context)
3360 self._function_cache.missed.add(call_context_key)
-> 3361 graph_function = self._create_graph_function(args, kwargs)
3362 self._function_cache.primary[cache_key] = graph_function
3364 return graph_function, filtered_flat_args
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/function.py:3196, in Function._create_graph_function(self, args, kwargs, override_flat_arg_shapes)
3191 missing_arg_names = [
3192 "%s_%d" % (arg, i) for i, arg in enumerate(missing_arg_names)
3193 ]
3194 arg_names = base_arg_names + missing_arg_names
3195 graph_function = ConcreteFunction(
-> 3196 func_graph_module.func_graph_from_py_func(
3197 self._name,
3198 self._python_function,
3199 args,
3200 kwargs,
3201 self.input_signature,
3202 autograph=self._autograph,
3203 autograph_options=self._autograph_options,
3204 arg_names=arg_names,
3205 override_flat_arg_shapes=override_flat_arg_shapes,
3206 capture_by_value=self._capture_by_value),
3207 self._function_attributes,
3208 function_spec=self.function_spec,
3209 # Tell the ConcreteFunction to clean up its graph once it goes out of
3210 # scope. This is not the default behavior since it gets used in some
3211 # places (like Keras) where the FuncGraph lives longer than the
3212 # ConcreteFunction.
3213 shared_func_graph=False)
3214 return graph_function
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py:990, in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes)
987 else:
988 _, original_func = tf_decorator.unwrap(python_func)
--> 990 func_outputs = python_func(*func_args, **func_kwargs)
992 # invariant: `func_outputs` contains only Tensors, CompositeTensors,
993 # TensorArrays and `None`s.
994 func_outputs = nest.map_structure(convert, func_outputs,
995 expand_composites=True)
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py:634, in Function._defun_with_scope.<locals>.wrapped_fn(*args, **kwds)
632 xla_context.Exit()
633 else:
--> 634 out = weak_wrapped_fn().__wrapped__(*args, **kwds)
635 return out
File ~/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py:977, in func_graph_from_py_func.<locals>.wrapper(*args, **kwargs)
975 except Exception as e: # pylint:disable=broad-except
976 if hasattr(e, "ag_error_metadata"):
--> 977 raise e.ag_error_metadata.to_exception(e)
978 else:
979 raise
AttributeError: in user code:
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:800 train_function *
return step_function(self, iterator)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:790 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:1259 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2730 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:3417 _call_for_each_replica
return fn(*args, **kwargs)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:783 run_step **
outputs = model.train_step(data)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:753 train_step
self.compiled_metrics.update_state(y, y_pred, sample_weight)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:387 update_state
self.build(y_pred, y_true)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:317 build
self._metrics = nest.map_structure_up_to(y_pred, self._get_metric_objects,
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1159 map_structure_up_to
return map_structure_with_tuple_paths_up_to(
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1257 map_structure_with_tuple_paths_up_to
results = [
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1258 <listcomp>
func(*args, **kwargs) for args in zip(flat_path_gen, *flat_value_gen)
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/util/nest.py:1161 <lambda>
lambda _, *values: func(*values), # Discards the path arg.
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:418 _get_metric_objects
return [self._get_metric_object(m, y_t, y_p) for m in metrics]
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:418 <listcomp>
return [self._get_metric_object(m, y_t, y_p) for m in metrics]
/home/lcl/anaconda3/envs/cellpose/lib/python3.8/site-packages/tensorflow/python/keras/engine/compile_utils.py:439 _get_metric_object
y_t_rank = len(y_t.shape.as_list())
AttributeError: 'tuple' object has no attribute 'shape'
Thank you for making this wonderful work available. I am currently running the spSeudoMap_example.ipynb file, but I encountered an error. Could you please assist me in resolving it? Thanks a lot!