ValueError: 4 columns passed, passed data had 8 columns

[Biomamba]

Dear author: Thank you for your contribution to stereopy. When I run data.tl.cell_cell_communication(), I met the following error: `import stereo as st import warnings warnings.filterwarnings('ignore')

data = st.io.read_h5ad('../data/adata_VCM_EP.h5ad')

data.tl.cal_qc()

data.tl.filter_cells(min_gene=1, max_gene=2000, min_n_genes_by_counts=3, max_n_genes_by_counts=800, pct_counts_mt=6, inplace=True)

data.tl.raw_checkpoint() data.tl.normalize_total()

data.tl.pca(n_pcs=50, res_key='pca', use_highly_genes=False) data.tl.neighbors(pca_res_key='pca', res_key='neighbors') data.tl.leiden(neighbors_res_key='neighbors', res_key='leiden') data.tl.cell_cell_communication( analysis_type='statistical', cluster_res_key='leiden', species='MOUSE', database='cellphonedb', res_key='cell_cell_communication' )`

It seem to have some problems of result transfer. Cloud you please tell me how to solved it?

[tanliwei-coder]

What is the version of your stereopy?

[Biomamba]

What is the version of your stereopy?

I update my stereopy into version 0.14.0b1. When I run the following code, a new problem came out. ccd = data.tl.community_detection( annotation='sim anno', out_path='results/whole_brain', win_sizes='150', sliding_steps='50', scatter_thres=0.12, downsample_rate=80, cluster_algo='agglomerative', n_clusters=16, resolution=0.25, plotting=5, hide_plots=True )

OperationalError Traceback (most recent call last) File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2336, in Engine._wrap_pool_connect(self, fn, connection) 2335 try: -> 2336 return fn() 2337 except dialect.dbapi.Error as e:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:364, in Pool.connect(self) 363 if not self._use_threadlocal: --> 364 return _ConnectionFairy._checkout(self) 366 try:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:778, in _ConnectionFairy._checkout(cls, pool, threadconns, fairy) 777 if not fairy: --> 778 fairy = _ConnectionRecord.checkout(pool) 780 fairy._pool = pool

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:495, in _ConnectionRecord.checkout(cls, pool) 493 @classmethod 494 def checkout(cls, pool): --> 495 rec = pool._do_get() 496 try:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:241, in NullPool._do_get(self) 240 def _do_get(self): --> 241 return self._create_connection()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:309, in Pool._create_connection(self) 307 """Called by subclasses to create a new ConnectionRecord.""" --> 309 return _ConnectionRecord(self)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:440, in _ConnectionRecord.init(self, pool, connect) 439 if connect: --> 440 self.__connect(first_connect_check=True) 441 self.finalize_callback = deque()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:661, in _ConnectionRecord.__connect(self, first_connect_check) 660 with util.safe_reraise(): --> 661 pool.logger.debug("Error on connect(): %s", e) 662 else:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:68, in safereraise.exit(self, type, value, traceback) 67 if not self.warnonly: ---> 68 compat.raise( 69 exc_value, 70 with_traceback=exc_tb, 71 ) 72 else:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/util/compat.py:182, in raise_(failed resolving arguments) 181 try: --> 182 raise exception 183 finally: 184 # credit to 185 # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/ 186 # as the traceback object creates a cycle

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:656, in _ConnectionRecord.__connect(self, first_connect_check) 655 self.starttime = time.time() --> 656 connection = pool._invoke_creator(self) 657 pool.logger.debug("Created new connection %r", connection)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py:114, in DefaultEngineStrategy.create..connect(connection_record) 113 return connection --> 114 return dialect.connect(*cargs, **cparams)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/default.py:508, in DefaultDialect.connect(self, *cargs, cparams) 506 def connect(self, *cargs, *cparams): 507 # inherits the docstring from interfaces.Dialect.connect --> 508 return self.dbapi.connect(cargs, cparams)

OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

OperationalError Traceback (most recent call last) Cell In[7], line 2 1 # 演示数据集是一个小鼠的数据集： ----> 2 data.tl.cell_cell_communication( 3 analysis_type='statistical',# 是否返回统计量 4 cluster_res_key='leiden',# 参与分析的注释标签 5 species='MOUSE',# 物种 6 database='cellphonedb',# 数据库来源 7 res_key='cell_cell_communication' # 输出结果存放 8 )

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/stereopy-0.14.0b1-py3.8.egg/stereo/algorithm/cell_cell_communication/main.py:131, in CellCellCommunication.main(self, analysis_type, cluster_res_key, micro_envs, species, database, homogene_path, counts_identifiers, subsampling, subsampling_log, subsampling_num_pc, subsampling_num_cells, pca_res_key, separator_cluster, separator_interaction, iterations, threshold, processes, pvalue, result_precision, output_path, means_filename, pvalues_filename, significant_means_filename, deconvoluted_filename, output_format, res_key) 128 logger.info(f'species: {species.upper()}') 129 logger.info(f'database: {database}') --> 131 interactions, genes, complex_composition, complex_expanded = self._get_ref_database(db_path) 133 counts, meta = self._prepare_data(cluster_res_key) 135 # 1. preprocess and validate input data 136 137 # 1.1. preprocess and validate meta data (cell name as index, cell type as the only column). 138 # meta = self._check_meta_data(meta) 139 140 # 1.2. preprocess and validate counts data

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/stereopy-0.14.0b1-py3.8.egg/stereo/algorithm/cell_cell_communication/main.py:370, in CellCellCommunication._get_ref_database(self, db_path) 367 database_manager.add_repository(ProteinRepository) 369 # get data form database --> 370 interactions = database_manager.get_repository('interaction').get_all_expanded(include_gene=False) 371 genes = database_manager.get_repository('gene').get_all_expanded() # join gene, protein, multidata 372 complex_composition = database_manager.get_repository('complex').get_all_compositions()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/stereopy-0.14.0b1-py3.8.egg/stereo/algorithm/cell_cell_communication/utils/sqlalchemy_repository.py:325, in InteractionRepository.get_all_expanded(self, include_gene, suffixes) 322 def get_all_expanded(self, include_gene=True, suffixes=('_1', '_2')): 323 interactions_query = self.database_manager.database.session.query(Interaction) --> 325 interactions = pd.read_sql(interactions_query.statement, self.database_manager.database.engine) 327 multidata_expanded: pd.DataFrame = self.database_manager.get_repository('multidata').get_all_expanded( 328 include_gene) 330 multidata_expanded = multidata_expanded.astype({'id_multidata': 'int64'})

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/pandas/io/sql.py:590, in read_sql(sql, con, index_col, coerce_float, params, parse_dates, columns, chunksize) 581 return pandas_sql.read_table( 582 sql, 583 index_col=index_col, (...) 587 chunksize=chunksize, 588 ) 589 else: --> 590 return pandas_sql.read_query( 591 sql, 592 index_col=index_col, 593 params=params, 594 coerce_float=coerce_float, 595 parse_dates=parse_dates, 596 chunksize=chunksize, 597 )

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/pandas/io/sql.py:1560, in SQLDatabase.read_query(self, sql, index_col, coerce_float, parse_dates, params, chunksize, dtype) 1512 """ 1513 Read SQL query into a DataFrame. 1514 (...) 1556 1557 """ 1558 args = _convert_params(sql, params) -> 1560 result = self.execute(*args) 1561 columns = result.keys() 1563 if chunksize is not None:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/pandas/io/sql.py:1405, in SQLDatabase.execute(self, *args, kwargs) 1403 def execute(self, *args, *kwargs): 1404 """Simple passthrough to SQLAlchemy connectable""" -> 1405 return self.connectable.execution_options().execute(args, kwargs)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2234, in Engine.execute(self, statement, *multiparams, params) 2214 def execute(self, statement, *multiparams, *params): 2215 """Executes the given construct and returns a 2216 :class:_engine.ResultProxy. 2217 (...) 2231 2232 """ -> 2234 connection = self._contextual_connect(close_with_result=True) 2235 return connection.execute(statement, multiparams, params)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2302, in Engine._contextual_connect(self, close_with_result, kwargs) 2299 def _contextual_connect(self, close_with_result=False, kwargs): 2300 return self._connection_cls( 2301 self, -> 2302 self._wrap_pool_connect(self.pool.connect, None), 2303 close_with_result=close_with_result, 2304 **kwargs 2305 )

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2339, in Engine._wrap_pool_connect(self, fn, connection) 2337 except dialect.dbapi.Error as e: 2338 if connection is None: -> 2339 Connection._handle_dbapi_exceptionnoconnection( 2340 e, dialect, self 2341 ) 2342 else: 2343 util.raise( 2344 sys.exc_info()[1], with_traceback=sys.exc_info()[2] 2345 )

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:1583, in Connection._handle_dbapi_exceptionnoconnection(cls, e, dialect, engine) 1581 util.raise(newraise, with_traceback=excinfo[2], from=e) 1582 elif shouldwrap: -> 1583 util.raise( 1584 sqlalchemy_exception, with_traceback=excinfo[2], from=e 1585 ) 1586 else: 1587 util.raise_(exc_info[1], with_traceback=exc_info[2])

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/util/compat.py:182, in raise_(failed resolving arguments) 179 exception.cause = replace_context 181 try: --> 182 raise exception 183 finally: 184 # credit to 185 # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/ 186 # as the traceback object creates a cycle 187 del exception, replacecontext, from, with_traceback

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/base.py:2336, in Engine._wrap_pool_connect(self, fn, connection) 2334 dialect = self.dialect 2335 try: -> 2336 return fn() 2337 except dialect.dbapi.Error as e: 2338 if connection is None:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:364, in Pool.connect(self) 356 """Return a DBAPI connection from the pool. 357 358 The connection is instrumented such that when its (...) 361 362 """ 363 if not self._use_threadlocal: --> 364 return _ConnectionFairy._checkout(self) 366 try: 367 rec = self._threadconns.current()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:778, in _ConnectionFairy._checkout(cls, pool, threadconns, fairy) 775 @classmethod 776 def _checkout(cls, pool, threadconns=None, fairy=None): 777 if not fairy: --> 778 fairy = _ConnectionRecord.checkout(pool) 780 fairy._pool = pool 781 fairy._counter = 0

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:495, in _ConnectionRecord.checkout(cls, pool) 493 @classmethod 494 def checkout(cls, pool): --> 495 rec = pool._do_get() 496 try: 497 dbapi_connection = rec.get_connection()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:241, in NullPool._do_get(self) 240 def _do_get(self): --> 241 return self._create_connection()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:309, in Pool._create_connection(self) 306 def _create_connection(self): 307 """Called by subclasses to create a new ConnectionRecord.""" --> 309 return _ConnectionRecord(self)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:440, in _ConnectionRecord.init(self, pool, connect) 438 self.pool = pool 439 if connect: --> 440 self.connect(first_connect_check=True) 441 self.finalize_callback = deque()

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:661, in _ConnectionRecord.__connect(self, first_connect_check) 659 except Exception as e: 660 with util.safe_reraise(): --> 661 pool.logger.debug("Error on connect(): %s", e) 662 else: 663 if first_connect_check:

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:68, in safereraise.exit(self, type, value, traceback) 66 self._exc_info = None # remove potential circular references 67 if not self.warnonly: ---> 68 compat.raise( 69 exc_value, 70 with_traceback=exc_tb, 71 ) 72 else: 73 if not compat.py3k and self._exc_info and self._exc_info[1]: 74 # emulate Py3K's behavior of telling us when an exception 75 # occurs in an exception handler.

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/util/compat.py:182, in raise_(failed resolving arguments) 179 exception.cause = replace_context 181 try: --> 182 raise exception 183 finally: 184 # credit to 185 # https://cosmicpercolator.com/2016/01/13/exception-leaks-in-python-2-and-3/ 186 # as the traceback object creates a cycle 187 del exception, replacecontext, from, with_traceback

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/pool/base.py:656, in _ConnectionRecord.__connect(self, first_connect_check) 654 try: 655 self.starttime = time.time() --> 656 connection = pool._invoke_creator(self) 657 pool.logger.debug("Created new connection %r", connection) 658 self.connection = connection

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/strategies.py:114, in DefaultEngineStrategy.create..connect(connection_record) 112 if connection is not None: 113 return connection --> 114 return dialect.connect(*cargs, **cparams)

File ~/miniconda3/envs/stereopy/lib/python3.8/site-packages/sqlalchemy/engine/default.py:508, in DefaultDialect.connect(self, *cargs, cparams) 506 def connect(self, *cargs, *cparams): 507 # inherits the docstring from interfaces.Dialect.connect --> 508 return self.dbapi.connect(cargs, cparams)

OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/13/e3q8)

What should I do?

[tanliwei-coder]

It looks like the databases are lost because of some unknown reasons, normally, the databases should be installed with stereopy, you can reinstall stereopy and try again.

Someone has met this similar error postted in issue-166.

[Biomamba]

It looks like the databases are lost because of some unknown reasons, normally, the databases should be installed with stereopy, you can reinstall stereopy and try again.

Someone has met this similar error postted in issue-166.

I have reinstalled my stereopy and the current version is 0.13.0b1. The directory cell_cell_communication/database contains all required files including database files of cellphonedb, liana and celltalkdb. However, only celltalkdb can be successful performed in data.tl.cell_cell_communication(). Moreover, while I used the celltalkdb, the functions data.plt.ccc_circos_plot() and data.plt.ccc_sankey_plot() both return the Error: TypeError: 'NoneType' object is not callable. Cloud you please tell me how to solve it?

[Zhenbin24]

Hello, you can try installing the latest version.