I got an error ”sqlite3.OperationalError: database is locked‘, when running "cellphonedb database generate --user-interactions interaction_input.csv --user-interactions-only".
I tried to run it on python3.7 and 3.8 on two linux computers(Ubuntu 16 and 18), and installed different versions of cellphonedb (from v2.1.7 to v2.0.0), but it couldn’t solve this problem. It has bothered me for two days, Do you have any suggestions?
(cpdb) gaolin@gaolin://192.168.0.201/e/desktop/cellphonedb_example_data/example_data$ cellphonedb database generate --user-interactions interaction_input.csv --
user-interactions-only
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.cluster.kmeans module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.cluster. Anything that cannot be imported from sklearn.cluster is now part of the private API.
warnings.warn(message, FutureWarning)
read local uniprot file
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:243: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'secreted_desc': pd.np.nan,
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:246: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'receptor_desc': pd.np.nan,
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:249: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'other_desc': pd.np.nan,
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:251: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'tags_reason': pd.np.nan,
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:252: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'tags_description': pd.np.nan,
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:253: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead
'pfam': pd.np.nan,
[ ][APP][26/06/21-12:57:36][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][APP][26/06/21-12:57:36][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
read local ensembl file
read local uniprot file
/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/core/generators/gene_generator.py:31: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
ensembl_db_filtered.dropna(inplace=True)
[ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][APP][26/06/21-12:57:52][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results
[ ][CORE][26/06/21-12:57:52][INFO] Initializing SqlAlchemy CellPhoneDB Core
[ ][CORE][26/06/21-12:57:52][INFO] Using custom database at /192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out/cellphonedb_user_2021-06-26-12_57.db
Traceback (most recent call last):
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/gaolin/anaconda3/envs/cpdb/bin/cellphonedb", line 8, in
sys.exit(cli())
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 722, in call
return self.main(args, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 535, in invoke
return callback(args, *kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), args, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/database_terminal_api_endpoints/database_terminal_commands.py", line 114, in generate
data_path=output_path)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/database/manager/DatabaseVersionManager.py", line 92, in collect_database
app.database_manager.database.create_all()
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/core/database/Database.py", line 25, in create_all
self.base_model.metadata.create_all(self.engine)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4665, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2095, in _run_visitor
conn._run_visitor(visitorcallable, element, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1656, in _run_visitor
visitorcallable(self.dialect, self, kwargs).traverse_single(element)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
return meth(obj, kw)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 787, in visit_metadata
_is_metadata_operation=True,
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
return meth(obj, kw)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 832, in visit_table
include_foreign_key_constraints, # noqa
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1073, in _execute_ddl
compiled,
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=excinfo[2], from=e
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked**
[SQL:
CREATE TABLE multidata_table (
id_multidata INTEGER NOT NULL,
name VARCHAR NOT NULL,
receptor BOOLEAN,
receptor_desc VARCHAR,
other BOOLEAN,
other_desc VARCHAR,
secreted_highlight BOOLEAN,
secreted_desc VARCHAR,
transmembrane BOOLEAN,
secreted BOOLEAN,
peripheral BOOLEAN,
integrin BOOLEAN,
is_complex BOOLEAN,
PRIMARY KEY (id_multidata),
UNIQUE (name),
CHECK (receptor IN (0, 1)),
CHECK (other IN (0, 1)),
CHECK (secreted_highlight IN (0, 1)),
CHECK (transmembrane IN (0, 1)),
CHECK (secreted IN (0, 1)),
CHECK (peripheral IN (0, 1)),
CHECK (integrin IN (0, 1)),
CHECK (is_complex IN (0, 1))
)
Hello !
I got an error ”sqlite3.OperationalError: database is locked‘, when running "cellphonedb database generate --user-interactions interaction_input.csv --user-interactions-only".
I tried to run it on python3.7 and 3.8 on two linux computers(Ubuntu 16 and 18), and installed different versions of cellphonedb (from v2.1.7 to v2.0.0), but it couldn’t solve this problem. It has bothered me for two days, Do you have any suggestions?
p.s.1 the interaction_input.csv is downloaded from https://github.com/Teichlab/cellphonedb-data/tree/master/data. p.s.2 The error seems to appear in “app.database_manager.database.create_all()” in the “collect_database” function.
############################################################################################
(cpdb) gaolin@gaolin://192.168.0.201/e/desktop/cellphonedb_example_data/example_data$ cellphonedb database generate --user-interactions interaction_input.csv -- user-interactions-only /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.cluster.kmeans module is deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.cluster. Anything that cannot be imported from sklearn.cluster is now part of the private API. warnings.warn(message, FutureWarning) read local uniprot file /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:243: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'secreted_desc': pd.np.nan, /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:246: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'receptor_desc': pd.np.nan, /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:249: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'other_desc': pd.np.nan, /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:251: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'tags_reason': pd.np.nan, /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:252: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'tags_description': pd.np.nan, /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/tools_terminal_api_endpoints/tools_terminal_commands.py:253: FutureWarning: The pandas.np module is deprecated and will be removed from pandas in a future version. Import numpy directly instead 'pfam': pd.np.nan, [ ][APP][26/06/21-12:57:36][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][APP][26/06/21-12:57:36][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results read local ensembl file read local uniprot file /home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/core/generators/gene_generator.py:31: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy ensembl_db_filtered.dropna(inplace=True) [ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][APP][26/06/21-12:57:51][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][APP][26/06/21-12:57:52][WARNING] Output directory (/192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out) exist and is not empty. Result can overwrite old results [ ][CORE][26/06/21-12:57:52][INFO] Initializing SqlAlchemy CellPhoneDB Core [ ][CORE][26/06/21-12:57:52][INFO] Using custom database at /192.168.0.201/e/desktop/cellphonedb_example_data/example_data/out/cellphonedb_user_2021-06-26-12_57.db Traceback (most recent call last): File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context cursor, statement, parameters, context File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute cursor.execute(statement, parameters) sqlite3.OperationalError: database is locked
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/gaolin/anaconda3/envs/cpdb/bin/cellphonedb", line 8, in
sys.exit(cli())
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 722, in call
return self.main(args, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, ctx.params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/core.py", line 535, in invoke
return callback(args, *kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), args, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/api_endpoints/terminal_api/database_terminal_api_endpoints/database_terminal_commands.py", line 114, in generate
data_path=output_path)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/database/manager/DatabaseVersionManager.py", line 92, in collect_database
app.database_manager.database.create_all()
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/cellphonedb/src/core/database/Database.py", line 25, in create_all
self.base_model.metadata.create_all(self.engine)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/schema.py", line 4665, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2095, in _run_visitor
conn._run_visitor(visitorcallable, element, kwargs)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1656, in _run_visitor
visitorcallable(self.dialect, self, kwargs).traverse_single(element)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
return meth(obj, kw)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 787, in visit_metadata
_is_metadata_operation=True,
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/visitors.py", line 145, in traverse_single
return meth(obj, kw)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 832, in visit_table
include_foreign_key_constraints, # noqa
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1073, in _execute_ddl
compiled,
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=excinfo[2], from=e
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/home/gaolin/anaconda3/envs/cpdb/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked**
[SQL:
CREATE TABLE multidata_table (
id_multidata INTEGER NOT NULL,
name VARCHAR NOT NULL,
receptor BOOLEAN,
receptor_desc VARCHAR,
other BOOLEAN,
other_desc VARCHAR,
secreted_highlight BOOLEAN,
secreted_desc VARCHAR,
transmembrane BOOLEAN,
secreted BOOLEAN,
peripheral BOOLEAN,
integrin BOOLEAN,
is_complex BOOLEAN,
PRIMARY KEY (id_multidata),
UNIQUE (name),
CHECK (receptor IN (0, 1)),
CHECK (other IN (0, 1)),
CHECK (secreted_highlight IN (0, 1)),
CHECK (transmembrane IN (0, 1)),
CHECK (secreted IN (0, 1)),
CHECK (peripheral IN (0, 1)),
CHECK (integrin IN (0, 1)),
CHECK (is_complex IN (0, 1))
)
] (Background on this error at: http://sqlalche.me/e/13/e3q8)