Chriszhangmw
commented
1 year ago after I kill the process related to sqlite, run again, but get another error:
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either:
- Avoid using
tokenizersbefore the fork if possible - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
Traceback (most recent call last):
File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/anaconda/envs/azureml_py38/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/paperetl/file/main.py", line 11, in
Execute.run( File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/paperetl/file/execute.py", line 176, in run db = Factory.create(url, replace) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/paperetl/factory.py", line 36, in create return SQLite(url.replace("sqlite://", ""), replace) File "/anaconda/envs/azureml_py38/lib/python3.8/site-packages/paperetl/sqlite.py", line 113, in init self.sindex = int(self.cur.execute(SQLite.SECTION_COUNT).fetchone()[0]) + 1 sqlite3.OperationalError: no such table: sections
davidmezzetti
the size is 0
!python -m paperetl.file paperetl/file/data paperetl/models
get the following error:
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks... To disable this warning, you can either:
tokenizersbefore the fork if possible