search

cebel / pyctd

PyCTD is a Python software package to query and analyse data from the CTD database
Apache License 2.0
11 stars 4 forks source link

Can't populate database #4

Closed cthoyt closed 7 years ago

cthoyt commented 7 years ago

I was unable to load the data from PyCTD and the error message is not helpful.

My config file:

[database]
sqlalchemy_connection_string = sqlite:////Users/cthoyt/.pyctd/data/pyctd.db

The code I ran, directly after cloning the repository from GitHub and pip3 install -e .:

>>> import pyctd
>>> pyctd.update()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cthoyt/dev/pyctd/src/pyctd/manager/database.py", line 396, in update
    DbManager(connection).db_import(urls, force_download)
  File "/Users/cthoyt/dev/pyctd/src/pyctd/manager/database.py", line 149, in db_import
    self.import_tables()
  File "/Users/cthoyt/dev/pyctd/src/pyctd/manager/database.py", line 200, in import_tables
    self.import_table(table)
  File "/Users/cthoyt/dev/pyctd/src/pyctd/manager/database.py", line 255, in import_table
    self.import_table_in_db(file_path, use_columns_with_index, column_names_in_db, table.name)
  File "/Users/cthoyt/dev/pyctd/src/pyctd/manager/database.py", line 323, in import_table_in_db
    for chunk in chunks:
  File "/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py", line 959, in __next__
    return self.get_chunk()
  File "/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py", line 1019, in get_chunk
    return self.read(nrows=size)
  File "/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py", line 982, in read
    ret = self._engine.read(nrows)
  File "/usr/local/lib/python3.6/site-packages/pandas/io/parsers.py", line 1719, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 890, in pandas._libs.parsers.TextReader.read (pandas/_libs/parsers.c:10862)
  File "pandas/_libs/parsers.pyx", line 924, in pandas._libs.parsers.TextReader._read_low_memory (pandas/_libs/parsers.c:11343)
  File "pandas/_libs/parsers.pyx", line 989, in pandas._libs.parsers.TextReader._read_rows (pandas/_libs/parsers.c:12175)
  File "pandas/_libs/parsers.pyx", line 1066, in pandas._libs.parsers.TextReader._convert_column_data (pandas/_libs/parsers.c:13286)
  File "pandas/_libs/parsers.pyx", line 1347, in pandas._libs.parsers.TextReader._get_column_name (pandas/_libs/parsers.c:18373)
IndexError: list index out of range
cebel commented 7 years ago

OK. Found the problem. CTD changed one table. I will change model and table_config.py. Should be fixed tomorrow.

cebel commented 7 years ago

Import in SQLite works now except 3 DtypeWarning warnings. Create a new issue.