search

atviriduomenys / spinta

Spinta is a framework to describe, extract and publish data (a DEP Framework).
MIT License
11 stars 4 forks source link

When trying to inspect `sqlite` database which has `sequence` table, get an error TypeError: Unknown type NullType(). #628

Open karina-klinkeviciute opened 4 months ago

karina-klinkeviciute commented 4 months ago

If some column in sqlite database is marked as autoincrement, sqlite creates a special table for it, called sqlite_sequence.

Columns of this table do not have types. So when trying to inspect such a database, the type of this column isn't recognized and we get an error.

Special tables in sqlite should be taken in account and ignored.

karina-klinkeviciute commented 4 months ago

traceback:

  File "spinta/cli/inspect.py", line 58, in inspect
    context, manifest = create_manifest_from_inspect(
  File "spinta/datasets/inspect/helpers.py", line 81, in create_manifest_from_inspect
    _merge(context, manifest, manifest, resource, has_manifest_priority, dataset)
  File "spinta/datasets/inspect/helpers.py", line 103, in _merge
    store = load_manifest(context, full_load=True)
  File "spinta/cli/helpers/store.py", line 120, in load_manifest
    commands.load(
  File "spinta/manifests/yaml/commands/load.py", line 110, in load
    commands.load(
  File "spinta/manifests/sql/commands/load.py", line 44, in load
    load_manifest_nodes(context, into, schemas, source=manifest)
  File "spinta/manifests/helpers.py", line 133, in load_manifest_nodes
    for eid, schema in schemas:
  File "spinta/manifests/sql/helpers.py", line 107, in read_schema
    'properties': dict(_read_props(insp, table, schema, mapping_data["mapping"])),
  File "spinta/manifests/sql/helpers.py", line 199, in _read_props
    dtype = _get_type(col['type'])
  File "spinta/manifests/sql/helpers.py", line 246, in _get_type
    raise TypeError(f"Unknown type {sql_type!r}.")
TypeError: Unknown type NullType().