search

atviriduomenys / spinta

Spinta is a framework to describe, extract and publish data (a DEP Framework).
MIT License
10 stars 4 forks source link

Error with unique constraints #500

Open sirex opened 11 months ago

sirex commented 11 months ago

Unique constraints are created with non existing table column names, this effects only ref type properties.

Traceback (most recent call last):
  File "spinta/asgi.py", line 29, in <module>
    commands.prepare(context, store.manifest)
  File "spinta/manifests/commands/init.py", line 14, in prepare
    commands.prepare(context, backend, manifest)
  File "spinta/backends/postgresql/commands/init.py", line 24, in prepare
    commands.prepare(context, backend, model)
  File "spinta/backends/postgresql/commands/init.py", line 53, in prepare
    main_table = sa.Table(
  File "sqlalchemy/sql/base.py", line 1156, in __getitem__
    return self._index[key]
KeyError: 'aob_kodas.aob_kodas'

This specific error was raised for https://github.com/atviriduomenys/manifest/blob/55e8944da136318c2ca8389bb55108db016957ce/datasets/gov/rc/ar/pastatas.csv manifest file.

More details:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ spinta/backends/postgresql/commands/init.py:53 in prepare                                        │                                                      │
│                                                                                                  │
│    50 │   # Create main table.                                                                   │
│    51 │   main_table_name = get_pg_name(get_table_name(model))                                   │
│    52 │   pkey_type = commands.get_primary_key_type(context, backend)                            │
│ ❱  53 │   main_table = sa.Table(                                                                 │
│    54 │   │   main_table_name, backend.schema,                                                   │
│    55 │   │   sa.Column('_txn', pkey_type, index=True),                                          │
│    56 │   │   sa.Column('_created', sa.DateTime),                                                │
│                                                                                                  │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │         backend = <spinta.backends.postgresql.components.PostgreSQL(name='default') at       │ │
│ │                   0x7f9b323ac4f0>                                                            │ │
│ │          column = Column('aob_nuo', Date(), table=<datasets/gov/rc/ar/pastatas/Pastatas>)    │ │
│ │ main_table_name = 'datasets/gov/rc/ar/pastatas/Pastatas'                                     │ │
│ │           model = <spinta.components.Model(name='datasets/gov/rc/ar/pastatas/Pastatas')>     │ │
│ │            name = 'aob_kodas.aob_kodas'                                                      │ │
│ │       pkey_type = UUID()                                                                     │ │
│ │            prop = <spinta.components.Property(name='aob_kodas', type='ref',                  │ │
│ │                   model='datasets/gov/rc/ar/pastatas/Pastatas')>                             │ │
│ │       prop_list = ['aob_kodas.aob_kodas']                                                    │ │
│ │             val = [                                                                          │ │
│ │                   │   <spinta.components.Property(name='aob_kodas', type='ref',              │ │
│ │                   model='datasets/gov/rc/ar/pastatas/Pastatas')>                             │ │
│ │                   ]                                                                          │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
sirex commented 11 months ago

A hot fix was added via https://github.com/atviriduomenys/spinta/pull/501, bet a test is needed and probably better solution, for example, it is not a good idea to have multiple places, where column name for ref type is hardcoded in multiple places, instead a function should be created and whole naming logic for ref columns should be kept in one place, to avoid similar errors.