perseas / Pyrseas

Provides utilities for Postgres database schema versioning.
https://perseas.github.io/
BSD 3-Clause "New" or "Revised" License
395 stars 67 forks source link

yamltodb: incorrect create order of scripts with materialized view #171

Closed Sergey778 closed 7 years ago

Sergey778 commented 7 years ago

Problem

Pyrseas creates a materialized view before the tables on which it depends when using yamltodb.

Reproducing

Load following script in db using yamltodb:

extension plpgsql:
  description: PL/pgSQL procedural language
  schema: pg_catalog
  version: '1.0'
schema public:
  description: standard public schema
  materialized view concept_subsumption:
    definition: "SELECT test.name FROM test;"
    indexes:
      name_index:
        keys:
        - name
        unique: true
    with_data: false
  table test:
    columns:
    - name:
        not_null: true
        type: text
    primary_key:
      pk_test:
        columns:
        - name

Problem exists on master-branch.

jmafc commented 7 years ago

Did you create that YAML by hand or with dbtoyaml (in master)? If you look at the test tests/dbobject/test_matview.py MatViewToSqlTestCase test_create_view() you'll notice that the materialized view has an additional attribute: depends_on. This is part of the dependency tracking changes implemented in master (but not yet fully documented). If I add the lines:

    depends_on:
      - table test

after the definition line, yamltodb creates the table first.

Sergey778 commented 7 years ago

This example I created by hands, but in my main project dbtoyaml generated script without depends_on attribute.

jmafc commented 7 years ago

Did you use the dbtoyaml from master in your main project? If you were previously using an installed version, please double check (if you run dbtoyaml --version, it should show 0.8.dev0, whereas the one installed via pip would show 0.7.3 or similar). In any case, after running the corrected YAML file into a database, if I run dbtoyaml on it, both Python 2 and 3 show the depends_on attribute:

Python 2:

  materialized view concept_subsumption:
    definition: " SELECT test.name\n   FROM test;"
    depends_on:
    - table test

Python 3:

  materialized view concept_subsumption:
    definition: |2-
       SELECT test.name
         FROM test;
    depends_on:
    - table test
Sergey778 commented 7 years ago

Oh, I don't know how this happened but I have installed old dbtoyaml and new yamltodb. So, yes, with master there is no problem. Sorry and thanks for help :)