perseas / Pyrseas

Provides utilities for Postgres database schema versioning.
https://perseas.github.io/
BSD 3-Clause "New" or "Revised" License
395 stars 67 forks source link

KeyError on Trigger Procedure with arguments #219

Closed satsva closed 3 years ago

satsva commented 4 years ago

First time user of Pyrseas and I am getting the below KeyError. All my Tables have Triggers which keeps an audit log of data changes. If I remove the arguments for the trigger procedure dbtoyaml works fine. Postgres allows passing arguments to trigger function so why is dbtoyaml failing?

dbtoyaml version 0.9.1

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/bin/dbtoyaml", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbtoyaml.py", line 49, in main
    dbmap = db.to_map()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/database.py", line 488, in to_map
    dbmap.update(self.db.schemas.to_map(self.db, opts))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/schema.py", line 357, in to_map
    schemas.update(self[sch].to_map(db, self, opts))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/schema.py", line 104, in to_map
    schobjs.append((obj, obj.to_map(db, dbschemas, opts)))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/table.py", line 556, in to_map
    dct['triggers'].update(self.triggers[k.name].to_map(db))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/trigger.py", line 156, in to_map
    dct = super(Trigger, self).to_map(db)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/__init__.py", line 361, in to_map
    deps -= self.get_implied_deps(db)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pyrseas/dbobject/trigger.py", line 239, in get_implied_deps
    deps.add(db.functions[fschema, fname, self.arguments or ''])
KeyError: ('audit', 'if_modified_func', "'true', 'pid', '{NULL}'")
jmafc commented 4 years ago

If you look at the line just before the KeyError, you'll see the third key to db.functions is self.arguments or '' and in the next line self.arguments is shown as "'true', 'pid', '{NULL}'". So it seems the third argument to your audit.if_modified_func trigger functions is (mis)interpreted as having the value {NULL}. Aside from perhaps providing a simple example that exhibits the problem, I would recommend that you examine the arguments of your problem function. First you can issue the query that is in lines 94-113, and pay special attention to the column named arguments, which comes from encode(tgargs, 'escape') AS arguments (line 104). Then you can put some debugging print()s before the if arguments ... in line 54 and after the assignment to self.arguments in lines 55-56.

satsva commented 4 years ago

Thank you for taking time to review and pointers. After much troubleshooting I was able to resolve the issue. Issue is the User I am using to connect to the database doesn't have search_path set to the schema where I have the trigger procedure i.e., Tables are in schema X and the audit procedure is in Schema Y (not real schema names of course). Once I set the search_path to both schemas everything went well.

However, wondering why it worked if I removed all the arguments?

jmafc commented 4 years ago

It worked when you removed all the arguments because if there are no arguments, self.arguments is set to None (in Trigger.__init__) and then in the line that I showed above the third key used to access db.functions is set to an empty string and that is present in that dict. This issue is similar to #99. In general, Pyrseas uses queries directly against the catalogs, which are generally visible to any user. However, in some places we use functions like the encode() mentioned above and those aren't as visible to anyone.