pganalyze / libpg_query

C library for accessing the PostgreSQL parser outside of the server environment
BSD 3-Clause "New" or "Revised" License
1.21k stars 182 forks source link

Add option to automatically try all parse modes #228

Open seanlinsley opened 11 months ago

seanlinsley commented 11 months ago

This adds a PG_QUERY_PARSE_ALL parse mode to automatically try every parse mode, which is helpful if you're parsing a query and don't know ahead of time which parse mode is valid for the query.

Additionally, this adds pg_query_normalize_opts which was missed by #216.

Note that the parse tree and fingerprints are different for some queries because multiple parse modes are able to parse the query. The NEW.name and NEW.author.first_name parse trees are actually incorrect so we may need to change the order in which the parse modes are tried.

# parse_opts
INVALID result for "v3.c1 := 4" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"v3","indirection":[{"String":{"sval":"c1"}}],"nnames":2,"val":{"targetList":[{"ResTarget":{"val":{"A_Const":{"ival":{"ival":4},"location":9}},"location":9}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"v3","indirection":[{"String":{"sval":"c1"}}],"nnames":1,"val":{"targetList":[{"ResTarget":{"val":{"A_Const":{"ival":{"ival":4},"location":9}},"location":9}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
INVALID result for "NEW.name = upper(cleanString(NEW.name))" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"new","indirection":[{"String":{"sval":"name"}}],"nnames":2,"val":{"targetList":[{"ResTarget":{"val":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}],"location":29}}],"funcformat":"COERCE_EXPLICIT_CALL","location":17}}],"funcformat":"COERCE_EXPLICIT_CALL","location":11}},"location":11}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Expr":{"kind":"AEXPR_OP","name":[{"String":{"sval":"="}}],"lexpr":{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}]}},"rexpr":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"name"}}],"location":29}}],"funcformat":"COERCE_EXPLICIT_CALL","location":17}}],"funcformat":"COERCE_EXPLICIT_CALL","location":11}},"location":9}}}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}
INVALID result for "NEW.author.first_name = upper(cleanString(NEW.author.first_name))" with 6 mode
expected: {"version":150001,"stmts":[{"stmt":{"PLAssignStmt":{"name":"new","indirection":[{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"nnames":3,"val":{"targetList":[{"ResTarget":{"val":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"location":42}}],"funcformat":"COERCE_EXPLICIT_CALL","location":30}}],"funcformat":"COERCE_EXPLICIT_CALL","location":24}},"location":24}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}}]}
  actual: {"version":150001,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget":{"val":{"A_Expr":{"kind":"AEXPR_OP","name":[{"String":{"sval":"="}}],"lexpr":{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}]}},"rexpr":{"FuncCall":{"funcname":[{"String":{"sval":"upper"}}],"args":[{"FuncCall":{"funcname":[{"String":{"sval":"cleanstring"}}],"args":[{"ColumnRef":{"fields":[{"String":{"sval":"new"}},{"String":{"sval":"author"}},{"String":{"sval":"first_name"}}],"location":42}}],"funcformat":"COERCE_EXPLICIT_CALL","location":30}}],"funcformat":"COERCE_EXPLICIT_CALL","location":24}},"location":22}}}}],"limitOption":"LIMIT_OPTION_DEFAULT","op":"SETOP_NONE"}}}]}

# fingerprint_opts
INVALID result for "v3.c1 := 4" with 6 mode
expected: "a8c86658ce26a653"
actual: "7f65202632663ba0"
actual tokens: INVALID result for "NEW.name = upper(cleanString(NEW.name))" with 6 mode
expected: "bb52450e4d46f7a1"
actual: "184cff1a84037010"
actual tokens: INVALID result for "NEW.author.first_name = upper(cleanString(NEW.author.first_name))" with 6 mode
expected: "a148e3f78b53c252"
actual: "b7de0b0d4f636a16"

Todo:

Per 1/9 meeting: