andialbrecht / sqlparse

A non-validating SQL parser module for Python
BSD 3-Clause "New" or "Revised" License
3.74k stars 697 forks source link

add an option for no string escape character #683

Open mtagle opened 2 years ago

mtagle commented 2 years ago

Right now, sqlparse treats \ as an escape character in strings, but this is not universal across all sql dialects. postgres and athena/presto both do not consider \ as string escape character, and, in both, the following is considered valid sql:

select '\' as str

however, sqlparse will not parse this correctly and attempting to format any sql that has a \' somewhere in it could result in some incorrectly formatted sql (and also possibly sqlparse.format making your sql now execute differently).

Here's some weird/incorrect stuff that can happen because of this:

import sqlparse

sql = "select '\\' as bs, '/' as fs, '>' as gt"
print sql
formatted_sql = sqlparse.format(sql, keyword_case="upper", use_space_around_operators=True)
print formatted_sql

will print:

select '\' as bs, '/' as fs, '>' as gt
SELECT '\' as bs, ' / ' as fs, ' > ' AS gt

If there was a parsing option to have \ be an escape character or not I believe that would solve this issue.

mrmasterplan commented 1 year ago

This needs to be addressed at the level of the lexer reg-exps. This is probably best handled when we have a dialect selection in place (which we will do at some point)