search

mitodl / micromasters

Portal for learners and course teams to access MITx Micromasters® programs
https://mm.mit.edu
BSD 3-Clause "New" or "Revised" License
28 stars 16 forks source link

chore(deps): update dependency sqlparse to v0.5.0 [security] #5356

Open renovate[bot] opened 8 months ago

renovate[bot] commented 8 months ago

Mend Renovate

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
sqlparse (changelog) ==0.3.0 -> ==0.5.0 age adoption passing confidence

GitHub Vulnerability Alerts

CVE-2023-30608

Impact

The SQL parser contains a regular expression that is vulnerable to ReDoS (Regular Expression Denial of Service). The vulnerability may lead to Denial of Service (DoS).

Patches

This issues has been fixed in sqlparse 0.4.4.

Workarounds

None.

References

This issue was discovered and reported by GHSL team member @​erik-krogh (Erik Krogh Kristensen).

CVE-2024-4340

Summary

Passing a heavily nested list to sqlparse.parse() leads to a Denial of Service due to RecursionError.

Details + PoC

Running the following code will raise Maximum recursion limit exceeded exception:

import sqlparse
sqlparse.parse('[' * 10000 + ']' * 10000)

We expect a traceback of RecursionError:

Traceback (most recent call last):
  File "trigger_sqlparse_nested_list.py", line 3, in <module>
    sqlparse.parse('[' * 10000 + ']' * 10000)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/__init__.py", line 30, in parse
    return tuple(parsestream(sql, encoding))
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/filter_stack.py", line 36, in run
    stmt = grouping.group(stmt)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 428, in group
    func(stmt)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 53, in group_brackets
    _group_matching(tlist, sql.SquareBrackets)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/engine/grouping.py", line 48, in _group_matching
    tlist.group_tokens(cls, open_idx, close_idx)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 328, in group_tokens
    grp = grp_cls(subtokens)
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 161, in __init__
    super().__init__(None, str(self))
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 165, in __str__
    return ''.join(token.value for token in self.flatten())
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 165, in <genexpr>
    return ''.join(token.value for token in self.flatten())
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  File "/home/uriya/.local/lib/python3.10/site-packages/sqlparse/sql.py", line 214, in flatten
    yield from token.flatten()
  [Previous line repeated 983 more times]
RecursionError: maximum recursion depth exceeded

Fix suggestion

The flatten() function of TokenList class should limit the recursion to a maximal depth:

from sqlparse.exceptions import SQLParseError

MAX_DEPTH = 100

    def flatten(self, depth=1):
        """Generator yielding ungrouped tokens.

        This method is recursively called for all child tokens.
        """
    if depth >= MAX_DEPTH:
        raise SQLParseError('Maximal depth reached')
        for token in self.tokens:
            if token.is_group:
                yield from token.flatten(depth + 1)
            else:
                yield token

Impact

Denial of Service (the impact depends on the use). Anyone parsing a user input with sqlparse.parse() is affected.

Release Notes

andialbrecht/sqlparse (sqlparse) ### [`v0.5.0`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-050-Apr-13-2024) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.4.4...0.5.0) Notable Changes - Drop support for Python 3.5, 3.6, and 3.7. - Python 3.12 is now supported (pr725, by hugovk). - IMPORTANT: Fixes a potential denial of service attack (DOS) due to recursion error for deeply nested statements. Instead of recursion error a generic SQLParseError is raised. See the security advisory for details: https://github.com/andialbrecht/sqlparse/security/advisories/GHSA-2m57-hf25-phgg The vulnerability was discovered by [@​uriyay-jfrog](https://togithub.com/uriyay-jfrog). Thanks for reporting! Enhancements: - Splitting statements now allows to remove the semicolon at the end. Some database backends love statements without semicolon (issue742). - Support TypedLiterals in get_parameters (pr649, by Khrol). - Improve splitting of Transact SQL when using GO keyword (issue762). - Support for some JSON operators (issue682). - Improve formatting of statements containing JSON operators (issue542). - Support for BigQuery and Snowflake keywords (pr699, by griffatrasgo). - Support parsing of OVER clause (issue701, pr768 by r33s3n6). Bug Fixes - Ignore dunder attributes when creating Tokens (issue672). - Allow operators to precede dollar-quoted strings (issue763). - Fix parsing of nested order clauses (issue745, pr746 by john-bodley). - Thread-safe initialization of Lexer class (issue730). - Classify TRUNCATE as DDL and GRANT/REVOKE as DCL keywords (based on pr719 by josuc1, thanks for bringing this up!). - Fix parsing of PRIMARY KEY (issue740). Other - Optimize performance of matching function (pr799, by admachainz). ### [`v0.4.4`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-044-Apr-18-2023) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.4.3...0.4.4) Notable Changes - IMPORTANT: This release fixes a security vulnerability in the parser where a regular expression vulnerable to ReDOS (Regular Expression Denial of Service) was used. See the security advisory for details: https://github.com/andialbrecht/sqlparse/security/advisories/GHSA-rrm6-wvj7-cwh2 The vulnerability was discovered by [@​erik-krogh](https://togithub.com/erik-krogh) from GitHub Security Lab (GHSL). Thanks for reporting! Bug Fixes - Revert a change from 0.4.0 that changed IN to be a comparison (issue694). The primary expectation is that IN is treated as a keyword and not as a comparison operator. That also follows the definition of reserved keywords for the major SQL syntax definitions. - Fix regular expressions for string parsing. Other - sqlparse now uses pyproject.toml instead of setup.cfg (issue685). ### [`v0.4.3`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-043-Sep-23-2022) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.4.2...0.4.3) Enhancements - Add support for DIV operator (pr664, by chezou). - Add support for additional SPARK keywords (pr643, by mrmasterplan). - Avoid tokens copy (pr622, by living180). - Add REGEXP as a comparision (pr647, by PeterSandwich). - Add DISTINCTROW keyword for MS Access (issue677). - Improve parsing of CREATE TABLE AS SELECT (pr662, by chezou). Bug Fixes - Fix spelling of INDICATOR keyword (pr653, by ptld). - Fix formatting error in EXTRACT function (issue562, issue670, pr676, by ecederstrand). - Fix bad parsing of create table statements that use lower case (issue217, pr642, by mrmasterplan). - Handle backtick as valid quote char (issue628, pr629, by codenamelxl). - Allow any unicode character as valid identifier name (issue641). Other - Update github actions to test on Python 3.10 as well (pr661, by cclaus). ### [`v0.4.2`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-042-Sep-10-2021) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.4.1...0.4.2) Notable Changes - IMPORTANT: This release fixes a security vulnerability in the strip comments filter. In this filter a regular expression that was vulnerable to ReDOS (Regular Expression Denial of Service) was used. See the security advisory for details: https://github.com/andialbrecht/sqlparse/security/advisories/GHSA-p5w8-wqhj-9hhf The vulnerability was discovered by [@​erik-krogh](https://togithub.com/erik-krogh) and [@​yoff](https://togithub.com/yoff) from GitHub Security Lab (GHSL). Thanks for reporting! Enhancements - Add ELSIF as keyword (issue584). - Add CONFLICT and ON_ERROR_STOP keywords (pr595, by j-martin). Bug Fixes - Fix parsing of backticks (issue588). - Fix parsing of scientific number (issue399). ### [`v0.4.1`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-041-Oct-08-2020) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.4.0...0.4.1) Bug Fixes - Just removed a debug print statement, sorry... ### [`v0.4.0`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-040-Oct-07-2020) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.3.1...0.4.0) Notable Changes - Remove support for end-of-life Python 2.7 and 3.4. Python 3.5+ is now required. - Remaining strings that only consist of whitespaces are not treated as statements anymore. Code that ignored the last element from sqlparse.split() should be updated accordingly since that function now doesn't return an empty string as the last element in some cases (issue496). Enhancements - Add WINDOW keyword (pr579 by ali-tny). - Add RLIKE keyword (pr582 by wjones1). Bug Fixes - Improved parsing of IN(...) statements (issue566, pr567 by hurcy). - Preserve line breaks when removing comments (issue484). - Fix parsing error when using square bracket notation (issue583). - Fix splitting when using DECLARE ... HANDLER (issue581). - Fix splitting of statements using CASE ... WHEN (issue580). - Improve formatting of type casts in parentheses. - Stabilize formatting of invalid SQL statements. ### [`v0.3.1`](https://togithub.com/andialbrecht/sqlparse/blob/HEAD/CHANGELOG#Release-031-Feb-29-2020) [Compare Source](https://togithub.com/andialbrecht/sqlparse/compare/0.3.0...0.3.1) Enhancements - Add HQL keywords (pr475, by matwalk). - Add support for time zone casts (issue489). - Enhance formatting of AS keyword (issue507, by john-bodley). - Stabilize grouping engine when parsing invalid SQL statements. Bug Fixes - Fix splitting of SQL with multiple statements inside parentheses (issue485, pr486 by win39). - Correctly identify NULLS FIRST / NULLS LAST as keywords (issue487). - Fix splitting of SQL statements that contain dollar signs in identifiers (issue491). - Remove support for parsing double slash comments introduced in 0.3.0 (issue456) as it had some side-effects with other dialects and doesn't seem to be widely used (issue476). - Restrict detection of alias names to objects that actually could have an alias (issue455, adopted some parts of pr509 by john-bodley). - Fix parsing of date/time literals (issue438, by vashek). - Fix initialization of TokenList (issue499, pr505 by john-bodley). - Fix parsing of LIKE (issue493, pr525 by dbczumar). - Improve parsing of identifiers (pr527 by liulk).

Configuration

📅 Schedule: Branch creation - "" in timezone US/Eastern, Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

This PR has been generated by Mend Renovate. View repository job log here.