andialbrecht / sqlparse

A non-validating SQL parser module for Python
BSD 3-Clause "New" or "Revised" License
3.71k stars 693 forks source link

Optimize sqlparse.utils.imt(). #766

Closed adamchainz closed 6 months ago

adamchainz commented 6 months ago

Thanks for contributing!

Before submitting your pull request please have a look at the following checklist:

Profiling a sqlparse.parse() on a bunch of large SELECT queries, 14ms out of 59ms, or ~24% of the time, was spent in sqlparse.utils.imt().

Looking at that function, it did some moving of the m and t parameters into lists, wasting work when they’re not set or a single element which can use a faster check. To mitigate this, I optimized it to do the check only when the parameters are set, and avoid the listification of single elements.

This doubled its speed, saving 7ms in the same profile.

codecov[bot] commented 6 months ago

Codecov Report

Attention: Patch coverage is 85.71429% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 96.92%. Comparing base (60486b9) to head (e7eab4c).

Files Patch % Lines
sqlparse/utils.py 85.71% 2 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #766 +/- ## ========================================== - Coverage 97.04% 96.92% -0.12% ========================================== Files 20 20 Lines 1558 1563 +5 ========================================== + Hits 1512 1515 +3 - Misses 46 48 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

andialbrecht commented 6 months ago

Thanks for the patch! That makes the code much more readable as well.