blue-yonder / turbodbc

Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with the Python Database API Specification 2.0.
http://turbodbc.readthedocs.io/en/latest
MIT License
607 stars 85 forks source link

Fix slowdown from utf conversion #400

Closed SimeonStoykovQC closed 6 months ago

SimeonStoykovQC commented 7 months ago

Motivation

4.5.8 (or 4.5.7) introduced a different utf16<->utf8 conversion (using wstring_convert instead of boost's conversion library), which considerably slows down the reading of string data - I observed 10x slowdown with 1m rows of characters.

Apart from the performance improvement, the changes are warranted because wstring_convert was deprecated in C++17.

Changes

xhochy commented 7 months ago

@SimeonStoykovQC Can you rebase/merge with main?

codecov-commenter commented 6 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (10f7133) 98.81% compared to head (6e1e5a2) 98.86%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #400 +/- ## ========================================== + Coverage 98.81% 98.86% +0.05% ========================================== Files 94 10 -84 Lines 3030 352 -2678 ========================================== - Hits 2994 348 -2646 + Misses 36 4 -32 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

xhochy commented 6 months ago

Please add a changelog entry.