search

pacman82 / arrow-odbc

Fill Apache Arrow record batches from an ODBC data source in Rust.
MIT License
52 stars 10 forks source link

Optimize parsing of Decimal(x, 0) #73

Open jonashaag opened 1 year ago

jonashaag commented 1 year ago

I think we can make a (small?) optimization for parsing decimals with scale=0. Snowflake will send any integers as Decimal(38, 0) by default. At the very least I guess we can skip removing the . char but maybe we can also just use one of the integer parsers.

pacman82 commented 1 year ago

Hello @jonashaag ,

yeah, this is possible. Personally I have been experimenting with fetching decimals into a numeric struct rather than text. See:

https://learn.microsoft.com/en-us/sql/odbc/reference/appendixes/retrieve-numeric-data-sql-numeric-struct-kb222831?view=sql-server-ver16

However, I only got it to work for SQLGetData reliabliy. Which can be only used for unbuffered row by row access. According to the documentation it should work with SQLBindCol as well. Yet so far, for me, it just does not. I'll try some stuff, before making a special case for DECIMAL(38,0) and also look into some improvements for atoi, which sadly would require breaking the interface.