tediousjs / tedious

Node TDS module for connecting to SQL Server databases.
http://tediousjs.github.io/tedious/
MIT License
1.56k stars 443 forks source link

Getting RangeError and Nodejs throws error for "bigint" type column if float values are provided in bulk load while using bulk insert #1610

Closed shivamrai45 closed 2 months ago

shivamrai45 commented 3 months ago

Software versions

Additional Libraries Used and Versions NA

Table schema Columns - MyBigInt (bigint 8)

Connection configuration { authentication: { type: 'default', options: { userName: '***', password: '***' } }, server: 'ms-sql-sql-server-2017.***.amazonaws.com', options: { database: 'MsSQL_Restart', encrypt: true, port: 1433, requestTimeout: 100000, trustServerCertificate: true } }

Problem description I am trying to bulk insert data in MsSQL using bulkLoad. The destination column contains bigint datatype column with size 8, If we provide it with a float/decimal value it crases the server while inserting in buffer here I've attached a sample script to replicate the behaviour tediousTest.txt

Expected behavior Ideally we should validate the data and return a error if invalid datatype is present in data or in case of float, round off to greatest minimum integer as we do in case of query run's for mssql.

Actual behavior Node internals throw error, thus crashing the server

Error message/stack trace RangeError: The number 0.5 cannot be converted to a BigInt because it is not an integer at BigInt () at WritableTrackingBuffer.writeInt64LE (node_modules/tedious/lib/tracking-buffer/writable-tracking-buffer.js:103:26) at Object.generateParameterData (node_modules/tedious/lib/data-types/bigint.js:33:12) at generateParameterData.next () at RowTransform._transform (node_modules/tedious/lib/bulk-load.js:126:18) at Transform._write (node:internal/streams/transform:175:8) at doWrite (node:internal/streams/writable:411:12) at clearBuffer (node:internal/streams/writable:572:7) at onwrite (node:internal/streams/writable:464:7) at node:internal/streams/transform:190:7 at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

Any other details that can be helpful

MichaelSun90 commented 3 months ago

Hi @shivamrai45 , we did some digging and instead of report this error within generateParameterData, we can catch it within the validate function and process it there. @arthurschreiber , is there any concern to add a line within the validate function BigInt(value) which will trigger this error, so we can catch error then either throw it or round the input float typed value to the nearest int?

shivamrai45 commented 3 months ago

Sure @MichaelSun90 , We can handle it in same way as it is being done for normal int data type, i.e. by using https://github.com/tediousjs/tedious/blob/7e84a2f54c6cb983b18a75ad1fc4bf274b2398e5/src/data-types/int.ts#L55. This way it acts similar to Math.trunc() and only considers the integer part of value. https://github.com/tediousjs/tedious/compare/master...shivamrai45:BigIntNodeError?expand=1

arthurschreiber commented 2 months ago

I guess it needs to be return value | 0n to keep the bigint type (not sure about that), but yeah, I agree that those should be handled the same way.

MichaelSun90 commented 2 months ago

We made a fix in # 1620. Will close this one for now. Feel free to reopen this if anything is needed related to this issue.