MSSQL Date checksum failing due to overflow

[ojgregory-uoe]

Issue Description

• Description of the issue: Hello,

I'm loading a csv into SQL server and as it's under 10000 rows sling is performing a checksum compare, this is erroring due to the data having this date in some of the fields 01/01/1900 00:00. This causes an overflow from the DATEDIFF function due to the large difference between the 2 compared values as it returns an INT not a BIGINT. Using DATEDIFF_BIG avoids this problem when I have replaced all the DATEDIFF calls in the generated SQL in my testing, though that is a function introduced in SQL Server 2016 so may not be the perfect solution.

This is the error returned when executing the generated SQL directly

The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.

• Sling version (sling --version): Version: 1.2.6
• Operating System (linux, mac, windows): windows
• Replication Configuration:

source: 
  'local://' 
target: 
  MY_SQLSERVER
defaults: 
  mode: full-refresh 
  object: 'dbo.{stream_file_name}' 
  source_options:
    format: csv
streams: 
  "file://./user.csv":
    object: 'dbo.users'
    skip_blank_lines: true
env: 
  SLING_LOADED_AT_COLUMN: 'true' 
  SAMPLE_SIZE: 20000
  SLING_KEEP_TEMP: 'true'

• Log Output (please run command with -d):

  
  fatal:
  --- sling_cli.go:418 func1 ---
  --- sling_cli.go:476 cliInit ---
  --- cli.go:284 CliProcess ---
  ~ failure running replication (see docs @ https://docs.slingdata.io/sling-cli)
  --- sling_logic.go:202 processRun ---

--- task_run.go:97 func1 --- --- task_run.go:387 runFileToDB --- --- task_run_write.go:366 WriteToDb --- --- database.go:2973 CompareChecksums --- --- panic.go:114 goPanicIndex --- --- panic.go:914 gopanic --- ~ execution failed --- task_run.go:75 1 --- panic occurred! runtime.boundsError{x:0, y:0, signed:true, code:0x0} goroutine 43 [running]: runtime/debug.Stack() C:/hostedtoolcache/windows/go/1.21.9/x64/src/runtime/debug/stack.go:24 +0x5e github.com/slingdata-io/sling-cli/core/sling.(TaskExecution).Execute.func1.1() D:/a/sling-cli/sling-cli/core/sling/task_run.go:75 +0x3d panic({0x7ff6edc99780?, 0xc000058f18?}) C:/hostedtoolcache/windows/go/1.21.9/x64/src/runtime/panic.go:914 +0x21f github.com/slingdata-io/sling-cli/core/dbio/database.(BaseConn).CompareChecksums(0xc000f361e0, {0xc000058858, 0x11}, {0xc002300000?, 0x5f, 0x9d}) D:/a/sling-cli/sling-cli/core/dbio/database/database.go:2973 +0x17af github.com/slingdata-io/sling-cli/core/sling.(TaskExecution).WriteToDb(0xc000f28160, 0xc0014a0780, 0xc000e805b0, {0x7ff6ee8f5960?, 0xc000f361e0}) D:/a/sling-cli/sling-cli/core/sling/task_run_write.go:366 +0x40c6 github.com/slingdata-io/sling-cli/core/sling.(TaskExecution).runFileToDB(0xc000f28160) D:/a/sling-cli/sling-cli/core/sling/task_run.go:387 +0xc7d github.com/slingdata-io/sling-cli/core/sling.(TaskExecution).Execute.func1() D:/a/sling-cli/sling-cli/core/sling/task_run.go:97 +0x51f created by github.com/slingdata-io/sling-cli/core/sling.(TaskExecution).Execute in goroutine 67 D:/a/sling-cli/sling-cli/core/sling/task_run.go:68 +0x393 D:/a/sling-cli/sling-cli/core/sling/task_run.go:68 +0x393

[flarco]

Thanks for reporting, fixed in next release. https://github.com/slingdata-io/sling-cli/pull/274

[ojgregory-uoe]

Amazing, thanks for looking at it so quickly