search

slingdata-io / sling-cli

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
https://docs.slingdata.io
GNU General Public License v3.0
299 stars 16 forks source link

Sling Incremental query using Timestamp with 3 decimals only breaks incremental #307

Closed jmarciloamo closed 1 month ago

jmarciloamo commented 1 month ago

Issue Description

I am using Sling Data to load incrementally some tables into another table in another database in PostgreSQL. The issue is that the store data in my final table is 2024-05-09 13:28:27.031153+00, but when Sling issue the incremental query it strips some of the last digits and that makes the query request data > 2024-05-09 13:28:27.031 which result in getting the same data I've got on the last run because 2024-05-09 13:28:27.031 is smaller than 2024-05-09 13:28:27.031153+00

source: <source>
target: <target>
streams:
  ...
Paste log here.
flarco commented 1 month ago

Thanks for reporting, I just pushed changes here to increase the decimal precision. Feel free to compile the binary yourself on branch v1.2.11 while waiting for next release.