search

keboola / db-extractor-mssql

MIT License
1 stars 2 forks source link

Only apply sorting if there is a limit #125

Closed pivnicek closed 5 years ago

pivnicek commented 5 years ago

If there's no limit applied on the incremental fetching there is no reason to sort the results.

So https://github.com/keboola/db-extractor-mssql/blob/master/src/Keboola/DbExtractor/Extractor/MSSQL.php#L812 should only be applied if limit != 0

tomasfejfar commented 5 years ago

We still need to sort the result to know the latest value we fetched :( Otherwise we'd need to sort the (potentially huge) CSV we dumped...

https://travis-ci.com/keboola/db-extractor-mssql/builds/118597841#L2101

tomasfejfar commented 5 years ago

So we need to do a separate "max" query to fetch the lastest value. We don't really need to worry about the fact that some new items may have been added in the meantime as those will be removed in the deduplication phase of storage import.