agrc / forklift

:tractor::package::sparkles: Slinging data all over the place :tractor::package::sparkles:
MIT License
28 stars 3 forks source link

fix: add support for data with global id fields that does not use change detection #370

Closed stdavis closed 5 months ago

stdavis commented 5 months ago

Description of Changes

This pull request fixes the following errors that have started showing up in the past few weeks:

image

It also fixes a bug where a crate that has data with a global id field would never be marked as updated.

Note: This is almost an exact revert of: 5c3c6b4 (#198)

This was before we had a truncate and load solution for datasets with global id fields.

Test results and coverage

---------- coverage: platform win32, python 3.11.8-final-0 -----------
Name                               Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------------
src\forklift\__init__.py               0      0      0      0   100%
src\forklift\arcgis.py               120     32     38      3    69%
src\forklift\change_detection.py      66      6     28      3    90%
src\forklift\config.py                56      1     28      2    96%
src\forklift\core.py                 253     17    134     14    91%
src\forklift\engine.py               549    205    220     20    60%
src\forklift\exceptions.py             3      0      0      0   100%
src\forklift\lift.py                 162     49     72      5    68%
src\forklift\messaging.py             98     43     34      3    52%
src\forklift\models.py               216     11     60      3    93%
src\forklift\seat.py                  19      0      6      0   100%
src\forklift\slack.py                216     35    106     33    76%
--------------------------------------------------------------------
TOTAL                               1758    399    726     86    74%
Coverage XML written to file cov.xml

=================================== 181 passed, 1 skipped, 19 warnings in 216.64s (0:03:36) ============
========================

Speed test results

Speed Test Results
Dry Run: 1.95 minutes
Repeat: 2.07 minutes