Currently, sqoop is set up to grab the most recent two tax years from each table in the iasWorld backend. This works reasonably well as it's exceedingly rare for historical records to be updated. However, with the introduction of the new omitted assessment classes (OA[0-9]), prior year record have indeed received some updates. See 16071290130000 for an example PIN.
We should work out a strategy to grab and process these prior year record updates without having to grab every row of the year.
dfsnow
commented
2 months ago
Currently, sqoop is set up to grab the most recent two tax years from each table in the iasWorld backend. This works reasonably well as it's exceedingly rare for historical records to be updated. However, with the introduction of the new omitted assessment classes (
OA[0-9]), prior year record have indeed received some updates. See16071290130000for an example PIN.We should work out a strategy to grab and process these prior year record updates without having to grab every row of the year.