After re-studying data from CEAP we decided it was unnecessary to keep historical changes in the reimbursement data; forthcoming changes in serenata-toolbox will target reimbursements we were not targeting so far and the need for historical tracking will be deprecated;
Loading reimbursement in production was taking up to a week, and that was far too much.
This PR intends to:
Speed-up considerably (from few days to a few minutes) the reimbursement import;
Remove the need for historcal tracking.
What was done to achieve this purpose?
Removing tools that helped us to keep historical changes (django-simple-history and the field available_in_latest_dataset from Reimbursement model);
And by using bulk_create on import (which we had dismissed in order to get django-simple-history working).
How to test if it really works?
Important: to allow this branch to be deployed this PR is based on #314 because that is the branch currently used for production. If you prefer to code review things in a baby step strategy it is strongly recommended to start by #314. Otherwise, review and eventually merge this and #314 will be automatically merged.
In order to re-build container locally, temporally edit docker-compose.yml:
What is the purpose of this Pull Request?
Given that:
serenata-toolbox
will target reimbursements we were not targeting so far and the need for historical tracking will be deprecated;This PR intends to:
What was done to achieve this purpose?
django-simple-history
and the fieldavailable_in_latest_dataset
fromReimbursement
model);bulk_create
on import (which we had dismissed in order to getdjango-simple-history
working).How to test if it really works?
Important: to allow this branch to be deployed this PR is based on #314 because that is the branch currently used for production. If you prefer to code review things in a baby step strategy it is strongly recommended to start by #314. Otherwise, review and eventually merge this and #314 will be automatically merged.
In order to re-build container locally, temporally edit
docker-compose.yml
:python manage.py migrate
python manage.py shell_plus
and thenReimbursement.objects.all().delete()
;Don't forget to undo changes in
docker-compose.yml
; )Who can help reviewing it?
@anaschwendler @irio @jtemporal