We've occasionally had trouble with the employee import memory usage already, now with the rebuilding of the love count it takes over twice as much memory as before. This PR hopes to address that:
It fetches DB employess only once instead of twice. Additionally it removes the fetching of each individual employee from the DB as we iterate through the S3 dataset, which should make it quite a bit faster.
It moves rebuilding the love count out of the _update_employees function so the memory of the data structures used there can be freed.
Let's see if it helps. I ran it locally and it worked, but I don't know how much memory it consumed.
We've occasionally had trouble with the employee import memory usage already, now with the rebuilding of the love count it takes over twice as much memory as before. This PR hopes to address that:
_update_employeesfunction so the memory of the data structures used there can be freed.Let's see if it helps. I ran it locally and it worked, but I don't know how much memory it consumed.