get_detailed_job_record_count() very slow

MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform

MIT License

26 stars 11 forks source link

get_detailed_job_record_count() very slow #376

Closed ghukill closed 5 years ago

ghukill commented 5 years ago

The Job method get_detailed_job_record_count() is designed to run only when the records of a Job are altered, and stores this information in job_details:

self.update_job_details({'detailed_record_count':r_count_dict})

This was to prevent recalculating this every time the Job is viewed. However, in a situation where this value is not pre-calculated, it can be very slow to count when there are 1-2m+ records in Mongo.

ghukill commented 5 years ago

Unsure what caused this, cannot recreate, even when detailed_record_count is removed from same Job, requiring a re-calculation. Closing.