When a lot of jobs are in the condor pool (> 50000), condor_history doesn't report the status of jobs that got completed earlier. The history log truncates around 75000 or there's a possibility of log corruption with condor_history returning an error saying the history file is not found.
Need to use a better approach in EMS. Once the completed jobs are found, don't query for them again. This also saves on a lot of CPU time from condor_history.
When a lot of jobs are in the condor pool (> 50000), condor_history doesn't report the status of jobs that got completed earlier. The history log truncates around 75000 or there's a possibility of log corruption with condor_history returning an error saying the history file is not found.
Need to use a better approach in EMS. Once the completed jobs are found, don't query for them again. This also saves on a lot of CPU time from condor_history.