Closed mesmith75 closed 3 days ago
Not reliably reproducible but here is a stack trace:
ERROR Exception in callback DiracBase.requeue_dirac_finished_jobs.<locals>.<lambda>(<Task cancell...Base.py:1302>>) at /cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py:1437 handle: <Handle DiracBase.requeue_dirac_finished_jobs.<locals>.<lambda>(<Task cancell...Base.py:1302>>) at /cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py:1437> Traceback (most recent call last): File "/cvmfs/ganga.cern.ch/Ganga/install/micromamba_el9/lib/python3.11/asyncio/events.py", line 80, in _run self._context.run(self._callback, *self._args) File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1437, in <lambda> task.add_done_callback(lambda task, job_id=j.id: DiracBase.job_exception_handler(job_id, task)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1406, in job_exception_handler exception = task.exception() ^^^^^^^^^^^^^^^^ File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1332, in job_finalisation await task File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1098, in _internal_job_finalisation await DiracBase.complete_dirac_job(job) File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1146, in complete_dirac_job await DiracBase._getStateTime(job, 'completing') File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Backends/DiracBase.py", line 1037, in _getStateTime be_statetime = await dm.execute(getStateTime, args_dict={ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Server/DiracProcessManager.py", line 107, in execute dirac_result = await self.get_task_result(task_id, task_results_dict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/cvmfs/ganga.cern.ch/Ganga/install/8.7.6/lib/python3.11/site-packages/ganga/GangaDirac/Lib/Server/DiracProcessManager.py", line 122, in get_task_result await asyncio.sleep(0.3) File "/cvmfs/ganga.cern.ch/Ganga/install/micromamba_el9/lib/python3.11/asyncio/tasks.py", line 649, in sleep return await future ^^^^^^^^^^^^ asyncio.exceptions.CancelledError
Not able to reproduce - looks like a problem of the user system
Not reliably reproducible but here is a stack trace: