Open HerrHorizontal opened 3 years ago
Would you please clarify what you mean with modifying the PRE_JOB_RANK
during negotiation process? Each negotiator has a predefined PRE_JOB_RANK
, that cannot be modified during the process of negotiation.
Following the description in https://htcondor.readthedocs.io/en/latest/admin-manual/configuration-macros.html?highlight=NEGOTIATOR_PRE_JOB_RANK#condor-negotiator-configuration-file-entries and https://htcondor.readthedocs.io/en/latest/admin-manual/user-priorities-negotiation.html?highlight=NEGOTIATOR_PRE_JOB_RANK#negotiation, the PRE_JOB_RANK
is just a higher prioritized rank in the matching process than the RANK
set by the user. As @tfesenbecker points out in her thesis, it might be nice to be able to set this PRE_JOB_RANK
via class ads, as it can be done for the RANK
.
Maybe I just have mistaken the code and this is already possible?
We considered this already for implementation. The issue is that we currently use a predefined ordering to keep the complexity of the scheduling process bounded. In case we properly add the PRE_JOB_RANK
we have to update the values of each job, and do the sorting before actually going on with everything else. This will have a huge impact on the simulation time.
We foresee to provide another type of scheduler that will properly implement the PRE_JOB_RANK
but will have a much higher cost than the currently implemented scheduler.
In case you have a great idea on what you actually want to do with the PRE_JOB_RANK
, we can prioritise this issue.
Actually, @tfesenbecker already suggested studying this in her thesis, but wasn't able to do so, because of the current implementation. However, I don't see this as high priority at the moment.
The current implementation of the
CondorClassAdScheduler
https://github.com/MatterMiners/lapis.caching/blob/8f4f6149dc536f9d35dfc7c97ce75e0c204a1512/lapis/scheduler.py#L668-L978 does not consider thePRE_JOB_RANK
in the negotiation process. Currently aPRE_JOB_RANK
classad that works in the context ofDrone
s instead ofCachingJob
s is assumed, as this improves the runtime of the scheduling process.Comparable to the implementation of
RANK
, the classad mechanism for thePRE_JOB_RANK
might also be used in the context of all available components includingCachingJob
s, instead of the current drone ranking:https://github.com/MatterMiners/lapis.caching/blob/8f4f6149dc536f9d35dfc7c97ce75e0c204a1512/lapis/scheduler.py#L702-L704
As this will heavily impact the performance of simulation this might be done in a separate class. A renaming of the current
CondorClassAdScheduler
might be reasonable to make clear, that it does not implement the whole functionality of the original HTCondor Scheduler.