mturilli
commented
7 years ago This requires providing new information to the reader. Here an extra paragraph I wrote but we may not have the space to add it:
It should be noted that increasing the number of brokers requires also to increase the number of concurrent jobs that can be submitted to Titan's queue. Each broker submits 1 job at a time, asking for no more than 300 worker nodes and at least 2 hours walltime. When backfill availability exceeds 300 worker nodes, multiple brokers are used to submit multiple jobs. Backfill availability is polled every 10 minutes, giving a maximum of 11 concurrent sets of jobs submitted to Titan. Considering the variable amount of time spent by each job in Titan's queue, a large number of brokers and concurrent queued jobs may be required to use all the backfill availability.
Titan's policy was already changed in September 2016, enabling the submission of 20 concurrent jobs. As shown in Fig.~\ref{fig:backfill-utilization}, this improved the efficiency of PanDA brokers but further study is required to understand the impact that a larger amount of concurrent queued jobs would have on the overall efficiency of Titan's scheduler.
wellsjc
In Section 4.2, we state that "Nonetheless, the current load suggests that the number of brokers per DTN could be increased.".
We have increased the number of pilots from 4 to 20 over the course of the project. And this is part of the reason for the increased volume of jobs completed. In discussion Figure 3, we could indicate the dates at which we increased the number of pilots.