search

dmwm / CRAB2

CRAB2
2 stars 11 forks source link

JobTracking and GetOutput fine tuning #51

Closed ericvaandering closed 10 years ago

ericvaandering commented 10 years ago

Original Savannah ticket 6873 reported by mcinquil on Tue May 20 03:32:48 2008.

Current settings may not be the best and need a fine tuning. I would suggest a reduction of the poll rates (e.g. pollInterval = 1'800) in order to reduce the overall load and in particular to free resources for status query threads and output handling threads. This while leaving the Query interval quite short (up to 120). This should results in a small delay in the finished jobs handling and a speed up of the job status update. This preliminary work should give an idea about the actual delay of the two components an open to new optimizations.

ericvaandering commented 10 years ago

Comment by mcinquil on Thu May 22 03:21:16 2008

As suggested I tried to increase the value of poll-interval and decrease the value for query-interval; also, seen that the memory usage seems not too high, I tries to rise-up the number of maximum jobs for each thread (*). At the moment I do not see particular problem hardware usage, but I need to test with more and more jobs.

(*) <ConfigBlock Name="JobTracking"> <Parameter Name="PoolThreadsSize" Value="5"/> <Parameter Name="ComponentDir" Value="/home/crabs/work/JobTracking"/> <Parameter Name="jobsToPoll" Value="300"/> <Parameter Name="RssFeed" Value="no"/> <Parameter Name="QueryInterval" Value="60"/> <Parameter Name="PollInterval" Value="900"/> </ConfigBlock>

ericvaandering commented 10 years ago

Comment by mcinquil on Wed May 28 05:54:57 2008

Using the configuration suggested by Giuseppe (*).

I am noticing a side effect: the resubmission (but the problem is more general) of a failed job is slower increasing the values of PollInterval for both the components...

Mattia

(*) <ConfigBlock Name="JobTracking"> <Parameter Name="PoolThreadsSize" Value="5"/> <Parameter Name="ComponentDir" Value="/home/crab/work/JobTracking"/> <Parameter Name="jobsToPoll" Value="300"/> <Parameter Name="RssFeed" Value="no"/> <Parameter Name="QueryInterval" Value="3"/>

<Parameter Name="PollInterval" Value="300"/>

            &lt;ConfigBlock Name=&quot;GetOutput&quot;&gt;
                    &lt;Parameter Name=&quot;OutputLocation&quot; Value=&quot;SE&quot;/&gt;
                    &lt;Parameter Name=&quot;GetOutputPoolThreadsSize&quot; Value=&quot;5&quot;/&gt;
                    &lt;Parameter Name=&quot;PollInterval&quot; Value=&quot;300&quot;/&gt;
                    &lt;Parameter Name=&quot;ComponentDir&quot; Value=&quot;/home/crab/work/GetOutput&quot;/&gt;
                    &lt;Parameter Name=&quot;RssFeed&quot; Value=&quot;no&quot;/&gt;
            &lt;/ConfigBlock&gt;
ericvaandering commented 10 years ago

Comment by gcodispo on Thu May 29 04:06:07 2008

With the provided setting, you can have a max delay of 10 minutes before triggering the getOutput and some more before the resubmission. Increasing the polling time you can reduce this delay even to few seconds. Thus reducing also resources for the threads querying the scheduler and increasing the delay for the status update, up to unespected values. I would prefer wait a bit for resubmission instead.

ericvaandering commented 10 years ago

Comment by spiga on Mon Aug 25 12:27:12 2008

which is the status of this task?

Daniele

ericvaandering commented 10 years ago

Comment by mcinquil on Tue Oct 28 16:44:56 2008

The actual default configuration seems to be a good compromize. These parameters will be checked with the next development and following up the next scale tests.

ericvaandering commented 10 years ago

Closed by mcinquil on Tue Oct 28 16:44:56 2008