search

UMCUGenetics / bamMetrics

Tool to generate bam statistics and pdf/html reports.
MIT License
2 stars 2 forks source link

respect requested resources #8

Closed hinri closed 8 years ago

hinri commented 8 years ago

Currently picard is using more cpu cores than requested from the scheduler. Please include: -XX:ParallelGCThreads=($queueThreads-1) in order to avoid overloads.

rernst commented 8 years ago

Hi Hinri,

We have seen this problem previously. Are you sure that one thread will solve it? We have seen spikes with java up to a few threads. This also means that the job still 'shows' more threads than requested. I thought that this would be handled by the new cluster setup. I would rather like to see it fixed there.

Robert

Verzonden vanaf mijn Samsung Galaxy-smartphone.

-------- Oorspronkelijk bericht -------- Van: hinri notifications@github.com Datum: 23-12-2015 09:04 (GMT+01:00) Aan: CuppenResearch/bamMetrics bamMetrics@noreply.github.com Onderwerp: [bamMetrics] respect requested resources (#8)

Currently picard is using more cpu cores than requested from the scheduler Please include: -XX:ParallelGCThreads=($queueThreads-1) in order to avoid overloads

— Reply to this email directly or view it on GitHubhttps://github.com/CuppenResearch/bamMetrics/issues/8.

De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct te informeren door het bericht te retourneren. Het Universitair Medisch Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197.

Denk s.v.p aan het milieu voor u deze e-mail afdrukt.

This message may contain confidential information and is intended exclusively for the addressee. If you receive this message unintentionally, please do not use the contents but notify the sender immediately by return e-mail. University Medical Center Utrecht is a legal person by public law and is registered at the Chamber of Commerce for Midden-Nederland under no. 30244197.

Please consider the environment before printing this e-mail.

hinri commented 8 years ago

Hi Robert,

The merit of binding jobs to cores in the new cluster is to avoid overload in cases a job escapes from the user expectations in terms of cpu usage.

In general applies that in a scheduled environment users should formulate job resource request as appropriate as possible. I would expect that in this case there is something to gain here;-)

Hinri

On Wed, 23 Dec 2015 00:13:09 -0800 Robert Ernst notifications@github.com wrote:

Hi Hinri,

We have seen this problem previously. Are you sure that one thread will solve it? We have seen spikes with java up to a few threads. This also means that the job still 'shows' more threads than requested. I thought that this would be handled by the new cluster setup. I would rather like to see it fixed there.

Robert

Verzonden vanaf mijn Samsung Galaxy-smartphone.

-------- Oorspronkelijk bericht -------- Van: hinri notifications@github.com Datum: 23-12-2015 09:04 (GMT+01:00) Aan: CuppenResearch/bamMetrics bamMetrics@noreply.github.com Onderwerp: [bamMetrics] respect requested resources (#8)

Currently picard is using more cpu cores than requested from the scheduler Please include: -XX:ParallelGCThreads=($queueThreads-1) in order to avoid overloads

— Reply to this email directly or view it on GitHubhttps://github.com/CuppenResearch/bamMetrics/issues/8.

De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct te informeren door het bericht te retourneren. Het Universitair Medisch Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W. (Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197.

Denk s.v.p aan het milieu voor u deze e-mail afdrukt.

This message may contain confidential information and is intended exclusively for the addressee. If you receive this message unintentionally, please do not use the contents but notify the sender immediately by return e-mail. University Medical Center Utrecht is a legal person by public law and is registered at the Chamber of Commerce for Midden-Nederland under no. 30244197.

Please consider the environment before printing this e-mail.

Reply to this email directly or view it on GitHub: https://github.com/CuppenResearch/bamMetrics/issues/8#issuecomment-166831462