search

WIPACrepo / iceprod

IceCube dataset management system
MIT License
4 stars 3 forks source link

dynamic resizing of pilot resource allocation #179

Open dsschult opened 7 years ago

dsschult commented 7 years ago

If the pilot is running multiple task and one goes over memory, consider a priority list for which to keep:

gonzalomerino commented 7 years ago

Related to this, it might be useful to have the capability to configure the iceprod pilot so that in a gpu node it grabs all resources (brian's submit classsds to request all available cpus...) and then it runs the cpu jobs with less mem requirements, so that the gpu job can grow.

Also, for the resource usage check, is Iceprod comparing usage with requested resources or with available resources? Being able to do the latter would be nice.

Gonzalo On Thu, Mar 9, 2017 at 12:01 PM David Schultz notifications@github.com wrote:

If the pilot is running multiple task and one goes over memory, consider a priority list for which to keep:

  • GPU task
  • significantly longer-running task
  • high memory task

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/WIPACrepo/iceprod/issues/179, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjryeOo7GVynniFx1AZzXVey77E6LMLks5rkD5zgaJpZM4MYbdB .

dsschult commented 7 years ago

It compares with both requested and available.