gc3pie spawns VMs and doesn't use them (plus doesn't terminate them)

[GoogleCodeExporter]

The Gc3pie can spawn VMs and leave them running and unused until new jobs 
appear. 
It happens when we create multiple (more than available resources on 1 VM) 
quick jobs (processing time is less than VM spawning time). In this case, 
Gc3pie sees pending jobs, spawns the first VM, after that it submits job/jobs 
to the VM, sees that there are still pending jobs for the submission, spawns 
next VM and here, if already submitted to the first VM jobs are done, gc3pie 
will submit the rest again to the first VM and do nothing with the second VM 
(after it is spawned it will stay running).

I am using the development branch, revision 4123.

Original issue reported on code.google.com by Karanda...@gmail.com on 2 Feb 2015 at 2:13

[GoogleCodeExporter]

I am not sure how we can handle this: GC3Pie is not really designed to
handle short-running jobs.  The main problem here is that
`Engine.progress` assumes that jobs do not change state more than one
time per cycle.

We could use `Application.requested_walltime` to estimate if a job
could change state during a single cycle, but that would only work for
applications that have good running time estimates.  Would that work for you?

Original comment by riccardo.murri@gmail.com on 9 Feb 2015 at 11:06

[GoogleCodeExporter]

For me, the problem is not very critical, I just reported that there is one. 
Because even if such a VM was spawned, it will stay not used only until we 
create another job. After that, gc3pie will correctly use the VM and terminate 
it later. 
Unless the situation repeats...

Original comment by Karanda...@gmail.com on 20 Feb 2015 at 9:55