Shall we introduce 'requested_disk_space' in Application class

[GoogleCodeExporter]

This becomes necessary for the openstack backend as the flavor is selected 
based on: cores, memory, disk

More than in other backends, requested_disk would guarantee a minimal 
availability of disk space resources that might, otherwise let the Application 
fail due to lack of local disk space

S.

Original issue reported on code.google.com by sergio.m...@gmail.com on 17 Apr 2014 at 8:30

[GoogleCodeExporter]

I like the idea, but it's a bit tricky. Try not to think about our
standard images, and then ask yourself how can you check *where* this
space is, and if the user can actually access it?
A virtual machine (or any other resource) can have multiple
mountpoints, not all accessible by the user. And what happen when we
use elasticluster so that the compute nodes will have little root/home
filesystems but a big /glusterfs partitions?

.a.

Original comment by arcimbo...@gmail.com on 17 Apr 2014 at 9:01

[GoogleCodeExporter]

What we agreed on 2014.04.17:

* Application will support 'requested_disk_space' as optional argument ("hint" 
for the backends)
* LRMS backends will choose whether take the requirement into account or not
* LRMS backends will provide 'available_disk_space' as resource information
* LRMS backends will set 'available_disk_space' to 'None' in case they could 
not provide meaningfull information
* Application will treat both information (Application.requested_disk_space and 
LRMS.available_disk_space) as optional in 'compatible_resources' method
* Shellcmd backend will provide local disk information according to what the 
variable $TMP_DIR, local to the resource, will point to (df -h $TMP_DIR). 
Alternatively '/tmp/ will be used.
* Openstack backend will record the 'image' disk foot-print (from the shellcmd 
subresource)
* Openstack backend will use the image disk foot-print information when 
selecting the 'valid_flavors' in  'get_instance_type_for_job'

S.

Original comment by sergio.m...@gmail.com on 17 Apr 2014 at 10:52

[GoogleCodeExporter]

Regarding the last two items: the EC2 backend should do the same.

Original comment by riccardo.murri@gmail.com on 17 Apr 2014 at 1:45

• Added labels: Backend-EC2, Backend-Shellcmd

[GoogleCodeExporter]

Original comment by riccardo.murri@gmail.com on 17 Apr 2014 at 1:48

• Added labels: Backend-OpenStack

[GoogleCodeExporter]

EC2 backend does not make any selection on the flavors as it does not hold 
valid information on them (only the name coming from the configuration file)

It might require more effort to bring EC2 backend to the point where it can 
retain flavor-specific information.
Additionally, it seems that boto (the cloud library we use to access the EC2 
compatible resources) do not allow to get flavor information, so we will have 
to build it within the EC2 backend.

Original comment by sergio.m...@gmail.com on 17 Apr 2014 at 2:36

[GoogleCodeExporter]

| Additionally, it seems that boto (the cloud library we use to access the EC2
| compatible resources) do not allow to get flavor information, so we will
| have to build it within the EC2 backend.

Just like we do caching of the image overhead for OpenStack, we can do
caching of the image overhead for EC2.

Regarding the flavors: I think we should keep a record of the mapping
"flavor name" -> features, but this is probably a different issue.

Original comment by riccardo.murri@gmail.com on 17 Apr 2014 at 3:43