search

abhilekhsingh / gc3pie

Automatically exported from code.google.com/p/gc3pie
0 stars 0 forks source link

problem in creating EC2 Virtual machines #447

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Hello,

I am new to GC3Pie and I tried to run a simple application on Amazon-EC2 
resources.

1. I created the ~/.gc3/gc3pie.conf file
2. I wrote the application
3. I ran the application

I got this error message:

gc3.gc3libs: ERROR: Could not create resource 'ec2': 'NoneType' object is not 
callable. Configuration file problem?
gc3.gc3libs: WARNING: Failed creating backend for resource 'amazon-ec2' of type 
'ec2+shellcmd': TypeError: 'NoneType' object is not callable
gc3.gc3libs: CRITICAL: No computational resources defined. Please edit the 
configuration file(s): '/etc/gc3/gc3pie.conf', 
'/home/mohamed/gc3pie/etc/gc3/gc3pie.conf', '/home/mohamed/.gc3/gc3pie.conf'

I am using Ubuntu 12.04
You find attached my gc3pie.conf file content.

Thank you in advance for any help.
Regards,

Original issue reported on code.google.com by mohamed....@gmail.com on 22 Jul 2014 at 8:01

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi Mohammed

either your gc3pie.conf is incomplete or is erroneous.
the ec2 credentials should be specified in a dedicated 
[auth/ ] section (e.g. [auth/aws] )
and then the specific resource should indicate what 'auth' to use:
in your case in
[resource/ec2] you should have (according to the example above)
auth=aws 

keep also in mind that you need to specify the units for memory and walltime
so, for example:
max_memory_per_core = 1GB   
max_walltime = 8 hours

Sergio

Original comment by sergio.m...@gmail.com on 22 Jul 2014 at 8:12

GoogleCodeExporter commented 9 years ago 
Hi Sergio,

Thanks for your response. I corrected the config file and now I got an other 
error message that I think it is related to keypair:
I tried 2 tests:
1. created a keypair 'metapiga' using AWS web interface and download it to may 
local home. Then I extracted 'metapiga.pub' using 'openssh rsa ...' -> I got a 
error message telling that the fingerprint does not match.
2. I removed the 'metapiga' from my local machine and AWS web, and kept only 
'metapiga.pub'. I got this error message :

gc3.gc3libs: INFO: Computational resource 'amazon-ec2' initialized successfully.
gc3.gc3libs: DEBUG: Checking auth: EC2Auth
gc3.gc3libs: INFO: Creating main sequential task
gc3.gc3utils: DEBUG: Added task 'MetaPIGA_ParallelWK' to session.
gc3.gc3libs: DEBUG: Submitting MetaPIGATask.5 ...
gc3.gc3libs: DEBUG: Performing brokering ...
gc3.gc3libs: DEBUG: Checking resource 'amazon-ec2' for compatibility with 
application requirements
gc3.gc3libs: DEBUG: Application scheduler returned 1 matching resources
gc3.gc3libs: DEBUG: Attempting submission to resource 'amazon-ec2'...
gc3.gc3libs: INFO: Keypair `metapiga` not found: creating it using public key 
`/home/mohamed/.ssh/metapiga.pub`
gc3.gc3libs: INFO: Error in submitting job to resource 'amazon-ec2': TypeError: 
not enough arguments for format string
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 242, in __submit_application
    lrms.submit_job(app)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 811, in submit_job
    user_data=user_data)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 403, in _create_instance
    self._import_keypair()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 515, in _import_keypair
    % self.keypair_name, ex)
TypeError: not enough arguments for format string

Thanks,
Mohamed

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 10:32

GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

the line numbers in the traceback do not match the current sources.
What version of GC3Pie are you using?

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 22 Jul 2014 at 10:39

GoogleCodeExporter commented 9 years ago 
Dear Riccardo,

I used the command line to install it 
sh -c "$(wget -O- http://gc3pie.googlecode.com/svn/install.sh)"

Thanks,
Mohamed

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 10:43

GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

apparently that bug has already been fixed, but the fixed version was never 
released.

I have now released the up-to-date sources, can you please install GC3Pie again 
or upgrade, and then retry?

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 22 Jul 2014 at 11:06

GoogleCodeExporter commented 9 years ago 
Dear Recardo,

Thanks for this fix. I re-installed the GC3Pie and the error message changed.
For the keypair. I generate it from AWS web interface and download it to may 
local machine (I remove the .pem extension). Then I extract the public key 
using:
openssl rsa -in ~/.ssh/metapiga  -pubout -out  ~/.ssh/metapiga.pub

...
gc3.gc3libs: INFO: Computational resource 'amazon-ec2' initialized successfully.
gc3.gc3libs: DEBUG: Checking auth: EC2Auth
gc3.gc3utils: WARNING: Session already exists, some command-line arguments 
might be ignored.
gc3.gc3libs: DEBUG: Submitting MetaPIGATask.11 ...
gc3.gc3libs: DEBUG: Performing brokering ...
gc3.gc3libs: DEBUG: Checking resource 'amazon-ec2' for compatibility with 
application requirements
gc3.gc3libs: DEBUG: Application scheduler returned 1 matching resources
gc3.gc3libs: DEBUG: Attempting submission to resource 'amazon-ec2'...
gc3.gc3libs: DEBUG: File `/home/mohamed/.ssh/metapiga` is not a valid DSS 
private key: not a valid DSA private key file
gc3.gc3libs: ERROR: Keypair `metapiga` is present but has different 
fingerprint: 51:42:b5:9f:81:75:65:71:84:5a:8d:37:0f:f2:05:34 != 
d5:92:e0:dc:72:5b:42:ab:64:0c:dd:20:cc:17:48:10:e4:45:bd:0f. Aborting!
gc3.gc3libs: INFO: Error in submitting job to resource 'amazon-ec2': 
UnrecoverableError: Keypair `metapiga` is present but has different 
fingerprint: 51:42:b5:9f:81:75:65:71:84:5a:8d:37:0f:f2:05:34 != 
d5:92:e0:dc:72:5b:42:ab:64:0c:dd:20:cc:17:48:10:e4:45:bd:0f. Aborting!
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 242, in __submit_application
    lrms.submit_job(app)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 811, in submit_job
    user_data=user_data)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 452, in _create_instance
    keypairs[self.keypair_name].fingerprint))
UnrecoverableError: Keypair `metapiga` is present but has different 
fingerprint: 51:42:b5:9f:81:75:65:71:84:5a:8d:37:0f:f2:05:34 != 
d5:92:e0:dc:72:5b:42:ab:64:0c:dd:20:cc:17:48:10:e4:45:bd:0f. Aborting!
...

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 11:25

GoogleCodeExporter commented 9 years ago 
Dear Mohammed,

The `openssl` command is not suitable for converting SSH
private/public keys; the `ssh-keygen` command should be used instead.

Can you please extract your private and public keys according to steps
1.-4. of this:

http://www.question-defense.com/2009/10/01/convert-amazon-web-services-private-r
sa-key-for-use-with-securecrt

You might want to substitute `amazon-ec2-key` with `metapiga` to match
the filenames you have already created.

Once you have extracted the keys according to the new procedure, can
you please retry and let us know?

Original comment by riccardo.murri@gmail.com on 22 Jul 2014 at 11:42

GoogleCodeExporter commented 9 years ago 
Using the AWS generated keypair gave the fingerprint error message. 
So I generated locally my keypair using ssh-keygen and upload the public key on 
AWS. Then, after launching the program, a virtual machine was created on AWS 
web interface. 

Is that how the output message should look if the submission is ok?
Thanks,
Mohamed

gc3.gc3libs: INFO: Computational resource 'amazon-ec2' initialized successfully.
gc3.gc3libs: DEBUG: Checking auth: EC2Auth
gc3.gc3utils: WARNING: Session already exists, some command-line arguments 
might be ignored.
gc3.gc3libs: DEBUG: Submitting MetaPIGATask.11 ...
gc3.gc3libs: DEBUG: Performing brokering ...
gc3.gc3libs: DEBUG: Checking resource 'amazon-ec2' for compatibility with 
application requirements
gc3.gc3libs: DEBUG: Application scheduler returned 1 matching resources
gc3.gc3libs: DEBUG: Attempting submission to resource 'amazon-ec2'...
gc3.gc3libs: INFO: Keypair `metapiga` not found: creating it using public key 
`/home/mohamed/.ssh/metapiga.pub`
gc3.gc3libs: INFO: Successfully imported key `metapiga` with fingerprint 
`90:8d:a3:55:7a:e0:54:e7:5a:4f:c3:9a:f6:db:81:f2` as keypair `metapiga`
gc3.gc3libs: INFO: Creating security group metpiga_ssh
gc3.gc3libs: DEBUG: Adding rule {'to_port': 22, 'from_port': 22, 'ip_protocol': 
'tcp', 'cidr_ip': '0.0.0.0/0'} to security group metpiga_ssh.
gc3.gc3libs: DEBUG: Adding rule {'to_port': -1, 'from_port': -1, 'ip_protocol': 
'icmp', 'cidr_ip': '0.0.0.0/0'} to security group metpiga_ssh.
gc3.gc3libs: DEBUG: Create new VM using image id `ami-9d996bea`
gc3.gc3libs: INFO: VM with id `i-c8c64788` has been created and is in pending 
state.
gc3.gc3libs: INFO: No available resource was found, but some VM is still in 
`pending` state. Waiting until the next iteration before creating a new VM. 
Pending VM ids: i-c8c64788
gc3.gc3libs: INFO: Submission of job MetaPIGATask.11 delayed
gc3.gc3libs: ERROR: Ignored error in submitting task 'MetaPIGATask.11': 
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
gc3.gc3libs: DEBUG: Ignored error in submitting task 'MetaPIGATask.11': 
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 969, in progress
    self._core.submit(task)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 168, in submit
    return self.__submit_application(app, resubmit, **extra_args)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 271, in __submit_application
    raise ex
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
gc3.gc3libs: DEBUG: Submitting MetaPIGATask.12 ...
gc3.gc3libs: DEBUG: Performing brokering ...
gc3.gc3libs: DEBUG: Checking resource 'amazon-ec2' for compatibility with 
application requirements
gc3.gc3libs: DEBUG: Application scheduler returned 1 matching resources
gc3.gc3libs: DEBUG: Attempting submission to resource 'amazon-ec2'...
gc3.gc3libs: INFO: No available resource was found, but some VM is still in 
`pending` state. Waiting until the next iteration before creating a new VM. 
Pending VM ids: i-c8c64788
gc3.gc3libs: INFO: Submission of job MetaPIGATask.12 delayed
gc3.gc3libs: ERROR: Ignored error in submitting task 'MetaPIGATask.12': 
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
gc3.gc3libs: DEBUG: Ignored error in submitting task 'MetaPIGATask.12': 
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 969, in progress
    self._core.submit(task)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 168, in submit
    return self.__submit_application(app, resubmit, **extra_args)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 271, in __submit_application
    raise ex
LRMSSkipSubmissionToNextIteration: Delaying submission until some of the VMs 
currently pending is ready. Pending VM ids: i-c8c64788
gc3.gc3libs: INFO: Closing connection to cloud 'amazon-ec2'...
Starting metapiga; use the '-v' command-line option to get a more verbose 
report of activity.
Status of jobs in the 'metapiga' session: (at 15:15:25, 07/22/14)
         NEW  3/3  (100.0%) 
     RUNNING  0/3   (0.0%)  
     STOPPED  0/3   (0.0%)  
   SUBMITTED  0/3   (0.0%)  
  TERMINATED  0/3   (0.0%)  
 TERMINATING  0/3   (0.0%)  
     UNKNOWN  0/3   (0.0%)  
       total  3/3  (100.0%)

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 1:25

GoogleCodeExporter commented 9 years ago 
Yes, that is how a successful submission log looks like.

To cross check: after a while, the command should show that 3 tasks are in 
RUNNING state.

Can we consider the issue solved?

Original comment by riccardo.murri@gmail.com on 22 Jul 2014 at 1:32

GoogleCodeExporter commented 9 years ago 

After a while, I got an error message. Sorry for that.
I relaunched the same command, and I got an error message gc3.gc3utils: 
CRITICAL: KeyError: 'ubuntu'. I can manually log-in the instance using 
ssh-client.

gc3.gc3libs: DEBUG: Creating resource 
'ec2-54-73-28-204.eu-west-1.compute.amazonaws.com@amazon-ec2' defined by: 
architecture=set(['x86_64']), auth='ubuntu', enabled=True, 
frontend=u'ec2-54-73-28-204.eu-west-1.compute.amazonaws.com', 
ignore_ssh_host_keys=True, keyfile='/home/mohamed/.ssh/metapiga', max_cores=30, 
max_cores_per_job=2, max_memory_per_core=Memory(2, unit=GB), 
max_walltime=Duration(8, unit=h), metapiga_image_id='ami-9d996bea', 
metapiga_instance_type='m1.small', 
name=u'ec2-54-73-28-204.eu-west-1.compute.amazonaws.com@amazon-ec2', 
override='True', security_group_name='metpiga_ssh', 
security_group_rules='tcp:22:22:0.0.0.0/0, icmp:-1:-1:0.0.0.0/0', 
transport='ssh', type='shellcmd', vm_auth='ubuntu'.
gc3.gc3libs: INFO: Computational resource 
'ec2-54-73-28-204.eu-west-1.compute.amazonaws.com@amazon-ec2' initialized 
successfully.
gc3.gc3libs: ERROR: Could not create resource 
'ec2-54-73-28-204.eu-west-1.compute.amazonaws.com@amazon-ec2': 'ubuntu'. 
Configuration file problem?
gc3.gc3utils: CRITICAL: KeyError: 'ubuntu'
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/cmdline.py", line 437, in run
    return cli.app.CommandLineApp.run(self)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/cli/app.py", line 245, in run
    return self.post_run(returned)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/cli/app.py", line 241, in run
    returned = self.main(*args)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/cmdline.py", line 1382, in _main
    self._controller.close()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 1177, in close
    self._core.close()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 642, in close
    lrms.close()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 900, in close
    self.get_resource_status()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 701, in get_resource_status
    resource = self._get_remote_resource(vm)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 482, in _get_remote_resource
    self.resources[vm.id] = self._make_resource(vm.public_dns_name)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 539, in _make_resource
    resource = cfg._make_resource(args)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/config.py", line 600, in _make_resource
    return cls(**dict(resdict))
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/shellcmd.py", line 199, in __init__
    auth = self._auth_fn()
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/config.py", line 465, in <lambda>
    return (lambda **extra_args: self.auth_factory.get(name, **extra_args))
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/authentication/__init__.py", line 111, in get
    a =  self._ctors[auth_name](**dict(params))
KeyError: 'ubuntu'
Starting metapiga; use the '-v' command-line option to get a more verbose 
report of activity.
Status of jobs in the 'metapiga' session: (at 15:35:21, 07/22/14)
         NEW  3/3  (100.0%) 
     RUNNING  0/3   (0.0%)  
     STOPPED  0/3   (0.0%)  
   SUBMITTED  0/3   (0.0%)  
  TERMINATED  0/3   (0.0%)  
 TERMINATING  0/3   (0.0%)  
     UNKNOWN  0/3   (0.0%)

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 1:38

GoogleCodeExporter commented 9 years ago 
If you have `vm_auth=ubuntu` in your `gc3pie.conf` configuration file, then you 
should have a matching "auth" section:

    [auth/ssh_gc3user]
    # Default user on "Hobbes" VM images is `gc3-user`
    type=ssh
    username=gc3-user

Can you please try adding this and see if it solves the problem?

Original comment by riccardo.murri@gmail.com on 22 Jul 2014 at 1:50

GoogleCodeExporter commented 9 years ago 
I re-installed the gc3pie environment.
I run my application: $> python metapiga.py  -v 
/home/mohamed/VENUSC/demo/metapiga/metapiga_input.tar

Status of jobs in the 'metapiga' session: (at 17:05:38, 07/22/14)
         NEW  3/3  (100.0%) 
     RUNNING  0/3   (0.0%)  
     STOPPED  0/3   (0.0%)  
   SUBMITTED  0/3   (0.0%)  
  TERMINATED  0/3   (0.0%)  
 TERMINATING  0/3   (0.0%)  
     UNKNOWN  0/3   (0.0%)  
       total  3/3  (100.0%) 

I wait until the virtual machine is started on AWS interface.
I re-launch the command: $> python metapiga.py  -v 
/home/mohamed/VENUSC/demo/metapiga/metapiga_input.tar

I got this message. 
gc3.gc3utils: INFO: Starting metapiga at Tue Jul 22 17:05:16 2014; invoked as 
'metapiga.py -v /home/mohamed/VENUSC/demo/metapiga/metapiga_input.tar'
gc3.gc3libs: INFO: Computational resource 'amazon-ec2' initialized successfully.
gc3.gc3utils: WARNING: Session already exists, some command-line arguments 
might be ignored.
gc3.gc3libs: INFO: Creating remote ShellcmdLrms resource for ip 
ec2-54-195-126-166.eu-west-1.compute.amazonaws.com
gc3.gc3libs: INFO: Computational resource 
'ec2-54-195-126-166.eu-west-1.compute.amazonaws.com@amazon-ec2' initialized 
successfully.
gc3.gc3libs: INFO: Ignoring ssh host key file.
gc3.gc3libs: INFO: creating resource file folder: /home/ubuntu/.gc3/shellcmd.d 
...
gc3.gc3libs: WARNING: `max_cores` value on resource 
ec2-54-195-126-166.eu-west-1.compute.amazonaws.com@amazon-ec2 mismatch: 
configurationfile says `30` while it's actually `1`. Updating current value.
gc3.gc3libs: WARNING: `max_memory_per_core` value on resource 
ec2-54-195-126-166.eu-west-1.compute.amazonaws.com@amazon-ec2 mismatch: 
configuration file says `2GB` while it's actually `1.69132e+06KiB`. Updating 
current value.
gc3.gc3libs: ERROR: Keypair `metapiga` is present but has different 
fingerprint: 3b:80:f6:a4:90:3e:4b:1c:16:ef:79:67:f0:47:8b:02 != 
99:15:ec:98:21:09:7a:c4:a3:a9:73:87:4d:91:b2:dd. Aborting!
gc3.gc3libs: INFO: Error in submitting job to resource 'amazon-ec2': 
UnrecoverableError: Keypair `metapiga` is present but has different 
fingerprint: 3b:80:f6:a4:90:3e:4b:1c:16:ef:79:67:f0:47:8b:02 != 
99:15:ec:98:21:09:7a:c4:a3:a9:73:87:4d:91:b2:dd. Aborting!
Traceback (most recent call last):
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/core.py", line 242, in __submit_application
    lrms.submit_job(app)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 811, in submit_job
    user_data=user_data)
  File "/home/mohamed/gc3pie/local/lib/python2.7/site-packages/gc3libs/backends/ec2.py", line 452, in _create_instance
    keypairs[self.keypair_name].fingerprint))

Original comment by mohamed....@gmail.com on 22 Jul 2014 at 3:08

GoogleCodeExporter commented 9 years ago 
Please find attached the log file of an execution. It seems that there is still 
an issue with the public key fingerprint.

Thanks.
Mohamed

Original comment by mohamed....@gmail.com on 23 Jul 2014 at 9:15

Attachments:

GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

I believe this is a configuration issue: somehow the SSH key injected
into the virtual machine is not the one GC3Pie is using to connect to
it.  Your earlier reports and logs list 5 different key fingerprints,
whereas there should be only one!

So, can you please post the output of the following commands:

1) ls -l ~/.ssh/

2) for each `.pub` file in the above output, run the following
command::

        ssh-keygen ~/.ssh/file.pub

(of course, substitute "file.pub" with the actual file name)

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 23 Jul 2014 at 4:23

GoogleCodeExporter commented 9 years ago 
Dear Riccardo,

Thank you for your response.
Yes, It seems to me that there is a format problem in calculating the 
fingerprint. What I did is:
1. remove all public private key from my ~/.ssh and from AWS-EC2 web portal.
2. generate a keypair using AWS-EC2 web portal and download it into 
~/.ssh/metapiga
3. run the application:  python metapiga.py metapiga_input.tar -vv   -C 25 

So, on the AWS-EC2 web portal, one VM is started. If I take a look on the 
uploaded publickey on AWS-EC2, I find  the value 
'1a:5e:92:4a:cd:8e:32:32:cd:65:da:59:90:14:45:06'. 
On my machine, I run these two commands:

$ ssh-keygen -lf  ~/.ssh/metapiga 
2048 0c:a9:83:3f:99:ae:5c:2e:49:19:5c:84:74:4b:5f:d1 
/home/benbelgacem/.ssh/metapiga.pub (RSA)
$ openssl pkey -in ~/.ssh/metapiga -pubout -outform DER | openssl md5 -c
(stdin)= 1a:5e:92:4a:cd:8e:32:32:cd:65:da:59:90:14:45:06

In the log file of the submission, I got this error message:
...
gc3.gc3libs: DEBUG: Ignoring error while submit to resource 
ec2-54-75-64-207.eu-west-1.compute.amazonaws.com@amazon-ec2: Resource 
ec2-54-75-64-207.eu-west-1
.compute.amazonaws.com@amazon-ec2 does not have enough available memory: 
1.73192e+09B < 2GB.. 
gc3.gc3libs: DEBUG: File `/home/benbelgacem/.ssh/metapiga` is not a valid DSS 
private key: not a valid DSA private key file
gc3.gc3libs: ERROR: Keypair `metapiga` is present but has different 
fingerprint: 0c:a9:83:3f:99:ae:5c:2e:49:19:5c:84:74:4b:5f:d1 != 
1a:5e:92:4a:cd:8e:32:32:
cd:65:da:59:90:14:45:06. Aborting!
gc3.gc3libs: INFO: Error in submitting job to resource 'amazon-ec2': 
UnrecoverableError: Keypair `metapiga` is present but has different 
fingerprint: 0c:a9:
83:3f:99:ae:5c:2e:49:19:5c:84:74:4b:5f:d1 != 
1a:5e:92:4a:cd:8e:32:32:cd:65:da:59:90:14:45:06. Aborting!
...

For me, it seems like the gc3Pie uses the SHA1 algorithm  and AWS-EC2 the MD5 
algorithm to compute the fingerprint.
What do you think ?

Thanks,
Mohamed

Original comment by mohamed....@gmail.com on 23 Jul 2014 at 7:08

GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

The logs you posted so far show no issue with the GC3Pie script or
application class.  Rather, there is evidently an authentication
problem which probably originates into different key pairs being used
by Amazon AWS and your local GC3Pie installation.

However, GC3Pie is able to *upload* its SSH key to Amazon, provided
the keypair does not exist on Amazon already.

So can you please make another attempt, following the steps I write here?

1. Create a new keypair on the computer where you run GC3Pie::

        ssh-keygen -f rsa -C "gc3pie" -f ~/.ssh/gc3pie

   Please make sure that you type the `-t rsa` option; Amazon AWS does
   not accept DSA keys, and your configuration file states that your
   SSH key is called `myec2_dsa.pub` which I guess is a DSA key :-)

   This should create *two* files:

        ~/.ssh/gc3pie
        ~/.ssh/gc3pie.pub

2. Modify your `~/.gc3/gc3pie.conf` configuration file to use this new keypair:

   (a) Replace `keypair_name=myec2_dsa` on line 19 with: `keypair_name=gc3pie`
   (b) Replace `public_key=~/.ssh/myec2_dsa.pub` on line 24 with: `public_key=~/.ssh/gc3pie.pub`

3. Now try to re-run your GC3Pie driver script, with max debugging level 
(`-vvvv`)

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 24 Jul 2014 at 12:45

GoogleCodeExporter commented 9 years ago 
Dear Riccardo,

Now things seem better. I generated the  keypair as you described: if the
'vm_pool_max_size'=1 in my `~/.gc3/gc3pie.conf` , then the creation of a VM
succeeds and the output file of my job is downloaded on my local machine.
However, If 'vm_pool_max_size' is greater than 1, then the error of
fingerprint appears. So, I commented the lines 447-452 in the file 'ec2.py'
and re-launched the submission of a parallel workflow of 10 jobs with
'vm_pool_max_size=5'
(see my gc3pie.conf content below) -> it works :)

I would like please to check the meaning of :
- 'max_cores = N':  means that we can use at max N cores per/VM?
- 'vm_pool_max_size=x':  means that the number of running VMs should be <=
x ?

My application is a ParallelTaskCollection based workflow  which is a kind
of Map/reduce workflow: it submits m independent jobs on AWS and retrieve
their output files.

During execution, I remarked that the application (the script session):
- spawns the x virtual machines in sequential ( where, vm_pool_max_size=x),
i.e., its starts the first Vn and wait until it is available, then, it
start the second one ...
- if the job is finished, it shutdowns its corresponding VM and starts a
new VM for the next job

Is it possible to starts x virtual machines in parallel? how?
Is there any way to re-use VMs without shutting them down during the same
computation?

Many thanks !!
Mohamed

[auth/aws]
ec2_access_key=AAAAAAAAAAAAAAA
ec2_secret_key=BBBBBBBBBBBBBBBBBBBB
type=ec2

[auth/sshaws]
type=ssh
username=ubuntu

[resource/amazon-ec2]
enabled=true
auth=aws

ec2_url=https://ec2.eu-west-1.amazonaws.com
ec2_region=eu-west-1
type=ec2+shellcmd

max_cores_per_job = 1
max_memory_per_core =1 GB
max_walltime = 8
max_cores = 1
architecture = x86_64

keypair_name=gc3pie
public_key=~/.ssh/gc3pie.pub

vm_auth=sshaws
instance_type=m1.small
image_id=ami-9d996bea
security_group_name=metpiga_ssh
security_group_rules=tcp:22:22:0.0.0.0/0, icmp:-1:-1:0.0.0.0/0
vm_pool_max_size = 5

#user_data=#!/bin/bash
# command -v java || (sudo apt-get update ; sudo apt-get install -y
openjdk-6-jre-headless)

2014-07-24 14:45 GMT+02:00 <gc3pie@googlecode.com>:

Original comment by mohamed....@gmail.com on 25 Jul 2014 at 10:18

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

Thank you for your patience and for replaying the steps; it seems we're 
approaching

regarding the issue of fingerprints:

> However, If 'vm_pool_max_size' is greater than 1, then the error of
> fingerprint appears. So, I commented the lines 447-452 in the file 'ec2.py'
> and re-launched the submission of a parallel workflow of 10 jobs with
> 'vm_pool_max_size=5'
> (see my gc3pie.conf content below) -> it works :)

Ok, then it seems we have a real issue here: AWS EC2 and Paramiko (the
Python library for doing SSH connections) are computing the key
fingerprint in different ways, which we cannot compare.

I'll add some code to disable the check.

> I would like please to check the meaning of :
> - 'max_cores = N':  means that we can use at max N cores per/VM?
> - 'vm_pool_max_size=x':  means that the number of running VMs should be <=
> x ?

> Is it possible to starts x virtual machines in parallel? how?

That's not currently supported.  The issue is that the AWS EC2 API
does not provide details on the VM features (how many cores, GBs of
RAM, etc.) that correspond to a certain "flavor" (e.g., "m1.small").
So, GC3Pie has to launch a VM, wait for it to be networked, log in and
find out.  It could well be that the VM has enough cores to launch all
the pending jobs, or maybe we need to spawn another VM...

So, if you are interested in job throughput, use a larger flavor.

Any ideas on how to handle this is welcome, but please open another
issue ticket if you want to discuss this issue further -- let's not
intertwine discussion threads.

> Is there any way to re-use VMs without shutting them down during the same
> computation?

That should be done automatically already (but maybe not in the 2.1.x
release -- I need to check).

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 25 Jul 2014 at 1:28

GoogleCodeExporter commented 9 years ago 
Thanks a lot for you response and help!

Original comment by mohamed....@gmail.com on 25 Jul 2014 at 3:11

GoogleCodeExporter commented 9 years ago 
Dear Mohamed,

I have just released GC3Pie 2.1.5, which should contain a fix for the digest 
incompatibility issue.

Can you please test it?

Thanks,
Riccardo

Original comment by riccardo.murri@gmail.com on 25 Jul 2014 at 5:45

GoogleCodeExporter commented 9 years ago 
Dear Riccardo,
Thanks for the fix. I tested the release 2.1.5 and it works :)

Bests,
Mohamed

2014-07-25 19:45 GMT+02:00 <gc3pie@googlecode.com>:

Original comment by mohamed....@gmail.com on 26 Jul 2014 at 10:57

GoogleCodeExporter commented 9 years ago 
Fixed in SVN r3957 for the 2.1.x series, and r3965-r3966 on the "trunk" 
development line (shortly thereafter released as 2.2).

Original comment by riccardo.murri@gmail.com on 29 Jul 2014 at 12:08