EC2: specifying key_name in the create should be sufficient to login later

[yarikoptic]

using

niceman create -t aws-ec2 test-ec2-west-small -b region_name=us-west-1 image=ami-059e7901352ebaef8 security_group=ssh-only key_name=yoh@hopa-west2 instance_type=t2.small

so I could reuse already existing key, I am specifying the key_name. Ideally niceman should find the key during login. If impossible for some reason, then it should record key_filename which is now None.

[mjtravers]

@yarikoptic

When an EC2 instance is created, AWS needs the key_name so it knows which public key to embed in the instance so that the user can SSH into it later. The AWS key and secret key are used for authentication to create the EC2 instance so at no point during creating instances do we have the private SSH key filename information.

It will have to be up to the user to know which private SSH key file is matched with the public key_name at AWS and either provide it at login or allow niceman to search and find it in the usual SSH private key places on the local drive.

[yarikoptic]

sorry @mjtravers if I am still missing something -- I (user) do specify which key to use (key_name=yoh@hopa-west2), so it is indeed our (niceman) job to check if such key is already known and use it.

[mjtravers]

@yarikoptic Yes, if the key_name provided exists on AWS, it is used and matched with the key_filename parameter for the matching SSH private key on the local machine. If the key_name is not specified, then the user is prompted for one and, if it doesn't exist at AWS, it is created with a public key created at AWS and a matching private key created on the local host. If the key_name is initially provided and does not exist at AWS, an error message is displayed and the create process is terminated.

[yarikoptic]

sounds logical, but initial issue text suggests that reliance on key_filename hinders it... was it mitigated - could you demonstrate (create one, reuse that key for the other new one)? (I just do not have any existing ec2 resource atm, didn't want to breed new ones to try)

[mjtravers]

If a SSH key-pair is created while creating an EC2 instance, the key_name and key_filename for the pair will be saved to the inventory file so that it does not have to be repeated for further interactions with this specific resource. However, a new niceman create command will not know anything about the key-pair unless the info passed in at the command line or saved in the inventory.cfg file. So the new key-pair can be used but, at least at the moment, the key-pair has to be restated in the new create command.

We could set something up so that if a new SSH key-pair is created and none is currently registered in the niceman.cfg file, we could save it there as the default key-pair for all future EC2 commands.

[yarikoptic]

or, if key_filename is empty/not provided, check our "stock" place where we store those keys:

$> ls -l ~/.local/share/niceman/ec2_keys/      
total 8
-r-------- 1 yoh yoh 1670 Sep  6 14:35 yoh\@hopa-west2.pem
-r-------- 1 yoh yoh 1670 Sep  6 14:19 yoh\@hopa.pem

and use from there... or am I still missing? ;)

[yarikoptic]

@mjtravers could you have a look at this one again? I hate to breed those keys ;)

[yarikoptic]

moreover, ATM if I specify the key_name in the cmdline - it would just try to use it instead of taking that as a name for the key to be created (if does not exist), so I have to go "interactive":

$> niceman --dbg create -t aws-ec2 test-ec2-west -b region_name=us-west-1 image=ami-059e7901352ebaef8 security_group=ssh-only instance_type=t2.medium secret_access_key=$secret_access_key access_key_id=$access_key_id key_name=test-ec2-west10
Traceback (most recent call last):
  File "/home/yoh/proj/repronim/niceman/venvs/dev3/bin/niceman", line 11, in <module>
    load_entry_point('niceman', 'console_scripts', 'niceman')()
  File "/home/yoh/proj/repronim/niceman/niceman/cmdline/main.py", line 251, in main
    ret = cmdlineargs.func(cmdlineargs)
  File "/home/yoh/proj/repronim/niceman/niceman/interface/base.py", line 288, in call_from_parser
    return cls.__call__(**kwargs)
  File "/home/yoh/proj/repronim/niceman/niceman/interface/create.py", line 137, in __call__
    parse_backend_parameters(backend_parameters))
  File "/home/yoh/proj/repronim/niceman/niceman/resource/base.py", line 311, in create
    resource_attrs = resource.create()
  File "/home/yoh/proj/repronim/niceman/niceman/resource/aws_ec2.py", line 134, in create
    instances = self._ec2_resource.create_instances(**create_kwargs)
  File "/usr/lib/python3/dist-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 320, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 624, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidKeyPair.NotFound) when calling the RunInstances operation: The key pair 'test-ec2-west10' does not exist

> /usr/lib/python3/dist-packages/botocore/client.py(624)_make_api_call()
-> raise error_class(parsed_response, operation_name)
(Pdb) c
(dev3) 1 10511 ->1.....................................:Wed 02 Jan 2019 02:20:26 PM EST:.
(git)hopa:~/proj/repronim/niceman[bfs]
$> niceman --dbg create -t aws-ec2 test-ec2-west -b region_name=us-west-1 image=ami-059e7901352ebaef8 security_group=ssh-only instance_type=t2.medium secret_access_key=$secret_access_key access_key_id=$access_key_id                   
You did not specify an EC2 SSH key-pair name to use when creating your EC2
environment.
Please enter a unique name to create a new key-pair or press [enter] to exit: test-ec2-west11

2019-01-02 14:20:42,578 [INFO   ] Created private key file /home/yoh/.local/share/niceman/ec2_keys/test-ec2-west11.pem 
2019-01-02 14:20:43,937 [INFO   ] Waiting for EC2 instance i-0ca892f5b2cd5652a to start running... 
2019-01-02 14:21:14,610 [INFO   ] EC2 instance i-0ca892f5b2cd5652a to start running! 
2019-01-02 14:21:14,610 [INFO   ] Waiting for EC2 instance i-0ca892f5b2cd5652a to complete initialization... 
2019-01-02 14:22:30,568 [INFO   ] EC2 instance i-0ca892f5b2cd5652a initialized! 
2019-01-02 14:22:30,688 [INFO   ] Created the environment test-ec2-west 

[kyleam]

[Based on a quick reading of these posts, I think there's larger issues being discussed here, but ...]

I hate to breed those keys ;)

With the fix from #354, specifying access_key_id and secret_access_key values in niceman.cfg should allow you to reuse the key.

[yarikoptic]

[Based on a quick reading of these posts, I think there's larger issues being discussed here, but ...]

I hate to breed those keys ;)

With the fix from #354, specifying access_key_id and secret_access_key values in niceman.cfg should allow you to reuse the key.

In this issue we are talking about SSH keys. ATM a new key pair generated for each instance, reuse is difficult if not impossible.

[kyleam]

With the fix from #354, specifying access_key_id and secret_access_key values in niceman.cfg should allow you to reuse the key.

In this issue we are talking about SSH keys.

Sorry, I pasted/pruned the wrong fields from my niceman.cfg without thinking. That should read "key_name and key_filename".

[kyleam]

should allow you to reuse the key.

And that "should allow" is phrased a bit weakly. I tested this after #354 and was able to use the configured SSH key without a prompt.