Open brad-charboneau opened 10 months ago
Files identified in the description:
If these files are incorrect, please update the component name
section of the description or use the !component
bot command.
cc @azenk @samdoran click here for bot help
i can provide more context. after testing with onepassword_raw lookup i found that the private key stored in the value field is what is wrong. however the key stored in private_key[ssh_formats][openssh][value] under the private_key field is the correct value. is there a way i can call this field?
here is the example from onepassword_raw for the private_key field
{
'id': 'private_key',
'type': 'SSHKEY',
'label': 'private key',
'value': '-----BEGIN PRIVATE KEY-----\\r\\nMFMCAQEwBQYDK2VwBCIEIBypyf49wp8pQ9S5j9I9pc+aJlABCDEFGcvTEc1MWk0\r\noSMDIQ1234567FXtDA8CzteO1nbLbyQDEWRdpyJtTGDyBp9mzw==\\r\\n-----END PRIVATE KEY-----\\r\\n',
'reference': 'op://SSH Keys/Ansible/private key',
'ssh_formats': {
'openssh': {
'reference': 'op://SSH Keys/Ansible/private key?ssh-format=openssh',
'value': '-----BEGIN OPENSSH PRIVATE KEY-----\\r\\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQBBBBBBBBBBBAAAAMwAAAAtzc2gtZWQyNTUxOQAAACDtxf5nzFXtDA8CzteO1nbLbyQDEWRdpyJtTGDyBp9mzwAAAIiPMEgFjzBIBQAAAAtzc2gtZWQyNTUxFKHJHDtxf5nzFXtDA8CzteO1nbLbyQDEWRdpyJtTGDyBp9mzwAAAEAcqcn+PcKfKUPUuY/SPaXPmiZV9spDcsOHL0xHNTFpNO3F/mfMVe0MDwLO147WdstvJAMRZF2nIm1MYPIGn2bPFFFFFECAwQF\\r\\n-----END OPENSSH PRIVATE KEY-----\\r\\n'
}
}
},
@brad-charboneau I hope that you used a test private SSH key here, or modified it before pasting it here, as otherwise now the whole world knows your private SSH key.
(Edit: I now noticed you mentioned that you adjusted it, so I guess it should be ok.)
@felixfontein yes, all SSH Keys have been Altered before posting
onepassword_raw
returns a dictionary, so you would access fields using Python dictionary syntax. This is a bit fragile, though.
"{{ lookup('onepassword_raw', 'op_item')['fields'][2]['ssh_formats']['openssh']['value'] }}"
But I don't think the lookup plugin is the issue. It is returning the data stored in the specified field correctly.
It seems like maybe the problem is the format of the key file stored in 1Password. I generated a new ED25519 key and stored it in 1Password. It has the same value in both fields.
Note: This is a test SSH private key used for demonstration purposes only.
Again, One password cli returns it correctly. It is just the Ansible module that does not.
From: Sam Doran @.> Sent: Tuesday, October 24, 2023 10:25 AM To: ansible-collections/community.general @.> Cc: Brad Charboneau @.>; Mention @.> Subject: Re: [ansible-collections/community.general] 1password lookup for Private Keys has incorrect key format (Issue #7417)
onepassword_raw returns a dictionary, so you would access fields using Python dictionary syntax. This is a bit fragile, though.
"{{ lookup('onepassword_raw', 'op_item')['fields'][2]['ssh_formats']['openssh']['value'] }}"
This seems to work correctly for me using the onepassword lookup and specifying field, which is a much more reliable way to get the value. It seems like maybe the problem is the format of the key file stored in 1Password. I generated a new ED25519 key and stored it in 1Password. It has the same value in both fields.
Note: This is a test SSH private key used for demonstration purposes only.
OP get item
op item get y2j5ak2k7m2kp6gf47zatft4aq --format json { "id": "y2j5ak2k7m2kp6gf47zatft4aq", "title": "Test SSH Key", "version": 1, "vault": { "id": "stpebbaccrq72xulgouxsk4p7y", "name": "Personal" }, "category": "SSH_KEY", "last_edited_by": "LSGPJERUYBH7BFPHMZ2KKGL6AU", "created_at": "2023-10-24T14:43:43Z", "updated_at": "2023-10-24T14:43:43Z", "additional_information": "SHA256:9LauV2SbRxcyfQfoikMpYLAldDG2WqxJZ2VM8e8j0Pw", "fields": [ { "id": "public_key", "type": "STRING", "label": "public key", "value": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH4lITPIj8pyhx8iipbtfalIlCns7lcvyFGIzmtsKC3r", "reference": "op://Personal/Test SSH Key/public key" }, { "id": "fingerprint", "type": "STRING", "label": "fingerprint", "value": "SHA256:9LauV2SbRxcyfQfoikMpYLAldDG2WqxJZ2VM8e8j0Pw", "reference": "op://Personal/Test SSH Key/fingerprint" }, { "id": "private_key", "type": "SSHKEY", "label": "private key", "value": "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\nQyNTUxOQAAACB+JSEzyI/KcocfIoqW7X2pSJQp7O5XL8hRiM5rbCgt6wAAAJibbQXjm20F\n4wAAAAtzc2gtZWQyNTUxOQAAACB+JSEzyI/KcocfIoqW7X2pSJQp7O5XL8hRiM5rbCgt6w\nAAAEC4N7mv8bv4kYMnoGBpdlKFDbivP0THqG4mE8nj94YbRn4lITPIj8pyhx8iipbtfalI\nlCns7lcvyFGIzmtsKC3rAAAAEXNkb3JhbkByaG1icC0yMDIxAQIDBA==\n-----END OPENSSH PRIVATE KEY-----\n", "reference": "op://Personal/Test SSH Key/private key", "ssh_formats": { "openssh": { "reference": "op://Personal/Test SSH Key/private key?ssh-format=openssh", "value": "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW\nQyNTUxOQAAACB+JSEzyI/KcocfIoqW7X2pSJQp7O5XL8hRiM5rbCgt6wAAAJibbQXjm20F\n4wAAAAtzc2gtZWQyNTUxOQAAACB+JSEzyI/KcocfIoqW7X2pSJQp7O5XL8hRiM5rbCgt6w\nAAAEC4N7mv8bv4kYMnoGBpdlKFDbivP0THqG4mE8nj94YbRn4lITPIj8pyhx8iipbtfalI\nlCns7lcvyFGIzmtsKC3rAAAAEXNkb3JhbkByaG1icC0yMDIxAQIDBA==\n-----END OPENSSH PRIVATE KEY-----\n" } } }, { "id": "key_type", "type": "STRING", "label": "key type", "value": "ed25519", "reference": "op://Personal/Test SSH Key/key type" }, { "id": "notesPlain", "type": "STRING", "purpose": "NOTES", "label": "notesPlain", "reference": "op://Personal/Test SSH Key/notesPlain" } ] }
— Reply to this email directly, view it on GitHub https://github.com/ansible-collections/community.general/issues/7417#issuecomment-1777477458 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AA6YPVXVL3PPEX2EHZLN2XDYA7MTDAVCNFSM6AAAAAA6KMHCYSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONZXGQ3TONBVHA . You are receiving this because you were mentioned. https://github.com/notifications/beacon/AA6YPVWZAJJQBBNEH4SGUSTYA7MTDA5CNFSM6AAAAAA6KMHCYSWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTTJ6ITVE.gif Message ID: @. @.> >
Again, One password cli returns it correctly. It is just the Ansible module that does not.
I wonder if the op
command is returning fields[2]['ssh_formats']['openssh']['value']
instead of fields[2]['value']
by default. I'll take a closer look at that.
Yes, that's exactly what's happening here. I created a new key in 1Password rather than with ssh-keygen
. This is what it looks like:
op item get
Looking through how the data is processed and the plugin interface, there is no way currently to request data from a dictionary in a field (ssh_formats
in this case). Using onepassword_raw
is the best way to get the correct value.
We could special case ssh keys to get the openssh type, but that may cause problems if someone wants the PKCS8 formatted data instead of the OpenSSH private key formatted data.
diff --git plugins/lookup/onepassword.py plugins/lookup/onepassword.py
index a36e6c94e..8b4afff40 100644
--- plugins/lookup/onepassword.py
+++ plugins/lookup/onepassword.py
@@ -453,6 +453,7 @@ class OnePassCLIv2(OnePassCLIBase):
}
"""
data = json.loads(data_json)
+ category = data.get("category")
for field in data.get("fields", []):
if section_title is None:
# If the field name exists in the section, return that value
@@ -461,11 +462,16 @@ class OnePassCLIv2(OnePassCLIBase):
# If the field name doesn't exist in the section, match on the value of "label"
# then "id" and return "value"
- if field.get("label") == field_name:
- return field["value"]
-
- if field.get("id") == field_name:
- return field["value"]
+ values_to_try = ("label", "id")
+ for value in values_to_try:
+ if field.get(value) == field_name:
+ if category == "SSH_KEY":
+ try:
+ return field["ssh_formats"]["openssh"]["value"]
+ except KeyError:
+ pass
+
+ return field["value"]
# Look at the section data and get an indentifier. The value of 'id' is either a unique ID
# or a human-readable string. If a 'label' field exists, prefer that since
or make it a new switch, that change might be surprising to those relying on the current behavior.
yeah such as pass it "open_ssh_key: true" to return open ssh key instead of normal value key
I just generated a new RSA 4096 Test Key and the Open SSH section is still there so it is not limited to ED25519 Keys.
seems like in this situation where Keys are generated by 1password, the RSA Key might be stored in Value, while the ED25519 Key is stored in ssh_formats['openssh']['value']
there is a possibility if we parse key type from the public key value, then we could chose the correct field programmatically:
"id": "public_key", "type": "STRING", "label": "public key", "value": "ssh-ed25519 public_key_string
"id": "public_key", "type": "STRING", "label": "public key", "value": "ssh-rsa public_key_string
value would either start with ssh-ed25519 or ssh-rsa if ed25519 then use openssh field, if rsa then use value filed what do you think?
I would rather avoid a flag argument. There are lots of clever ways this could be done, but I'm sure they will break something for someone.
This seems like a very good use case for the onepassword_raw
plugin, which allows parsing the full data when the regular lookup doesn't return what you need.
Another option would be to create a onepassword_ssh_key
lookup plugin. That would be better than adding a flag argument even though it's more work.
to add a little more context of more oddities:
when I run op item get 'My Test Key' --vault='SSH KEYS' --fields=private_key --reveal
it returns the open_ssh_key. it is only when it has to get it from json that it is incorrect. is this sounding more like a 1passwordcli bug instead of an ansible bug perhaps? for example when i run: op item get 'My Test Key' --vault='SSH KEYS' --fields=private_key --reveal --format=json
the value field the ansible lookup returns is the top value instead of the nested value under open_ssh
human-readable:
op item get 'My Test Key' --vault='SSH KEYS' --fields=private_key --reveal
"
-----BEGIN OPENSSH PRIVATE KEY-----
my_key_data
-----END OPENSSH PRIVATE KEY-----
"
json
op item get 'My Test Key' --vault='SSH KEYS' --fields=private_key --reveal --format=json
{
"id": "private_key",
"type": "SSHKEY",
"label": "private key",
"value": "-----BEGIN PRIVATE KEY-----\r\nmy_key_data\r\n-----END PRIVATE KEY-----\r\n",
"reference": "op://SSH Keys/Ansible/private key",
"ssh_formats": {
"openssh": {
"reference": "op://SSH Keys/Ansible/private key?ssh-format=openssh",
"value": "-----BEGIN OPENSSH PRIVATE KEY-----\r\nmy_key_data\r\n-----END OPENSSH PRIVATE KEY-----\r\n"
}
}
}
It's not a bug in 1Password. 1Password is storing generated keys in PEM formatted PKCS8 since that is the most flexible means of storing private key information. PKCS8 data can easily be converted to other formats such as OpenSSH.
but if the public key that corresponds to the generated key is expecting the ed25519 Openssh format, because the user opted for ed25519 in 1password, then the pem(pkcs8) key is not useful unless the user takes extra steps to convert.
I agree the op
CLI is doing the least surprising thing while the Ansible lookup plugin is being brutally accurate and unpleasantly surprising.
I don't know if this is a bug or a difference in behavior.
It seems the best fix would be to add a onepassword_ssh_key
lookup since processing the data for SSH keys stored in 1Password requires different behavior than other item types.
I have submitted a ticket with 1password on this as well, just to get their take on it before going the route of new development in ansible. Will keep this thread up to date with how that goes.
I'm fairly certain they will state that storing the key in PKCS8 format is intentional.
Files identified in the description:
If these files are incorrect, please update the component name
section of the description or use the !component
bot command.
I agree the
op
CLI is doing the least surprising thing while the Ansible lookup plugin is being brutally accurate and unpleasantly surprising.I don't know if this is a bug or a difference in behavior.
It seems the best fix would be to add a
onepassword_ssh_key
lookup since processing the data for SSH keys stored in 1Password requires different behavior than other item types.
if someone does create a new module for ssh key lookup, might i suggest using these commands as the intended targets?
ED25519: op read "op://SSH Keys/Ansible/private key?ssh-format=openssh" PKCS8: op read "op://SSH Keys/Ansible/private key"
and Public Key: op read "op://SSH Keys/Ansible/public key"
you cannot pass anything other than openssh to the ssh-format query.
Summary
``issue is not with 1password-cli as the cli is able to return the correct value. Only Ansible returns an incorrect value.
Ansible returns the incorrectly formatted SSH Private Key from One Password Vault
Issue Type
Bug Report
Component Name
onepassword
Ansible Version
Community.general Version
Configuration
OS / Environment
Debian 12
Steps to Reproduce
In my use case I use ssh-ed25519 SSH Keys
Expected Results
it should return the correct SSH Private Key Contents. stating with -----BEGIN OPENSSH PRIVATE KEY----- and end with -----END OPENSSH PRIVATE KEY----- and be the valid ed25519 Key
example output from op item get --vault "op_vault" op_item --reveal
(changed many pieces in the output to prevent releasing a real private key)
Actual Results
Code of Conduct