search

tpm2-software / tpm2-pytss

Python bindings for TSS
https://tpm2-pytss.readthedocs.io/en/latest/
BSD 2-Clause "Simplified" License
65 stars 45 forks source link

Read endorsement key with the usage of tpm2-pytss #350

Closed Dvergatal closed 2 years ago

Dvergatal commented 2 years ago

Hi all, just like in the subject I wanted to ask you guys is it possible to read/retrieve the endorsement key - public part - from the TPM with this API?

Moreover, if we are in the subject of endorsement key I have another question to you, because I am currently struggling with Microsoft Azure and its tpm_device_provision binary which is used for DPS provisioning (it retrieves Registration id and Endorsement Key). On Microsoft Azure documentation sites it is written, that this binary retrieves public part of this RSA TPM key, but when i wanted to decode this base64 with the usage of openssl, it says, that this is not a public key.

In our tests we are using SWTPM software TPM implementation on qemu machine and according to the SWTPM documentation it creates it's own ek_key during TPM creation. Question is, if this can be rubbish or does it have some completely different format? I can also paste you here this TPM ek_pub:

Endorsement Key:
AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAq4+zaAAAv0nsuYwCm+XzzdldDYDWR+565eLh9E2PUe9x9oEn+9xsGS5z2mzj=

Moreover I have observed, that it starts completely differently, than any other RSA public key created by openssl.

whooo commented 2 years ago

For getting the EK you can use https://tpm2-pytss.readthedocs.io/en/1.1.0/utils.html#tpm2_pytss.utils.create_ek_template Check the four first lines of https://github.com/tpm2-software/tpm2-pytss/blob/707c23f91381b8f3f40cfacb1824314b9ecf6753/test/test_utils.py#L487 for a basic usage example

Regarding the base64 output from tpm_device_provision, I don't know what format that is, but it is to small to be a a RSA key and doesn't seem to be a key template either

Dvergatal commented 2 years ago

@whooo thx ok so this part:

        nv_read = NVReadEK(self.ectx)
        _, rsa_template = create_ek_template("EK-RSA2048", nv_read)
        _, rsa, _, _, _ = self.ectx.create_primary(
            TPM2B_SENSITIVE_CREATE(), rsa_template, ESYS_TR.ENDORSEMENT
        )
        self.assertEqual(rsa.publicArea.type, TPM2_ALG.RSA)

should give me this ek_pub, right?

Regarding the base64 output from tpm_device_provision, I don't know what format that is, but it is to small to be a a RSA key and doesn't seem to be a key template either

According to the documentation this should be public part of the key but still it is some how strange because common rsa public key starts completely different i.e.:

MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDigN4obCtg61Knvqp9e78RQ02P
ihxs9HBwOabgj9MjxJAwtspdls7OEn4qmQvca4AyWPgUUc0rc1/Kz56dEIWsFGtg
NvIz1Q67hE3uKyRmzJfmnYQPYTy60y/2hfnG/0FcxczI4ciTsnRomPoCYi8Usfhh
lvqLXjdadfNME5cN9QIDAQAB

From what i remember it always starts with MI letters.

whooo commented 2 years ago

@whooo thx ok so this part:

        nv_read = NVReadEK(self.ectx)
        _, rsa_template = create_ek_template("EK-RSA2048", nv_read)
        _, rsa, _, _, _ = self.ectx.create_primary(
            TPM2B_SENSITIVE_CREATE(), rsa_template, ESYS_TR.ENDORSEMENT
        )
        self.assertEqual(rsa.publicArea.type, TPM2_ALG.RSA)

All but the self.assertEqual part yes

should give me this ek_pub, right?

Regarding the base64 output from tpm_device_provision, I don't know what format that is, but it is to small to be a a RSA key and doesn't seem to be a key template either

According to the documentation this should be public part of the key but still it is some how strange because common rsa public key starts completely different i.e.:

MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDigN4obCtg61Knvqp9e78RQ02P
ihxs9HBwOabgj9MjxJAwtspdls7OEn4qmQvca4AyWPgUUc0rc1/Kz56dEIWsFGtg
NvIz1Q67hE3uKyRmzJfmnYQPYTy60y/2hfnG/0FcxczI4ciTsnRomPoCYi8Usfhh
lvqLXjdadfNME5cN9QIDAQAB

From what i remember it always starts with MI letters.

Public part might mean a TPM structure, not a "raw" RSA key, but regardless the example you showed is to small for a RSA key, might be a ECC key tho

Dvergatal commented 2 years ago

@whooo big thx again. Hmm as it is a software TPM, it might be possible, that this is in fact ECC key, more over, it is possible that this is a TPM structure. I have to check tpm_device_provision source code and verify it.

Dvergatal commented 2 years ago

@whooo you were right the endorsment key is much longer and looks like that:

AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAq4+zaAAAv0nsuYwCm+XzzdldDYDWR+565eLh9E2PUe9x9oEn+9xsGS5z2mzj1YY6xUvMkN6pFd8gv2H0d1KBdqyruGSH8qP35QDwLI7t0jcEeeemDJ5te/A92Z+yfzLDrGqypz064VhP3Qrha/zhF5g+crZpOcnlshwsnVNh7awcqNjk0ki5QrmaBbiI9g/D9xv/gFqGXdhkGwaSTnJeBWRefmrXhtF+iGwCRffEjlG/qsxQ/hvRbeAdybwQk+cysdSk4w9jTMrsf7b8AoYBOlkrGSTy2mzk1F2ePJEr2W+gayYDVNLOa3pagnxtkoz0ER2F5BzFS+glrgl6rOjE1w==

My console was cutting it and i didn't noticed it.

whooo commented 2 years ago

I did a b64decode on the data and was able to unmarshal it as a TPM2B_PUBLIC structure

Dvergatal commented 2 years ago

So it is indeed an endorsement public key but in tpm format:] glad to here it.

williamcroberts commented 2 years ago

So it is indeed an endorsement public key but in tpm format:] glad to here it.

Once you have it in a TPM2B_PUBLIC you can call .to_pem on it to get a PEM representation of the public key to use in OpenSSL. For an example, look at: https://github.com/tpm2-software/tpm2-pytss/blob/master/test/test_crypto.py#L332

Dvergatal commented 2 years ago

@williamcroberts thx, but what I want to do is to remove the usage of this microsoft bullshit program.... I was digging if, that is really the key, because I wanted to verify, what was returning this application.

Dvergatal commented 2 years ago

@williamcroberts and is there equivalent method to get base64 like i have given as an example?

Dvergatal commented 2 years ago

BTW. this is my code:

from tpm2_pytss import *
from tpm2_pytss.internal import *
from base64 import b64decode
from hashlib import sha256, sha384

nv_read = NVReadEK(self.ectx)
_, rsa_template = create_ek_template("EK-RSA2048", nv_read)
_, rsa, _, _, _ = self.ectx.create_primary(TPM2B_SENSITIVE_CREATE(), rsa_template, ESYS_TR.ENDORSEMENT)

and I'm getting error:

Traceback (most recent call last):
  File "endorsement_key.py", line 6, in <module>
    nv_read = NVReadEK(self.ectx)

What can cause it? Because according to https://tpm2-pytss.readthedocs.io/en/latest/utils.html NVReadEK is in utils which is in tpm2_pytss, but I'm importing from it * so this should be working.

whooo commented 2 years ago

BTW. this is my code:

from tpm2_pytss import *
from tpm2_pytss.internal import *
from base64 import b64decode
from hashlib import sha256, sha384

nv_read = NVReadEK(self.ectx)
_, rsa_template = create_ek_template("EK-RSA2048", nv_read)
_, rsa, _, _, _ = self.ectx.create_primary(TPM2B_SENSITIVE_CREATE(), rsa_template, ESYS_TR.ENDORSEMENT)

and I'm getting error:

Traceback (most recent call last):
  File "endorsement_key.py", line 6, in <module>
    nv_read = NVReadEK(self.ectx)

What can cause it? Because according to https://tpm2-pytss.readthedocs.io/en/latest/utils.html NVReadEK is in utils which is in tpm2_pytss, but I'm importing from it * so this should be working.

You shouldn't need to import anything from tpm2_pytss.internal. the utils aren't included in the wildcard import from tpm2_pytss, so do from tpm2_pytss.utils import NVReadEK, create_ek_template When that is working, do something like:

rb = rsa.marshal()
b64_rsa = b64encode(rb)

And then you have the base64 encoded public part

Dvergatal commented 2 years ago

@whoo thx btw. i didn't notice that the example you have given is within a TestUtils class, which inherits from TSS2_BaseTest and i see that i need to make first this setup which is in it for setting this ectx?

whooo commented 2 years ago

This works for me:

from tpm2_pytss.ESAPI import ESAPI                                                                                                                                                                                                     
from tpm2_pytss.utils import NVReadEK, create_ek_template                                                                                                                                                                              
from tpm2_pytss.types import TPM2B_SENSITIVE_CREATE                                                                                                                                                                                    
from tpm2_pytss.constants import ESYS_TR                                                                                                                                                                                               
from base64 import b64encode                                                                                                                                                                                                           

with ESAPI() as ectx:                                                                                                                                                                                                                  
    nv_read = NVReadEK(ectx)                                                                                                                                                                                                           
    _, templ = create_ek_template("EK-RSA2048", nv_read)                                                                                                                                                                               
    _, pub, _, _, _ = ectx.create_primary(TPM2B_SENSITIVE_CREATE(), templ, ESYS_TR.ENDORSEMENT)                                                                                                                                        
    pb = pub.marshal()                                                                                                                                                                                                                 
    b64pub = b64encode(pb)                                                                                                                                                                                                             
    print(b64pub)
Dvergatal commented 2 years ago

Yeah big thx i have to learn more about this API, but I have also errors in here:

WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_NV_ReadPublic.c:309:Esys_NV_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:209:Esys_TR_FromTPMPublic_Finish() Error NV_ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b) 
WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_NV_ReadPublic.c:309:Esys_NV_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:209:Esys_TR_FromTPMPublic_Finish() Error NV_ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b) 
WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_NV_ReadPublic.c:309:Esys_NV_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:209:Esys_TR_FromTPMPublic_Finish() Error NV_ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b)

P.S. But the public base64 key is printed perfectly fine P.S.2 This error is on qemu linux image with SWTPM

whooo commented 2 years ago

The errors are due to how logging is configured for tpm2-tss and how NVReadEK works, I don't have any real workaround for that besides disabling parts of the logging. Basically NVReadEK tries to read from a certain set of NV areas (the cert, the template and the nonce), none of the is really required for getting the EK, but if they are defined they affect the EK template logic so the code tries to read from them and as you are using a simulator none of the NV areas are defined so you get three groups of errors/warnings

Dvergatal commented 2 years ago

Ahhh i see, so basically on a physical machine i shouldn't get these errors. It's completely OK for me if it is only on qemu machine :] Big thx @whooo for support.

whooo commented 2 years ago

you would still get some of the errors, I haven't seen any TPM that has all three NV areas defined

Dvergatal commented 2 years ago

Ah OK sure and it is not possible to somehow turn off these errors?

whooo commented 2 years ago

trying setting the TSS2_LOG environment variable to "esys+none"

Dvergatal commented 2 years ago

Nice:) it works thx. Btw. I also need to calculate sha256 hash from the public key for the registration id in the azure, but first I will verify in the C source code of the Microsoft tpm_device_provision from what variable, it is being calculated...

whooo commented 2 years ago

if it's the name you are after, bytes(pub.get_name()) to get the digest of the public part

Dvergatal commented 2 years ago

@whooo you mean to calculate the hash from this bytes?

Dvergatal commented 2 years ago

Ahhh i was wondering why this does not look the same with the result being returned from tpm_device_provision binary and I've found that they are making also some encoding stuff from this sha256 to base32.

I will do step by step analyze of this code. Hope I will succeed with it:]

Dvergatal commented 2 years ago

OK i have printed sha256 hash calculated by this MS app and it is equal 4A:6B:93:C3:E2:4C:68:AC:1E:FC:C7:27:0D:FD:AE:1E:02:E3:BA:CC:74:DD:F3:F0:E3:35:C6:45:7E:2B:3E:B3 and from the code given by you which i have adjusted:

output = hashlib.sha256(bytes(pub.get_name())).hexdigest()

I'm gettin 3c0ecf0b1570498edb2ebd68b1b79a6a75914a014c0f657930e646427ef38411 which is not equal...

whooo commented 2 years ago

No need to hash the name, for keys and NV areas it's already the a digest of the nameAlg

Dvergatal commented 2 years ago

OK but pub.get_name() is returning me 000bfd906c806f9391b1489fd67c6c4dd92c831e44b1ce59c51aa84d5ce8bbe8caf9 and I need sha256 calculated from ek_pub.

Dvergatal commented 2 years ago

Ah i found something more problematic, which i haven't noticed before is that the returned endorsement key value is slight different than the one returned from python code.

This is from tpm_device_provision AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAq4+zaAAAv0nsuYwCm+XzzdldDYDWR+565eLh9E2PUe9x9oEn+9xsGS5z2mzj1YY6xUvMkN6pFd8gv2H0d1KBdqyruGSH8qP35QDwLI7t0jcEeeemDJ5te/A92Z+yfzLDrGqypz064VhP3Qrha/zhF5g+crZpOcnlshwsnVNh7awcqNjk0ki5QrmaBbiI9g/D9xv/gFqGXdhkGwaSTnJeBWRefmrXhtF+iGwCRffEjlG/qsxQ/hvRbeAdybwQk+cysdSk4w9jTMrsf7b8AoYBOlkrGSTy2mzk1F2ePJEr2W+gayYDVNLOa3pagnxtkoz0ER2F5BzFS+glrgl6rOjE1w==

and this is the value from python code AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAnjwGiAAAdMj4D/FS7NdZkzhgEb+zP3Y3GZXUUvy5eakk1zxNzy1+Ok55JDHgaVwLOKM+vqV72PuLoEpLHVWYty+emITUx2R8Vp/G/ojI+M8v7HbSlYlW6TPoynpqdHjxLfwu8YLJvrJSpS4SRqJZEetYXT+ZttFeFiGYVgsjtBioiiP1pw94YjzQ0Zr055dImEYSEhCV9AOfl2kmWBUo9W/mQisJBwoNuhKTq8U69FbWF4h3O47LiVmilw+11GfTGVOj2UUR7z35GFdJ/6ClsX8zfDxQvt7il2QVR2uGi6R9SR2BM3iZzei79hzMOh0Xr0Pv+UGlDTtF71jX2FMM3w==

As you can see it starts the same but at some point it differs.

Dvergatal commented 2 years ago

No need to hash the name, for keys and NV areas it's already the a digest of the nameAlg

I found that in the code that nameAlg by default is SHA256 so i get it now:D

Dvergatal commented 2 years ago

Hahahahah now it has been fixed:D the swtpm_setup command has not been started in the docker container with all configuration and thus is why these rubbish data were being produced.... The endorsement keys are now the same. Need also to verifiy this registration id.

Dvergatal commented 2 years ago

Hahahahah now it has been fixed:D the swtpm_setup command has not been started in the docker container with all configuration and thus is why these rubbish data were being produced.... The endorsement keys are now the same. Need also to verifiy this registration id.

Hi @whooo. Unfortunately this is not a solution. I have returned to this problem, while working with physical device. The python code returns me completely different value for the endorsement key.

I was also wondering about verifying this outputs with some other applications or even writing own code with the usage of tss api and I have found this https://github.com/tpm2-software/tpm2-tools/blob/master/man/tpm2_getekcertificate.1.md. I have managed to retrive EK certificate using tpm2_getekcertificate -o ECcert.bin command and read it using openssl x509 -in ECcert.bin -inform der -noout -text and it is a common certificate with all attributes, but I am able to verify is only comparison between the output from this python code and tpm_device_provision binary from azure.

Additionally, how would it be to decode these keys as soon as possible to verify the differences?

whooo commented 2 years ago

tpm2_getekcertificate gets the EK certificate, while the example I provided and what azure seems to do is get the EK as a TPM 2.0 public part key. Done right both represent the same key but with different metadata and in different formats, it's better to compare it to tpm2_createek then tpm2_getekcertificate.

Dvergatal commented 2 years ago

@whooo yes this is also what i have read in microsoft documentation but still this is really strange for me that these values differs i.e. these are the outputs for the physicall device:

 root@eg ~> python3 tpm.py
WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_NV_ReadPublic.c:309:Esys_NV_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:209:Esys_TR_FromTPMPublic_Finish() Error NV_ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b) 
WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_NV_ReadPublic.c:309:Esys_NV_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:209:Esys_TR_FromTPMPublic_Finish() Error NV_ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b) 
('hch6u236c4a4hi7w3m7iviz5gahxnmn5mf62swnqskenqhbklvra', 'AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAgvAE8heVwv60baEy5j8XvH7tQNeR/zTuQvCsKDhVacbDugAsLR3KfdHDrLscygZu4eB0p1vTNvT40t82gj2/AnMp/yxrsMWYftPNcZBGSKH3baPrtcxclWFhqxwCPqjO5uvDkz3qGt3024ErA7WamzrBWWnw8ejv95BennOILMeFafgVGVZoX0E1XOi5u8FuZbDWjxN6bLoDkRcWpw2IrWUy1rVGcIbgmkGDi63JxgBl2lsULiBhs1mWamA6mmLiIGryqZVjIw4sL6//E06hXYout37VOtFrs9mmI3hFHn8wRArE3ppLcPOkr3amiiLZD18m+crLWWL3JNY2eqPTpQ==')
root@eg ~> tpm_device_provision 
Gathering the registration information...

Registration Id:
exvjykmny25qynh3oarky73memadol7cegm4i7yflrbtwl67ipxa

Endorsement Key:
AToAAQALAAMAsgAgg3GXZ0SEs/gakMyNRqXXJP1S124GUgtk8qHaGzMUaaoABgCAAEMAEAgAAAAAAAEAtiUlyTyT7pqQKgvaUOWUT3Ma3kKV0Euq1L5BGhrkZ1ek4YEvmfhceKNPhu9JcRKtec3zM4o0m+b/vQF3F4qC8I1/nciHNHCO4xAsl6VnzgOOWKQTQdwwUrwnNL75LeVXHyKPxQ8D/f5U+dyCPLu6jk3ZowReC7PPbPstJk2Fr2Qx3mdyHsdVWOu5zD1HMK7YcyJsfDle0jfOi3OMUEdZ/IbGBQxBaDs/nje+P+NsH1PhG+UX5K97teiu2tPakieKBtllhgjTxHqBjrKU9oE4TMNi2jfIKru5wAq/OvdIbLAPa/NMplCUYHwQvtLh9l8ooePk51JJ0mirNkZdym/caw==

Press any key to continue:

root@eg ~>

which I really do not understand.

Btw. what do you mean in this sentence?

Done right both represent the same key but with different metadata and in different formats, it's better to compare it to tpm2_createek then tpm2_getekcertificate.

Dvergatal commented 2 years ago

Btw. what do you mean in this sentence?

Done right both represent the same key but with different metadata and in different formats, it's better to compare it to tpm2_createek then tpm2_getekcertificate.

Ah you mean that one is a certificate and the other one a tss structure. Yeah I get it:] this is what I meant from the beginning and that is why I have written it in first place, that I wanted to use it, but this was a stupid idea.

Btw. I am trying to use this tpm2_createek, but I do not fully understand this manual for it.

Dvergatal commented 2 years ago

@whooo so this is:

tpm2_createek --ek-context rsa_ek.ctx  --key-algorithm rsa --public rsa_ek.pub

how I should get the public value right?

Dvergatal commented 2 years ago

@whooo ok i have succeded with tpm2_createk and the command above. I have also encoded it's value into base64:

base64 -w0 rsa_ek.pub >rsa_ek.pub.base64

and the output of it, is the same as from python code. So question is what is the difference and where it comes from.

whooo commented 2 years ago

Could you run tpm2_nvreadpublic on the machine you are testing on and give me the output? The key template seems to be the same, so the only thing I can think of is that the azure iot tools adds some kind of nonce to the key creation

Dvergatal commented 2 years ago

@whooo here you go:

0x1c00002:
  name: 000bec00c657a4e2724101954c2c9d51ddd45c825c3997ec0786c3afeb0f7fca3ec7
  hash algorithm:
    friendly: sha256
    value: 0xB
  attributes:
    friendly: ppwrite|writedefine|ppread|ownerread|authread|no_da|written|platformcreate
    value: 0x1200762
  size: 1177

0x1c0000a:
  name: 000b2571404112c8aae1cde797c438d921093fc89b74d44564c25c296aaa26a6f041
  hash algorithm:
    friendly: sha256
    value: 0xB
  attributes:
    friendly: ppwrite|writedefine|ppread|ownerread|authread|no_da|written|platformcreate
    value: 0x1200762
  size: 781
Dvergatal commented 2 years ago

@whooo I have forgotten to mention that, if I use tpm2_clear the azure tpm_device_provision takes about minute period time, so I suspect that you are absolutely right about that:

The key template seems to be the same, so the only thing I can think of is that the azure iot tools adds some kind of nonce to the key creation

But the question is, if they do such thing, will the key returned by the python code or tpm2_createek be sufficient? On saturday, as I already told you, we had problems with registration of these keys (the qemu version and also with physical devices). Yesterday, they were registered, so there might be azure problem as well during the weekend.

Nevertheless, I would like to discover, what this azure iot tool is doing. Moreover, I do not fully understand one thing, because endorsement key is burned during tpm production by the factory and it is a private asymmetric key from which user/owner of the tpm has access only to public part of it, which is what your example and tpm2_createek does. Now my question is, how is it even possible to add to it, some kind of nonce?

whooo commented 2 years ago

The endorsement key is not really burnt in, the endorsement certificate might be. All primary keys are generated by a KDF which has some inputs, which includes the templates and a per hierarchy seed, if you create a primary key under the endorsement hierarchy with a non standard template it will not match any endorsement certificates. I think you should reach out to the Azure IOT SDK team to understand how their tooling works, as both the python code and tpm2_createek generates the same endorsement key it don't think it's a bug in tpm2-pytss

Dvergatal commented 2 years ago

Ok @whooo, now I get the picture. Moreover as I have said before:

Hahahahah now it has been fixed:D the swtpm_setup command has not been started in the docker container with all configuration and thus is why these rubbish data were being produced.... The endorsement keys are now the same. Need also to verifiy this registration id.

So, another question is, if the nonce is written to the TPM? If yes than probably that is why the tpm_device_provision returns the same value for the endorsement key (in case of qemu with swtpm), because it knows that it was generated, moreover this

@whooo I have forgotten to mention that, if I use tpm2_clear the azure tpm_device_provision takes about minute period time, so I suspect that you are absolutely right about that:

The key template seems to be the same, so the only thing I can think of is that the azure iot tools adds some kind of nonce to the key creation

also suggests it. Is it possible somehow, that we could verify it, with python code, to generate the nonce for the creation of endorsement key?

Dvergatal commented 2 years ago

The endorsement key is not really burnt in, the endorsement certificate might be. All primary keys are generated by a KDF which has some inputs, which includes the templates and a per hierarchy seed, if you create a primary key under the endorsement hierarchy with a non standard template it will not match any endorsement certificates. I think you should reach out to the Azure IOT SDK team to understand how their tooling works, as both the python code and tpm2_createek generates the same endorsement key it don't think it's a bug in tpm2-pytss

@whooo after digging their code, I have a little bit more knowledge. So first of all they are generating two persistent keys. The first one is Endorsement Key and the second one is Storage Root Key. I have also found templates for them (I'm sorry for the code in c, but it shouldn't be a problem). For the ek it looks like that:

static TPM2B_PUBLIC* GetEkTemplate ()
{
    static TPM2B_PUBLIC EkTemplate = { 0,   // size will be computed during marshaling
    {
        TPM_ALG_RSA,                    // TPMI_ALG_PUBLIC      type
        TPM_ALG_SHA256,                 // TPMI_ALG_HASH        nameAlg
        { 0 },                          // TPMA_OBJECT  objectAttributes (set below)
        {32,
        { 0x83, 0x71, 0x97, 0x67, 0x44, 0x84, 0xb3, 0xf8,
        0x1a, 0x90, 0xcc, 0x8d, 0x46, 0xa5, 0xd7, 0x24,
        0xfd, 0x52, 0xd7, 0x6e, 0x06, 0x52, 0x0b, 0x64,
        0xf2, 0xa1, 0xda, 0x1b, 0x33, 0x14, 0x69, 0xaa }
        },                              // TPM2B_DIGEST         authPolicy
        { 0 },                          // TPMU_PUBLIC_PARMS    parameters (set below)
        { 0 }                           // TPMU_PUBLIC_ID       unique
    } };
    EkTemplate.publicArea.objectAttributes = ToTpmaObject(
        Restricted | Decrypt | FixedTPM | FixedParent | AdminWithPolicy | SensitiveDataOrigin);
    EkTemplate.publicArea.parameters.rsaDetail = RsaStorageParams;
    return &EkTemplate;
}

and for srk:

static TPM2B_PUBLIC* GetSrkTemplate()
{
    static TPM2B_PUBLIC SrkTemplate = { 0,  // size will be computed during marshaling
    {
        TPM_ALG_RSA,                // TPMI_ALG_PUBLIC      type
        TPM_ALG_SHA256,             // TPMI_ALG_HASH        nameAlg
        { 0 },                      // TPMA_OBJECT  objectAttributes (set below)
        { 0 },                      // TPM2B_DIGEST         authPolicy
        { 0 },                      // TPMU_PUBLIC_PARMS    parameters (set before use)
        { 0 }                       // TPMU_PUBLIC_ID       unique
    } };
    SrkTemplate.publicArea.objectAttributes = ToTpmaObject(
        Restricted | Decrypt | FixedTPM | FixedParent | NoDA | UserWithAuth | SensitiveDataOrigin);
    SrkTemplate.publicArea.parameters.rsaDetail = RsaStorageParams;
    return &SrkTemplate;
}

In addition I have noticed that they are creating these keys under specified addresses, ek is at 0x81010001 and srk at 0x81000001, so I have used tpm2_readpublic to read them and finally I got my answers:D 

root@eg ~> tpm2_readpublic -c 0x81010001  -o output.dat -f pem -t primary.handle
name: 000b3a5d31f597c0ffe0b62500e88b5a058cb903e046d2599bfb92a0c8be96b506ef
qualified name: 000b1eeda0d707eed7b6dfb88b591b03be5678e3c6ada2bc9325789b095740a87896
name-alg:
  value: sha256
  raw: 0xb
attributes:
  value: fixedtpm|fixedparent|sensitivedataorigin|adminwithpolicy|restricted|decrypt
  raw: 0x300b2
type:
  value: rsa
  raw: 0x1
exponent: 65537
bits: 2048
scheme:
  value: null
  raw: 0x10
scheme-halg:
  value: (null)
  raw: 0x0
sym-alg:
  value: aes
  raw: 0x6
sym-mode:
  value: cfb
  raw: 0x43
sym-keybits: 128
rsa: b62525c93c93ee9a902a0bda50e5944f731ade4295d04baad4be411a1ae46757a4e1812f99f85c78a34f86ef497112ad79cdf3338a349be6ffbd0177178a82f08d7f9dc88734708ee3102c97a567ce038e58a41341dc3052bc2734bef92de5571f228fc50f03fdfe54f9dc823cbbba8e4dd9a3045e0bb3cf6cfb2d264d85af6431de67721ec75558ebb9cc3d4730aed873226c7c395ed237ce8b738c504759fc86c6050c41683b3f9e37be3fe36c1f53e11be517e4af7bb5e8aedad3da92278a06d9658608d3c47a818eb294f681384cc362da37c82abbb9c00abf3af7486cb00f6bf34ca65094607c10bed2e1f65f28a1e3e4e75249d268ab36465dca6fdc6b
authorization policy: 837197674484b3f81a90cc8d46a5d724fd52d76e06520b64f2a1da1b331469aa
root@eg ~> tpm2_readpublic -c 0x81000001  -o output.dat -f pem -t primary.handle
name: 000bcc53626cf97ae5799adba7dfa52206857f4c87af1f070ace5f46804a69d3e152
qualified name: 000ba879ac6817c5fe65d5259e92392bc9ca0a7af9ccd25636248225ff3e924b39a0
name-alg:
  value: sha256
  raw: 0xb
attributes:
  value: fixedtpm|fixedparent|sensitivedataorigin|userwithauth|noda|restricted|decrypt
  raw: 0x30472
type:
  value: rsa
  raw: 0x1
exponent: 65537
bits: 2048
scheme:
  value: null
  raw: 0x10
scheme-halg:
  value: (null)
  raw: 0x0
sym-alg:
  value: aes
  raw: 0x6
sym-mode:
  value: cfb
  raw: 0x43
sym-keybits: 128
rsa: d392079163ddc12b8d22cce0f1c5be47f4379df7ac72f1e56d11ddef26c720f1eb8f4a289b920bafd6b75517876b69965142a67ee4d8320a41a48bcead8ddff4017ad6282e721cbce1bf262f2cc4ae1e4dd8e7360871e1d54bd7ae904660c1cc8e0e0d8cf577d5939c53f90d2d8cc0066153f338882a4ec61cd99cffbcdd8a52a8f58d03f5ebfdd0fcad92755db481db38ef7c07b55be0e52307a686ef5bff8a8acc62d902d688b795ddb15856f2e25724eedd5983558b0fa886986cd2def58448005ff1492d3cda67fe243520973dadd1f38148d817c8eb337ef61351c34db362dd92acd88561fd5cf5c93747b0aebcefa9b264c17198aca87e30ada1f80cb9

As you can see the authorization policy for the ek is the same. Probably this is missing in python code, as well as generation of srk.

I have also used tpm2_clear to verify, if it would be deleted and yes it has been. ~Additionally these keys are always different.~ Correction only SRK differs, EK is always the same, probably it is due to how this template looks.

whooo commented 2 years ago

The authorization_policy is the same in the python code, if both the python code tpm2_createek creates the same key the same issue would be in both the tools and the python code, which I suspect it doesn't Can you paste tpm.py

Dvergatal commented 2 years ago

@whooo:

import sys
from typing import Tuple
from hashlib import sha256
from base64 import b64encode, b32encode
from tpm2_pytss.ESAPI import ESAPI
from tpm2_pytss.utils import NVReadEK, create_ek_template
from tpm2_pytss.types import TPM2B_SENSITIVE_CREATE, TPM2B_PUBLIC
from tpm2_pytss.constants import ESYS_TR

def get_data_from_tpm_module() -> Tuple[str, str]:

    def create_ek_pub() -> TPM2B_PUBLIC:
        with ESAPI() as ectx:
            nv_read = NVReadEK(ectx)
            _, templ = create_ek_template("EK-RSA2048", nv_read)
            _, pub, _, _, _ = ectx.create_primary(TPM2B_SENSITIVE_CREATE(), templ, ESYS_TR.ENDORSEMENT)
            return pub

    pub = create_ek_pub()
    hash_function = sha256()
    hash_function.update(pub.marshal())
    reg_id = b32encode(hash_function.digest()).replace(b'=', b'').decode().lower()
    end_key = b64encode(pub.marshal()).decode()

    return reg_id, end_key

print(get_data_from_tpm_module())

Btw. I have verified one thing, that if I use this code than this command:

tpm2_readpublic -c 0x81010001 -o ek_pub.pem -f pem -t primary.handle

this command does not return me the key but only error:

WARNING:esys:../tpm2-tss-3.2.0/src/tss2-esys/api/Esys_ReadPublic.c:320:Esys_ReadPublic_Finish() Received TPM Error 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:230:Esys_TR_FromTPMPublic_Finish() Error ReadPublic ErrorCode (0x0000018b) 
ERROR:esys:../tpm2-tss-3.2.0/src/tss2-esys/esys_tr.c:320:Esys_TR_FromTPMPublic() Error TR FromTPMPublic ErrorCode (0x0000018b) 
ERROR: Esys_TR_FromTPMPublic(0x18B) - tpm:handle(1):the handle is not correct for the use
ERROR: Unable to run tpm2_readpublic

Moreover the tpm_device_provision takes some time to generate their EK and SRK keys if they aren't any on the addresses I have given you yesterday and python code together with tpm2_createek is really fast.

Dvergatal commented 2 years ago

@whooo nevertheless these EK values are different I was also wondering if it really matters, because I am able to register it but the connection isn't able to be establish due to missing SRK. Could we do some test also to generate SRK in this python code for this EK in order to verify if my assumptions are OK?

Dvergatal commented 2 years ago

@whooo OK I have done one more test, which was to confirm my reasoning, meaning I have created EK with the usage of this command:

tpm2_createek -c 0x81010001 -G rsa -u ek.pub

than I have used tpm_device_provision just to generate SRK - it will not regenerate EK because it has been already generated by the tpm2_createek and I have confirmed that in my earlier posts, besides tpm_device_provision returns this new value.

With this additional step of SRK creation, I was now able to establish a connection with Azure cloud :) So I am wondering now how I can do this (make EK persistent, generate SRK and make it also persistent) with pytss?

williamcroberts commented 2 years ago

@whooo OK I have done one more test, which was to confirm my reasoning, meaning I have created EK with the usage of this command:

tpm2_createek -c 0x81010001 -G rsa -u ek.pub

than I have used tpm_device_provision just to generate SRK - it will not regenerate EK because it has been already generated by the tpm2_createek and I have confirmed that in my earlier posts, besides tpm_device_provision returns this new value.

With this additional step of SRK creation, I was now able to establish a connection with Azure cloud :) So I am wondering now how I can do this (make EK persistent, generate SRK and make it also persistent) with pytss?

Dvergatal commented 2 years ago 
* Make EK Persistent - use ESAPI.EvictControl

  * You can also create it with ESAPI.CreatePrimary with the proper template or use the helper create_ek_template.
  * If you use create_ex_template from utils, I think the proper template is "EK-RSA2048" and you can set the Callable to class NVReadEK or even your own class that on call just returns NoSuchIndex exception but it will be less portable on the off chance the TPM defines a template at an NV index.

As @whooo has given code sample, which I'm using and it uses ESAPI.CreatePrimary with template from create_ek_template and Callable to class NVReadEK it doesn't make EK persistent.

Moreover, how can I first read that EK public, to verify if it is already persistent? I'm trying to use read_public, but there is not persistent_handle like in tpm2_readpublic -c 0x81010001 -o ek_pub.pem -f pem -t primary.handle command, but only variable object_handle, which I'm setting to ESYSTR.ENDORSEMENT i.e. `pub, , _ = ectx.read_public(ESYS_TR.ENDORSEMENT)but it throws an exceptiontpm2_pytss.TSS2_Exception.TSS2_Exception: tpm:handle(1):value is out of range or is not correct for the context`

whooo commented 2 years ago

Use https://tpm2-pytss.readthedocs.io/en/latest/esys.html#tpm2_pytss.ESAPI.tr_from_tpmpublic to get an ESYS_TR handle from the persistent handle, then you can pass that to read_public

Dvergatal commented 2 years ago

Ok I have found informations how to do that in issue #298 and niooss-ledger is right, that it is not as straightforward with ESAPI.read_public function.

P.S. @whooo I haven't noticed that you have already written the solution :P