Closed embetrix closed 8 months ago
The pkcs11 TA heap (where the TA allocates memory) default size is 32kBytes.
It can be tuned with CFG_PKCS11_TA_HEAP_SIZE=<size-in-bytes>
.
Indeed 32kB is not very big. Maybe we should default to something bigger. In any case, you can set this size according to your needs. The max size depends on the overall TA_RAM size (CFG_TZDRAM_SIZE
), used by core for all TAs (maybe there are other TAs than the pkcs11 TA in your system).
@etienne-lms I increased the CFG_PKCS11_TA_HEAP_SIZE to 128KB but it doesn't solve the problem, there seems to be a hard limit at 4096Bytes:
# dd if=/dev/urandom of=/tmp/data.bin bs=1 count=4096
4096+0 records in
4096+0 records out
4096 bytes (4.1 kB) copied, 0.0235162 s, 174 kB/s
# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --write-object /tmp/data.bin --type data --id 1137 --label DATA
Using slot 0 with a present token (0x0)
I/TA: PKCS11 session 1: login
Created Data Object:
Data object 1
label: 'DATA'
application: ''
app_id: <empty>
flags: modifiable
# dd if=/dev/urandom of=/tmp/data.bin bs=1 count=4097
4097+0 records in
4097+0 records out
4097 bytes (4.1 kB) copied, 0.0235554 s, 174 kB/s
# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --write-object /tmp/data.bin --type data --id 1137 --label DATA
Using slot 0 with a present token (0x0)
I/TA: PKCS11 session 1: login
error: PKCS11 function C_CreateObject failed: rv = CKR_DEVICE_MEMORY (0x31)
my CFG_TZDRAM_SIZE = 0x01e00000
That's strange. I don't see where such limit is set.
I've tried, unsing QEMU with OP-TEE's qemu_armv8a distribution built with:
make all CFG_PKCS11_TA=y CFG_PKCS11_TA_HEAP_SIZE=128000
From the non-secure console:
(...)
Welcome to Buildroot, type root or test to login
buildroot login: root
# export PS1="shell> "
shell>
shell> export PKCS11_MODULE_PATH=/usr/lib/libckteec.so.0
shell> export PIN="12345"
shell> export TOKEN_NAME="token0"
shell>
shell> pkcs11-tool --module $PKCS11_MODULE_PATH --label test --slot 0 --init-token --so-pin $PIN
Token successfully initialized
shell> pkcs11-tool --module $PKCS11_MODULE_PATH --label test --so-pin $PIN --init-pin --pin $PIN
Using slot 0 with a present token (0x0)
User PIN successfully initialized
shell>
shell> dd if=/dev/urandom of=/tmp/data.bin bs=1 count=4096
4096+0 records in
4096+0 records out
shell> pkcs11-tool --module $PKCS11_MODULE_PATH --login --pin $PIN --write-object /tmp/data.bin --type data --label sz4096
Using slot 0 with a present token (0x0)
Created Data Object:
Data object 1
label: 'sz4096'
application: ''
app_id: <empty>
flags: modifiable
shell>
shell> dd if=/dev/urandom of=/tmp/data.bin bs=1 count=15000
15000+0 records in
15000+0 records out
shell> pkcs11-tool --module $PKCS11_MODULE_PATH --login --pin $PIN --write-object /tmp/data.bin --type data --id 15000 --label sz15000
Using slot 0 with a present token (0x0)
Created Data Object:
Data object 1
label: 'sz15000'
application: ''
app_id: <empty>
flags: modifiable
shell>
shell> pkcs11-tool --module $PKCS11_MODULE_PATH --login --pin $PIN --list-objects
Using slot 0 with a present token (0x0)
Data object 1
label: 'sz15000'
application: ''
app_id: <empty>
flags: modifiable
Data object 2
label: 'sz4096'
application: ''
app_id: <empty>
flags: modifiable
shell>
@etienne-lms I used optee-os 3.19 Version on imx8mp , I also increased CFG_CORE_HEAP_SIZE
CFG_CORE_HEAP_SIZE=262144
CFG_PKCS11_TA_HEAP_SIZE=131072
still hitting the same issue :-(
I can see in the code that
in https://github.com/OP-TEE/optee_os/blob/master/ta/pkcs11/src/user_ta_header_defines.h#L19
#define TA_DATA_SIZE CFG_PKCS11_TA_HEAP_SIZE
https://github.com/OP-TEE/optee_os/blob/master/ta/pkcs11/src/user_ta_header_defines.h#L17C1
#define TA_STACK_SIZE (4 * 1024)
Maybe this is the limitation ?
I don't think TA_STACK_SIZE=4096
is the limitation. The pkcs11 TA does not use its stack to store object's data payloads or attributes. All are stored in the heap which size is given by TA_DATA_SIZE
that is set by CFG_PKCS11_TA_HEAP_SIZE
.
Add some trace messages (e.g. EMSG(...)
) in error cases of entry_create_object()
and create_object()
. I hope you'll find where the TA fails.
@etienne-lms I tested it also on an stm32mp1 it give me the same errors !
I will try to build some debug message to pin-point where this come from.
I've- just tested on my stm32mp157c-dk2 boards. The commands above (https://github.com/OP-TEE/optee_os/issues/6482#issuecomment-1822887214) work fine. I'm using the latest top master branches of optee_* Git repositories.
edited: I forgot to say I built my images with:
make PLATFORM=stm32mp1-157C_DK2 CFG_TEE_CORE_LOG_LEVEL=2 CFG_PKCS11_TA=y all CFG_PKCS11_TA_HEAP_SIZE=128000
With that config, I manage to create 14 objects of 15000 bytes each. Maybe the objects do not have the size I expect...
thanks for the information.
I used the versions:
3.19 for the imx8mp 3.16 for the stm32mp1
for both I set _CFG_PKCS11_TA_HEAPSIZE=131072
I don't have the possibility to update to latest for those device, so I need to find out which changes I can cherry-pick to solve this issue
You should be able to pick the source tree of ta/pkcs11/ from optee_os master branch and build that TA sources with your several build env based on older versions of OP-TEE. I would expect that it works ok. Oh, likely that you'll need to revert changes form commit c7f1b4f7881f.
@etienne-lms I had to increase also the CFG_TEE_RAM_VA_SIZE
with following:
CFG_CORE_HEAP_SIZE=2097152
CFG_TEE_RAM_VA_SIZE=4194304
CFG_PKCS11_TA_HEAP_SIZE=524288
I can import without errors
root@imx8mp:~# dd if=/dev/urandom of=/tmp/data.bin bs=1024 count=4
4+0 records in
4+0 records out
4096 bytes (4.1 kB) copied, 0.00011981 s, 34.2 MB/s
root@imx8mp:~#
root@imx8mp:~#
root@imx8mp:~#
root@imx8mp:~# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --write-object /tmp/data.bin --type data --label DATA
Using slot 0 with a present tokenI/TA: PKCS11 session 1: login
(0x0)
Created Data Object:
Data object 1
label: 'DATA'
application: ''
app_id: <empty>
flags: modifiable
root@imx8mp:~# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --read-object --type data --label DATA -o data.bin.out
Using slot 0 with a present token (0x0)
I/TA: PKCS11 session 1: login
root@imx8mp:~# diff data.bin.out /tmp/data.bin
however data over 5000 Bytes seems to be truncated :
root@imx8mp:~# dd if=/dev/urandom of=/tmp/data.bin bs=1024 count=6
6+0 records in
6+0 records out
6144 bytes (6.1 kB) copied, 0.000165208 s, 37.2 MB/s
root@imx8mp:~# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --write-object /tmp/data.bin --type data --label DATA
Using slot 0 with a present token (0x0)
I/TA: PKCS11 session 1: login
Created Data Object:
Data object 1
label: 'DATA'
application: ''
app_id: <empty>
flags: modifiable
root@imx8mp:~# pkcs11-tool --pin $PIN --module $PKCS11_MODULE_PATH --login --read-object --type data --label DATA -o data.bin.out
Using slot 0 with a present tokenI/TA: PKCS11 session 1: login
(0x0)
root@imx8mp:~# diff data.bin.out /tmp/data.bin
Binary files data.bin.out and /tmp/data.bin differ
root@imx8mp:~# ls -l data.bin.out
-rw-r--r-- 1 root root 5000 Nov 27 11:25 data.bin.out
I tested the same setup with SoftHSM, I get also the truncation to 5000 Bytes, there seems to be a limit in the standard.
the 5000 Bytes limit is actually set in pkcs11-tool:
https://github.com/OpenSC/OpenSC/blob/master/src/tools/pkcs11-tool.c#L3925
https://raymii.org/s/articles/Storing_arbitraty_data_in_the_Nitrokey_HSM.html
the 5000 Bytes limit is actually set in pkcs11-tool: https://github.com/OpenSC/OpenSC/blob/master/src/tools/pkcs11-tool.c#L3925
Thanks for the info and the pointers.
I noticed that I cannot import object in PKCS11 TA that are over 4K:
4KB still okay:
5KB not okay:
where can I increase this limit ?