Dynamic Linking of Trusted Applications

[Jayanth94]

Hello @jenswi-linaro ,

I am writing a trusted application which needs to be linked with my custom cryptographic library dynamically. I have seen a optee-test examples folder in which they have tried link an application dynamically , which is again like an other trusted app but instead of BINARY they have built it as a SHAREDLIB and then they have linked it with the main application dynamically via LDADD += -L ltest

1. I need to know whether this sharedlib which is a trusted app should be stored in lib/optee_tz folder or in other location.?

2. Secondly, Is there a way wherein I am allowed to stored this sharedlibs in a persistent object ? Can I load the Sharedlibs from the perssitent object at runtime and make use of the shared library ? Could you please share your thoughts on this ? Thanks again.

[jenswi-linaro]

1. I need to know whether this sharedlib which is a trusted app should be stored in lib/optee_tz folder or in other location.?

The lib should be stored in the same directory as the TAs. So lib/optee_tz should be good.

2. Secondly, Is there a way wherein I am allowed to stored this sharedlibs in a persistent object ?

No

[jayanthdc]

1. I need to know whether this sharedlib which is a trusted app should be stored in lib/optee_tz folder or in other location.?

The lib should be stored in the same directory as the TAs. So lib/optee_tz should be good.

1. Secondly, Is there a way wherein I am allowed to stored this sharedlibs in a persistent object ?

No

Many thanks for your reply. I am thinking of an approach to keep my built libraries in the persistent object and load them during runtime via dlopen , dlsym calls. Isn't this not possible ? Is that what you refer here ?? Could you please confirm?

[jenswi-linaro]

dlopen and friends are supported, but only when using the normal TA/lib storage.

[jayanthdc]

dlopen and friends are supported, but only when using the normal TA/lib storage.

Sorry, then you said no before to my previous question . May I get to know what were you trying to convey . normal TA/lib storage : Means ?

Could you please elaborate on this. I am a little confused. Thanks

[jayanthdc]

Did you understood that I was trying to load the libraries inside Persistent Object ?

I am trying to keep my libraries in my persistent object and load them onto the secure memory at runtime and not using the compile time linking option. This was my point. Hope I have conveyed it correctly now.

[jenswi-linaro]

You can do dlopen on a library which is stored in the normal library or TA location, for instance /lib/optee_tz.

It is not possible to load libraries from persistent objects.

[jayanthdc]

You can do dlopen on a library which is stored in the normal library or TA location, for instance /lib/optee_tz.

It is not possible to load libraries from persistent objects.

Ok. Thanks for your answer.

[jayanthdc]

TA2TA, which is what been referred here in the optee_ faqs section.

Can a Trusted Application communicate with other trusted application via shared memory or so ? Are there any such examples ? I have seen there is such possibility in the documentation section , but not much more than that.

Could you please let me know, any examples which I can refer to understand them? Thanks

[jenswi-linaro]

It's worth skimming through GlobalPlatform TEE Internal Core API Specification v1.2 covering TEE_InvokeTACommand() and friends.

We have an example here in our regression test suite. Note that this is tests covering certain aspects, not examples of how to write secure TAs. Non-secure user space: https://github.com/OP-TEE/optee_test/blob/7404c07b9457ae1a5bfe6bc7b13affc479e58054/host/xtest/regression_1000.c#L871-L911 TA: https://github.com/OP-TEE/optee_test/blob/7404c07b9457ae1a5bfe6bc7b13affc479e58054/ta/rpc_test/ta_entry.c#L43-L80

[jayanthdc]

It's worth skimming through GlobalPlatform TEE Internal Core API Specification v1.2 covering TEE_InvokeTACommand() and friends.

We have an example here in our regression test suite. Note that this is tests covering certain aspects, not examples of how to write secure TAs. Non-secure user space: https://github.com/OP-TEE/optee_test/blob/7404c07b9457ae1a5bfe6bc7b13affc479e58054/host/xtest/regression_1000.c#L871-L911 TA: https://github.com/OP-TEE/optee_test/blob/7404c07b9457ae1a5bfe6bc7b13affc479e58054/ta/rpc_test/ta_entry.c#L43-L80

Thanks for your answer. I will go through them.

[jayanthdc]

All code executed in a Trusted Application is said to be executed by Tasks. A Task keeps a record of its execution history (typically realized with a stack) and current execution state. This record is collectively called a Task context. A Task SHALL be created each time the Trusted OS calls an entry point of the Trusted Application.

I see this in the TEE_Internal Core API documentation.

Here the Task means the OPTEE OS is creating a task for each and every TA's and is managing it or is it with respect to the Normal World Linux. Because OP-TEE itself is being scheduled by the Normal World Linux. So which is correct here ?

[jenswi-linaro]

OP-TEE is compliant with or as good as compliant with the spec. Task as you describe it is generic enough to cover what we're doing in OP-TEE. Why should you statements contradict each other?

[jayanthdc]

OP-TEE is compliant with or as good as compliant with the spec. Task as you describe it is generic enough to cover what we're doing in OP-TEE. Why should you statements contradict each other?

Excuse if I am incorrect. I am trying to understand what is happening at the back when a Trusted Application is executed. Because I was under the assumption that OP-TEE is being scheduled from Normal World Linux, and when this OP-TEE gets scheduled it internally does the work of running the Trustedapplication.

So my confusion is , when OP-TEE gets scheduled from Linux, will OP-TEE now during its running time will also create a task internal to it and this task is responsible for the TrustedApp control ? Or when we hear the term called Task its all with respect to Rich OS linux.

Hope my statement is clear now.

[jenswi-linaro]

We have no tasks in OP-TEE, it's only a term used in the spec. From TA or API point of view it looks like the TA is executed as a task as described in the specification. OP-TEE is internally working with a fixed number of threads instead, one thread can handle one request at a time. Each active thread is scheduled on behalf of the calling user space program (or sometimes a kernel thread).

If you're curious about the internals of OP-TEE you could also try reading https://optee.readthedocs.io/en/latest/architecture/index.html and browsing the code.

[jayanthdc]

We have no tasks in OP-TEE, it's only a term used in the spec. From TA or API point of view it looks like the TA is executed as a task as described in the specification. OP-TEE is internally working with a fixed number of threads instead, one thread can handle one request at a time. Each active thread is scheduled on behalf of the calling user space program (or sometimes a kernel thread).

If you're curious about the internals of OP-TEE you could also try reading https://optee.readthedocs.io/en/latest/architecture/index.html and browsing the code.

Thanks for your reply. I understand it better now.

[jayanthdc]

Hi @jenswi-linaro,

I am trying to use the dynamic loading functionality in my trusted application. I have very less understanding regarding the memory allocation here. To be precise, I will load the required shared library using dlopen( ) function call. If it is loaded successfully I will try to get the address of a method from the opened shared object , assign the address to the function pointer and use it.

As per my understanding, the shared library gets loaded into the secure memory as they are loaded by the OP-TEE core similar to a trusted application. Hope I am correct here ??

Secondly as its already mentioned , programmers have to take care of allocating memory for a trusted application by defining it in the user_ta_define_header.h as below.

define TA_STACK_SIZE (8 * 1024)

define TA_DATA_SIZE (900 * 1024)

My doubt : Does the shared library occupies the alloted memory space ( of a TA which is loading it) or it is loaded somewhere else in the secured memory by OP-TEE core.

Could you please let me know. Thanks.

[jforissier]

Hello @jayanthdc,

The TA heap is common, in other words TA_DATA_SIZE is declared by the TA and used by malloc() and 'TEE_Malloc()` irrespective of the caller (the TA or a shared library). The memory space used to map the shared library binary itself is not comprised in this space. Same as the memory to map the TA. That memory is managed separately by the TEE core.

[jayanthdc]

Hello @jayanthdc,

The TA heap is common, in other words TA_DATA_SIZE is declared by the TA and used by malloc() and 'TEE_Malloc()` irrespective of the caller (the TA or a shared library). The memory space used to map the shared library binary itself is not comprised in this space. Same as the memory to map the TA. That memory is managed separately by the TEE core.

Thanks for your reply.

Sorry I understood a little.

So I am just rephrasing to clarify . As you are stating that ,the shared library binary ( for example : libos_test_dl having a size of around 60 Kb) will get loaded into a secure memory space and not into the invoking Trusted Application memory space.

On the other side , the heap allocated with a TA will be used for malloc ( ) calls from both the Trustedapp and the Shared library which is being loaded .

Hope I have understood correctly.

[jforissier]

Yes you're correct. Sorry for being unclear :-/

[jayanthdc]

Yes you're correct. Sorry for being unclear :-/

Thanks a lot and sorry it was my mistake . You had written it correctly. I just wanted to make sure I read it correctly so I drafted it again. Thanks again for your reply.

[jayanthdc]

Hi , I have built a shared library and trying to load it dynamically from an other trusted application. I am facing this error while loading it. Could anyone let me know whats the missing part here.

D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbc D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbc D/TC:? 0 tee_ta_invoke_command:824 Error: ffff0006 of 4 E/TA: load_module:682 E: loading 'd4e94f5b-43e1-461a-ac4c-8ffff03745cb' failed:

I have done the following -

1. I have compiled the trsuted application as a shared library by adding these lines in the Makefile SHLIBNAME = libjayanth_dl SHLIBUUID = d4e94f5b-43e1-461a-ac4c-8ffff03745cb

Then the built d4e94f5b-43e1-461a-ac4c-8ffff03745cb.ta is being used in another TA.

I am using dlopen( "d4e94f5b-43e1-461a-ac4c-8ffff03745cb",RTLD_NOW) call .

Kindly let me know if there is any step missed during compilation.

Thanks

[jayanthdc]

Hi @jforissier, Could you please share your thoughts on this issue. Thanks in advance.

Best regards, Jayanth

[jforissier]

Hi @jayanthdc,

Some more debug traces are needed. Normally you should see something like that (output from xtest 1022):

D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbebc
D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF b3091a65-9751-4784-abf7-0298a7cc35ba (Secure Storage TA)
D/TC:? 0 system_open_ta_binary:261 res=0xffff0008
D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF b3091a65-9751-4784-abf7-0298a7cc35ba (REE)
D/TC:? 0 system_open_ta_binary:261 res=0x0
D/LD:  ta_elf_add_library:1457 ELF (b3091a65-9751-4784-abf7-0298a7cc35ba) at 0x40068000

In your case it seems you have a TEE_ERROR_BAD_PARAMETERS error (0xffff0006) even before the TEE tries to open the shared library. You should add more debug traces, for example in tee_ta_invoke_command() to see where the error code comes from.

[jayanthdc]

Hi @jayanthdc,

Some more debug traces are needed. Normally you should see something like that (output from xtest 1022):

D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbebc
D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF b3091a65-9751-4784-abf7-0298a7cc35ba (Secure Storage TA)
D/TC:? 0 system_open_ta_binary:261 res=0xffff0008
D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF b3091a65-9751-4784-abf7-0298a7cc35ba (REE)
D/TC:? 0 system_open_ta_binary:261 res=0x0
D/LD:  ta_elf_add_library:1457 ELF (b3091a65-9751-4784-abf7-0298a7cc35ba) at 0x40068000

In your case it seems you have a TEE_ERROR_BAD_PARAMETERS error (0xffff0006) even before the TEE tries to open the shared library. You should add more debug traces, for example in tee_ta_invoke_command() to see where the error code comes from.

Thanks for your reply . I will try again with more tracing settings.

[jayanthdc]

Hi @jforissier ,

The trace logs Error case : F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #8 (syscall_check_access_rights) F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) D/TC:? 0 tee_ta_invoke_command:784 Error: ffff0006 of 4 F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command)

The trace logs success case :

F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #8 (syscall_check_access_rights) F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) D/TC:? 0 system_open_ta_binary:256 Lookup user TA ELF d4e94f5b-43e1-461a-ac4c-8ffff0) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x1B F/TC:? 0 plat_prng_add_jitter_entropy:72 0x80 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x8F D/TC:? 0 system_open_ta_binary:260 res=0xffff0008 D/TC:? 0 system_open_ta_binary:256 Lookup user TA ELF d4e94f5b-43e1-461a-ac4c-8ffff0) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x12 D/TC:? 0 system_open_ta_binary:260 res=0x0 F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #33 (syscall_cryp_random_number_generate) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x63 F/TC:? 0 plat_prng_add_jitter_entropy:72 0xA3 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x06 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x24 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x64 F/TC:? 0 plat_prng_add_jitter_entropy:72 0xFB F/TC:? 0 plat_prng_add_jitter_entropy:72 0xDC F/TC:? 0 plat_prng_add_jitter_entropy:72 0xAB F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) D/LD: ta_elf_add_library:1457 ELF (d4e94f5b-43e1-461a-ac4c-8ffff03745cb) at 0x402f70 F/TC:? 0 trace_syscall:128 syscall #6 (syscall_close_ta_session) D/TC:? 0 tee_ta_close_session:492 csess 0x101781f0 id 3 D/TC:? 0 tee_ta_close_session:512 Destroy session F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) I/TA: SharedObject Loaded Successfully F/TC:? 0 trace_syscall:128 syscall #8 (syscall_check_access_rights) F/TC:? 0 trace_syscall:128 syscall #8 (syscall_check_access_rights) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command)

The trace logs after loading the library correctly.

The only change I made is to add additional flags in dlopen call.

In the first case i tried using dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW )

then I added additional flags and tried again to load it dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW | RTLD_GLOBAL | RTLD_NODELETE ) In the second case it loaded successfully.

Could you please explain is it mandatory to add additional flags ? or what was causing the issue?. Thanks in advance

[jforissier]

In the first case i tried using dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW )

then I added additional flags and tried again to load it dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW | RTLD_GLOBAL | RTLD_NODELETE ) In the second case it loaded successfully.

Could you please explain is it mandatory to add additional flags ? or what was causing the issue?. Thanks in advance

It is expected behavior, as mentioned in the header file: https://github.com/OP-TEE/optee_os/blob/master/lib/libdl/include/dlfcn.h#L17-L18. The reason is limitations in the OP-TEE libdl implementation. We want the flags to reflect how things are implemented in ldelf (the user space ELF loader in OP-TEE).

• Symbol resolution happens immediately, it cannot be deferred therefore RTLD_NOW is required.
• Symbols are made available globally to other shared library loaded afterwards. Hence the requirement for RTLD_GLOBAL.
• Symbols are not removed when a shared library is closed, hence RTLD_NODELETE.

I realize the simple comment in dlfcn.h is likely to be missed. If you have an idea how we could make this more obvious to users, please suggest!

[jayanthdc]

In the first case i tried using dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW ) then I added additional flags and tried again to load it dlopen("d4e94f5b-43e1-461a-ac4c-8ffff0", RTLD_NOW | RTLD_GLOBAL | RTLD_NODELETE ) In the second case it loaded successfully. Could you please explain is it mandatory to add additional flags ? or what was causing the issue?. Thanks in advance

It is expected behavior, as mentioned in the header file: https://github.com/OP-TEE/optee_os/blob/master/lib/libdl/include/dlfcn.h#L17-L18. The reason is limitations in the OP-TEE libdl implementation. We want the flags to reflect how things are implemented in ldelf (the user space ELF loader in OP-TEE).

• Symbol resolution happens immediately, it cannot be deferred therefore RTLD_NOW is required.
• Symbols are made available globally to other shared library loaded afterwards. Hence the requirement for RTLD_GLOBAL.
• Symbols are not removed when a shared library is closed, hence RTLD_NODELETE.

I realize the simple comment in dlfcn.h is likely to be missed. If you have an idea how we could make this more obvious to users, please suggest!

Thanks a lot for your answer. Excuse me , as I missed out on checking this header file. Infact the next question I was about to raise was that dlerror call is not getting compiled. But after this https://github.com/OP-TEE/optee_os/blob/master/lib/libdl/include/dlfcn.h#L17-L18. its clear that only dlopen , dlsym and dlclose calls are being supported.

[jforissier]

Thanks a lot for your answer. Excuse me , as I missed out on checking this header file. Infact the next question I was about to raise was that dlerror call is not getting compiled. But after this https://github.com/OP-TEE/optee_os/blob/master/lib/libdl/include/dlfcn.h#L17-L18. its clear that only dlopen , dlsym and dlclose calls are being supported.

You're welcome. In case you need more than what we have already, dlerror() for instance, feel free to propose patches. I will gladly review them.

[jayanthdc]

Thanks a lot for your answer. Excuse me , as I missed out on checking this header file. Infact the next question I was about to raise was that dlerror call is not getting compiled. But after this https://github.com/OP-TEE/optee_os/blob/master/lib/libdl/include/dlfcn.h#L17-L18. its clear that only dlopen , dlsym and dlclose calls are being supported.

You're welcome. In case you need more than what we have already, dlerror() for instance, feel free to propose patches. I will gladly review them.

Thanks for your suggestions. Currently I am working on my master's project and completely new to these features. So sure I would try to work on it in future and update here.

[jayanthdc]

@jforissier , I have also noticed that whenever I have used IMSG calls in the functions which are built as a shared library and then when I try to load the those library it fails. D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #8 (syscall_check_access_rights) F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #5 (syscall_open_ta_session) D/TC:? 0 tee_ta_init_session_with_context:586 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7c F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) D/TC:? 0 system_open_ta_binary:256 Lookup user TA ELF e00508c9-4144-4be4-bcf2-616014) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x23 F/TC:? 0 plat_prng_add_jitter_entropy:72 0xE0 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x6B D/TC:? 0 system_open_ta_binary:260 res=0xffff0008 D/TC:? 0 system_open_ta_binary:256 Lookup user TA ELF e00508c9-4144-4be4-bcf2-616014) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x71 D/TC:? 0 system_open_ta_binary:260 res=0x0 F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #33 (syscall_cryp_random_number_generate) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) F/TC:? 0 plat_prng_add_jitter_entropy:72 0x53 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x47 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x28 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x2D F/TC:? 0 plat_prng_add_jitter_entropy:72 0xE4 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x35 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x57 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x53 F/TC:? 0 plat_prng_add_jitter_entropy:72 0x40 F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) E/LD: resolve_sym:210 Symbol trace_level not found D/TC:? 0 tee_ta_invoke_command:784 Error: ffff0008 of 4 F/TC:? 0 trace_syscall:128 syscall #7 (syscall_invoke_ta_command) E/TA: inc_value:118 Dlopen call failed D/TC:? 0 tee_ta_invoke_command:784 Error: ffff0000 of 4

Error is generic ( 0XFFFF0000).

On removal of the IMSG call , If I try to load the library again , it works completely fine. Dont know whats the issue .

[jayanthdc]

Hi @jforissier , Could you please share your thoughts on my previous question. Would be really helpful Thank you.

[jforissier]

Hi @jayanthdc ,

E/LD: resolve_sym:210 Symbol trace_level not found

Interesting. We have a similar trace in optee_test here and it's working fine (I just checked on QEMU). Which version of OP-TEE are you using? There have been some changes in the symbol resolution done by ldelf. Perhaps you're missing some fixes.

[jayanthdc]

Hi @jayanthdc ,

E/LD: resolve_sym:210 Symbol trace_level not found

Interesting. We have a similar trace in optee_test here and it's working fine (I just checked on QEMU). Which version of OP-TEE are you using? There have been some changes in the symbol resolution done by ldelf. Perhaps you're missing some fixes.

OP-TEE version 3.8

[jayanthdc]

Hi @jayanthdc ,

E/LD: resolve_sym:210 Symbol trace_level not found

Interesting. We have a similar trace in optee_test here and it's working fine (I just checked on QEMU). Which version of OP-TEE are you using? There have been some changes in the symbol resolution done by ldelf. Perhaps you're missing some fixes.

OP-TEE version 3.8. Is the issue fixed in this version? or Should I upgrade to latest version

[jforissier]

If you can please try with latest version (master, or 3.11.0 if you want a more stable reference). I think it should work.

[jayanthdc]

If you can please try with latest version (master, or 3.11.0 if you want a more stable reference). I think it should work.

Sure. Thanks for your input. I will test and update here.

[jayanthdc]

@jforissier , D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbc D/TC:? 0 tee_ta_init_session_with_context:609 Re-open TA 3a2f8978-5dc0-11e8-9c2d-fa7ae01bbc D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF e00508c9-4144-4be4-bcf2-616014c3e8e8) D/TC:? 0 system_open_ta_binary:261 res=0xffff0008 D/TC:? 0 system_open_ta_binary:257 Lookup user TA ELF e00508c9-4144-4be4-bcf2-616014c3e8e8) D/TC:? 0 system_open_ta_binary:261 res=0xffff0008 D/TC:? 0 tee_ta_invoke_command:824 Error: ffff0008 of 4 E/LD: init_elf:438 sys_open_ta_bin(e00508c9-4144-4be4-bcf2-616014c3e8e8) D/TC:? 0 tee_ta_invoke_command:824 Error: ffff0008 of 4 E/TA: inc_value:118 Dlopen call failed D/TC:? 0 tee_ta_invoke_command:824 Error: ffff0000 of 4

I am getting these errors and not able to load even without IMSG calls which was working with 3.8 version Confused. Kindly help

[jayanthdc]

Should we need to place .elf instead of .ta format of the shared object , because currently I am placing the signed binary built out of the SHLIB. Hope I am following correctly.

[jforissier]

No the .ta is what is needed. It should be in the same directory as the main TA binary.

[jayanthdc]

No the .ta is what is needed. It should be in the same directory as the main TA binary.

Then I have followed the steps correctly.

[jayanthdc]

CFG_TEE_TA_LOG_LEVEL ?= 4

The UUID for the Trusted Application BINARY=8aaaf200-2450-11e4-abe2-0002a5d5c51b

LDADD = -ldl

-include $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk

ifeq ($(wildcard $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk), ) clean: @echo 'Note: $$(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk not found, cannot clean TA' @echo 'Note: TA_DEV_KIT_DIR=$(TA_DEV_KIT_DIR)' endif

Makefile of the main trusted application.

[jayanthdc]

CFG_TEE_TA_LOG_LEVEL ?= 4

The UUID for the Trusted Application SHLIBNAME = libadd_dl SHLIBUUID = e00508c9-4144-4be4-bcf2-616014c3e8e8

-include $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk

ifeq ($(wildcard $(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk), ) clean: @echo 'Note: $$(TA_DEV_KIT_DIR)/mk/ta_dev_kit.mk not found, cannot clean TA' @echo 'Note: TA_DEV_KIT_DIR=$(TA_DEV_KIT_DIR)' endif

Makefile of the sharedobject TA

[jayanthdc]

Are my makefiles correct ? Anything missing here. Please check and correct me If I am wrong.

[jayanthdc]

Tested with 3.11 version as well, same issue. If the IMSG call is used in the shared object ( a.k.a TA) issue appears and states sym_trace not found. E/LD: resolve_sym:210 Symbol trace_level not found

So kindly @jforissier let me know which version has to be picked up on solving this. I am testing this on raspberrypi 3b+ platform. Thanks in advance

[jayanthdc]

Hello all , Could anyone please suggest any solution to my problem, I have raised here. Might be a simple one, If I missed anything during build or during testing , Kindly let me know. Thanks in advance

[jforissier]

The trace_level symbol is supposed to be in your TA main binary:

$ nm out-br/build/optee_test_ext-1.0/ta/os_test/out/5b9e0e40-2636-11e1-ad9e-0002a5d5c51b.elf | grep -w trace_level
00018598 D trace_level
optee_os $ git grep -w trace_level ta/
ta/arch/arm/user_ta_header.c:13:int trace_level = TRACE_LEVEL;

Can you check if it is present in you TA ELF file? Is user_ta_header.c compiled and linked properly in your TA?

[jayanthdc]

/ta$ nm 8aaaf200-2450-11e4-abe2-0002a5d5c51b.elf | grep -w trace_level 000000000000c420 D trace_level

Yes. trace_level is found in my main TA elf.

[jayanthdc]

@jforissier I am summarising again the steps followed so far.

I am trying to use the dynamic loading feature here. So I have a main trusted application (8aaaf200-2450-11e4-abe2-0002a5d5c51b.ta )which is invoked from the normal world client .

Next, this trusted application is loading a shared object using dlopen call. Here the shared object binary is (e00508c9-4144-4be4-bcf2-616014c3e8e8.ta).

In my main trusted app , I have used the IMSG and DMSG calls which works completely fine. In my sharedobject i.e (e00508c9-4144-4be4-bcf2-616014c3e8e8.ta) I have used IMSG call which is causing me the problem.

In fact the source file of my sharedobject trsuted app is very simple . Its just a add function.

include

include

include

int add (int a , int b);

int add (int a , int b) { int res=0; res = a + b; //IMSG(" Shared library print ** : %d\n", res);** return res; }

As you can see, I have commented the IMSG in this code. If I uncomment this during loading time from the main TA it throws me an error.

So let me know is there anything I missed here. I have even shared the makefiles of both of the Trusted apps above. Kindly help.

[jforissier]

I think you need to debug the resolve_sym() function in ldelf, adding DMSG() should help figure out why the symbol is not found. What should normally happen is, unresolved symbols in the shared object should be found either in the main application or in external libraries. In your case ldelf should first look for trace_level in the main executable and find it.

Alternately, if you can write a simple reproducer running on QEMU (based on optee_test or optee_examples for instance), then I can look into it. I have not been able to reproduce the issue by adding IMSG() to optee_test/ta/os_test_lib_dl/os_test_lib_dl.c.

[jayanthdc]

I think you need to debug the resolve_sym() function in ldelf, adding DMSG() should help figure out why the symbol is not found. What should normally happen is, unresolved symbols in the shared object should be found either in the main application or in external libraries. In your case ldelf should first look for trace_level in the main executable and find it.

Alternately, if you can write a simple reproducer running on QEMU (based on optee_test or optee_examples for instance), then I can look into it. I have not been able to reproduce the issue by adding IMSG() to optee_test/ta/os_test_lib_dl/os_test_lib_dl.c.

So you mean try to run the same application which is causing the issue in RaspberryPi platform on QEMU platform ? If so I will give it a try.