Is it possible to call TEEC_InvokeCommand more than once in a same open session?

[cezane]

I am refactoring my code and adding more functions. Now, I have three trusted functions: one to generate an RSA key pair, one to encrypt some data and another to decrypt some data. I would like to create the key pair and maintain it in the session, until I encrypt and decrypt some data.

When I create the key pair, I return to the CA without free my key variable, that is in a session struct. This first function is called by rsa_gen_keys(&ta); in the main function of CA (below). After it returns ok, I am getting a segmentation fault. Is this because I continue with the session opened and follow calling another TEEC_InvokeCommand in sequence? This second call is the rsa_encrypt function in the main, as can be seen below:

int main(int argc, char *argv[])
{
    struct ta_attrs ta;
    char clear[RSA_MAX_PLAIN_LEN_1024];
    char ciph[RSA_CIPHER_LEN_1024];

    prepare_ta_session(&ta);

    printf("\nType something to be encrypted and decrypted in the TA:\n");
    fflush(stdin);
    fgets(clear, sizeof(clear), stdin);

    rsa_gen_keys(&ta);
    rsa_encrypt(&ta, clear, RSA_MAX_PLAIN_LEN_1024, ciph, RSA_CIPHER_LEN_1024);
    rsa_decrypt(&ta, ciph, RSA_CIPHER_LEN_1024, clear, RSA_MAX_PLAIN_LEN_1024);

    terminate_tee_session(&ta);
    return 0;
}

If this is not the case of segmentation fault, can you point me in order I can give more details about my problem?

Thank you.

[jforissier]

Calling TEEC_InvokeCommand several times is perfectly fine.

[cezane]

I do not know why I am receiving a segmentation fault. The rsa_gen_keys() function, that is called in the main, is:

void rsa_gen_keys(struct ta_attrs *ta) {
    TEEC_Result res;
    res = TEEC_InvokeCommand(&ta->sess, TA_RSA_CMD_GENKEYS,
                 NULL, NULL);
    if (res != TEEC_SUCCESS)
        errx(1, "TEEC_InvokeCommand(TA_RSA_CMD_GENKEYS) failed 0x%x", res);
    printf("\nKeys already generated.\n");
}

When I call it, the key pair is successful generated, without error, and I get this message:

Type something to be encrypted and decrypted in the TA: test

Keys already generated. [ 13.690073] rsa[1158]: unhandled level 2 translation fault (11) at 0x00000004, esr 0x92000046 [ 13.690286] pgd = ffff80003c29e000 [ 13.690393] [00000004] pgd=000000007c27f003, pud=000000007c2a5003, *pmd=0000000000000000 [ 13.690735] CPU: 0 PID: 1158 Comm: rsa Not tainted 4.12.0 #1 [ 13.690864] Hardware name: QEMU QEMU Virtual Machine, BIOS 0.0.0 02/06/2015 [ 13.691040] task: ffff80003d4ae800 task.stack: ffff80003c270000 [ 13.691321] PC is at 0x400f20 [ 13.691412] LR is at 0x400f1c [ 13.691509] pc : [<0000000000400f20>] lr : [<0000000000400f1c>] pstate: 60000000 [ 13.691644] sp : 0000ffffd299d7c0 [ 13.691727] x29: 0000ffffd299d7c0 x28: 0000000000000000 [ 13.691865] x27: 0000000000000000 x26: 0000000000000000 [ 13.691974] x25: 0000000000000000 x24: 0000000000000000 [ 13.692080] x23: 0000000000000000 x22: 0000000000000000 [ 13.692191] x21: 0000000000400c68 x20: 0000000000000000 [ 13.692305] x19: 0000000000411000 x18: 0000000000000000 [ 13.692420] x17: 0000ffffb0c85c38 x16: 0000000000411370 [ 13.692531] x15: 0000000000000252 x14: 00000000000003f3 [ 13.692668] x13: 0000000000000000 x12: 00000000000003f3 [ 13.692808] x11: 0000000000000008 x10: 0101010101010101 [ 13.695642] x9 : 0000ffffd299d7c0 x8 : 3d3d3d3d3d3d3d3d [ 13.695787] x7 : 3d3d3d3d45444953 x6 : 000000003cb29064 [ 13.695899] x5 : 0000000000000000 x4 : 0000000000000000 [ 13.696034] x3 : 0000ffffb0db96f0 x2 : 0000ffffb0db9000 [ 13.696140] x1 : 0000000000000000 x0 : 0000000000000000 Segmentation fault

Any suggestion?

[jforissier]

[ 13.690073] rsa[1158]: unhandled level 2 translation fault (11) at 0x00000004, esr 0x92000046

You have a NULL pointer somewhere (0x00000004 = NULL + 4). ta maybe?

[cezane]

Thank you, again, @jforissier . The only part I can see I have NULL pointers, is in the TEEC_InvokeCommand:

res = TEEC_InvokeCommand(&ta->sess, TA_RSA_CMD_GENKEYS, NULL, NULL);

But, I think there is no problem with this, according to the specification. Please correct me, if I am wrong.

[jbech-linaro]

@cezane , no intention to sound rude or anything here, but have you tried debug using gdb? I think it is usually much easier to spot errors when running it using gdb compared to just reading logs and code. If you can run your code using QEMU I can guide you in setting up GDB.

[cezane]

Hello, @jbech-linaro . I totally agree and understand you. Indeed, I still did not use gdb. As I am still improving my basic knowledge about C programming, I was searching a bit regarding the debugging process. I saw a step by step to debug TA, but I did not follow yet, since my problem now is with the CA. I am running my application on QEMU, following your instructions in the build README. If you really can guide me to setup the debugging tools, it will be very good and appreciated.

Thank you.

[jforissier]

@cezane I'll post something soon as a pull request. What you need is called remote debugging with gdbserver ; well I think it will help you significantly ;-) Stay tuned!

[jforissier]

See https://github.com/OP-TEE/build/pull/293 (documentation).

[cezane]

Hello, @jforissier . Thank you for this documentation. In parallel, I started to read a GDB tutorial, but I am still at the beginning. I tried to do as you mention: gdbserver :12345 xtest 4002, but I am getting the following output:

(qemu) gdbserver :12345 xtest 4002 gdbserver: extraneous characters at the end of line

Same occurs if I try with my rsa application instead of xtest. The help of gdbserver, inside qemu, does not help much.

Besides, I did not find the following file: arm-buildroot-linux-gnueabihf-gdb. I've searched at all the optee folder.

[jforissier]

gdbserver should be run inside the QEMU guest, not at the QEMU prompt... Sorry if the doc is misleading.

[cezane]

I tried gdbserver inside the guest (Normal World), before my last comment, but I got:

-sh: gdbserver: not found

Maybe the GDBSERVER flag is not working here...

[jforissier]

Did you update your checkout of build.git? This feature was merged yesterday : https://github.com/OP-TEE/build/pull/293/files "repo sync build" should get it.

[jbech-linaro]

@jforissier , @cezane , just want to mention that I've tried this and it works fine. I synced my tree to latest and then just followed the instructions provided.

[cezane]

Since yesterday, when I updated the project, I am getting errors when running make. I decided to delete all the project and get it again, following all the steps. Now, after make toolchains, I am still getting errors when running make:

> -- The C compiler identification is unknown
-- Check for working C compiler: /home/.../optee/out-br/host/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /home/.../optee/out-br/host/bin/aarch64-linux-gnu-gcc -- broken
CMake Error at /home/.../optee/out-br/host/share/cmake-3.8/Modules/CMakeTestCCompiler.cmake:51 (message):
  The C compiler
  "/home/.../optee/out-br/host/bin/aarch64-linux-gnu-gcc" is not able to compile a simple test program.

> It fails with the following output:
   Change Dir: /home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp

>  Run Build Command:"/usr/bin/make" "cmTC_fca72/fast"    
  make[3]: Entering directory '/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp'

>  /usr/bin/make -f CMakeFiles/cmTC_fca72.dir/build.make
  CMakeFiles/cmTC_fca72.dir/build

>  make[4]: Entering directory '/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp'
  Building C object CMakeFiles/cmTC_fca72.dir/testCCompiler.c.o
>  /home/.../optee/out-br/host/bin/aarch64-linux-gnu-gcc
  -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os
  -DNDEBUG -o CMakeFiles/cmTC_fca72.dir/testCCompiler.c.o -c
  /home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp/testCCompiler.c

>  ccache: error: execv of
  /home/.../optee/out-br/host/bin/aarch64-linux-gnu-gcc.br_real
  failed: No such file or directory

 > CMakeFiles/cmTC_fca72.dir/build.make:65: recipe for target
  'CMakeFiles/cmTC_fca72.dir/testCCompiler.c.o' failed

 > make[4]: *** [CMakeFiles/cmTC_fca72.dir/testCCompiler.c.o] Error 1
  make[4]: Leaving directory '/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp'

 > Makefile:126: recipe for target 'cmTC_fca72/fast' failed
  make[3]: *** [cmTC_fca72/fast] Error 2
  make[3]: Leaving directory
  '/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeTmp'  

 > CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

>-- Configuring incomplete, errors occurred!
See also "/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeOutput.log".
See also "/home/.../optee/out-br/build/optee_client-1.0/CMakeFiles/CMakeError.log".
package/pkg-generic.mk:211: recipe for target '/home/.../optee/out-br/build/optee_client-1.0/.stamp_configured' failed
make[2]: *** [/home/.../optee/out-br/build/optee_client-1.0/.stamp_configured] Error 1
Makefile:16: recipe for target '_all' failed
make[1]: *** [_all] Error 2
make[1]: Leaving directory '/home/.../optee/out-br'
common.mk:201: recipe for target 'buildroot' failed
make: *** [buildroot] Error 2

Any suggestion regarding this?

[jforissier]

Dunno. I've just tried building from scratch on a Ubuntu 18.04 box, and it worked.

mkdir optee-repo-qemu
cd optee-repo-qemu
repo init -u https://github.com/OP-TEE/manifest.git --reference ~/repo-ref  # ref cloned from https://github.com/jbech-linaro/manifest/tree/reference without the <!-- v8M related --> section in default.xml
repo sync -j20
cd build
make toolchains
make -j40 GDBSERVER=y

[cezane]

I just finished the process in another computer, also with Ubuntu 18.04, and same problem occurs. These are the steps I followed for both computers with Ubuntu 18.04:

mkdir optee
cd optee
repo init -u https://github.com/OP-TEE/manifest.git -m qemu_v8.xml
repo sync
cd build
make toolchains
make
make -j8 run GDBSERVER=y

As I said, the same errors occurred in both computers... Thank you for any help.

[jforissier]

Oh, you're using qemu_v8.xml, I missed that. Let me try again and see what's happening.

[jbech-linaro]

Actually, I saw similar issues as @cezane reports yesterday when building a QEMU v7 setup. I saw it twice and I just cleaned it all including and deleted both out folders and then it worked again. I've never seen that before, so I wonder if this is something that has sneaked in recently. But I haven't figure out what causes it to happen.

[jforissier]

Interesting... worked fine for me with qemu_v8.xml too.

mkdir optee-repo-qemu_v8
cd optee-repo-qemu_v8
repo init -u https://github.com/OP-TEE/manifest.git -m qemu_v8.xml --reference ~/repo-ref 
repo sync -j20
cd build
make toolchains
make -j40 run GDBSERVER=y

[jbech-linaro]

Worth mention that I haven't seen the issue on a clean build. I saw it either after a rebuild or after updating a git, cannot remember which. I'm about to update my v8 setup here on my side, so I'll also give it a try there.

[jforissier]

Ha! Problem reproduced here!!!

rm -rf ../out-br
make -j40 buildroot
make -j40 buildroot GDBSERVER=y
# => error

@cezane don't mix builds with and without GDBSERVER=y. To fix your environment, run rm -rf ../out-br and build again (make -j8 GDBSERVER=y, or make -j8 run GDBSERVER=y to build and run QEMU). If you want to start QEMU without re-building anything, use make run-only.

[jbech-linaro]

don't mix builds with and without GDBSERVER=y

Yes, well in line with what I did yesterday, I saw the issue when I was trying that stuff.

[cezane]

It really worked, @jforissier . Thank you. I had tried before to remove the out-br folder and "remake" but I also mixed the GDBSERVER flag and got the same errors. Now, it is running with gdbserver enabled. My difficult now is how to use gdb to find the problem that is causing the segfault. I was trying to debug with VS Code, but I will try again with only gdb at terminal.

By the way, I am using this: out-br/host/bin/aarch64-buildroot-linux-gnu-gdb, and its respective sysroot.

[cezane]

7 days fighting this error:

Program received signal SIGSEGV, Segmentation fault. 0x0000000000400f24 in prepare_op (out_sz=128, out=0xfffffffffbe0 "", in_sz=86, in=0xfffffffffb88 "test\n", op=0x0) at host/main.c:46 46 op->paramTypes = TEEC_PARAM_TYPES(TEEC_MEMREF_TEMP_INPUT, (gdb) s

Only now, with the help of gdb I found that the problem was in the following function:

void prepare_op(TEEC_Operation *op, char *in, size_t in_sz, char *out, size_t out_sz) {
    memset(op, 0, sizeof(*op));
    op->paramTypes = TEEC_PARAM_TYPES(TEEC_MEMREF_TEMP_INPUT,
                            TEEC_MEMREF_TEMP_OUTPUT,
                            TEEC_NONE, TEEC_NONE);
    op->params[0].tmpref.buffer = in;
    op->params[0].tmpref.size = in_sz;
    op->params[1].tmpref.buffer = out;
    op->params[1].tmpref.size = out_sz;
}

At first line, I had: memset(&op, 0, sizeof(*op)); (note the "&op")... I will never find this, since this function is only called at rsa_encrypt and rsa_decrypt functions, and both are called only after the rsa_gen_keys function (as can be seen in my first comment in this issue). I really do not understand why a printf that I put right after rsa_gen_keys and right before rsa_encrypt functions did not work. For me, if rsa_gen_keys was succeed, the printf should print my message before enter the rsa_encrypt function. But it did not happen and I was not able to identify that the problem was inside the rsa_encrypt function, since the printf right before it was not working. If someone has an explanation for this, I really appreciate it.

Thank you @jforissier and @jbech-linaro .

[jforissier]

The probable cause why you did not see the printf message is output buffering. The process crashed before the C library could flush its standard output buffer. You can avoid this by calling fflush(stdout) yourself after each printf(), it may be convenient to write a helper function or a macro for this purpose. Alternatively, I believe that printing to stderr is not buffered so that would be another option: fprintf(stderr, "message").

Anyway, glad you finally made it work ;-)

[cezane]

Thank you also for these last comments, @jforissier .

Now I can continue my implementation until the next error... =)