beehive-lab / mambo

A low-overhead dynamic binary instrumentation and modification tool for ARM (both AArch32 and AArch64 support) and RISC-V (RV64GC).
Apache License 2.0
318 stars 69 forks source link

Simple multithreaded program hangs on RISC-V #97

Closed TylerJordan1 closed 7 months ago

TylerJordan1 commented 11 months ago

Hi,

I am trying to use MAMBO on RISC-V. It works just fine for single-threaded applications. However, when I try running multithreaded applications through it it just hangs right at the moment a thread is created.

I reproduced the issue on simple program which creates a thread that prints something. Here's the reproducer:

$ cat mt_test.c

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>

void *thread2(void *vargp)
{
        sleep(1);
        printf("Hello world from thread2\n");
        return NULL;
}

int main()
{
        pthread_t thread_id;
        printf("Before starting thread 2\n");
        int error = pthread_create(&thread_id, NULL, thread2, NULL);
        if (error != 0) {
                fprintf(stderr, "pthread_create: %s\n", strerror(error));
                return 1;
        }
        error = pthread_join(thread_id, NULL);
        if (error != 0) {
                fprintf(stderr, "pthread_join: %s\n", strerror(error));
                return 1;
        }
        printf("After joining thread 2\n");
        return 0;
}

Compile with:

$ gcc mt_test.c -pthread -o riscv-mt-dummy-bin

Then run it without MAMBO:

$ ./riscv-mt-dummy-bin
Before starting thread 2
Hello world from thread2
After joining thread 2

Works fine. Running through MAMBO:

$ ./dbm ./riscv-mt-dummy-bin
Before starting thread 2
^C^ZKilled

The program just hangs and doesn't seem to respond to SIGINT. Sending SIGKILL kills it. No mambo plugins are enabled.

Also I get the following output when running through gdb:

$ gdb --args ./dbm ./riscv-mt-dummy-bin
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
...
Reading symbols from ./dbm...
(gdb) r
Starting program: /home/ubuntu/work/projects/mambo/dbm ./riscv-mt-dummy-bin
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/riscv64-linux-gnu/libthread_db.so.1".
Before starting thread 2
[New Thread 0x7ffff7fd5480 (LWP 1851)]
[Thread 0x7ffff7fd5480 (LWP 1851) exited]

<gdb hangs>

The newly created thread exits instantly.

My configuration: Ubuntu 22.04.3 LTS inside QEMU v8.1.0. CPU architecture is rv64imafdch.

My question is - Is it expected? Does MAMBO support multithreading? If it does, what could have gone wrong here?

jkressel commented 11 months ago

Hi @TylerJordan1 Thanks for your interest! Currently this is expected behaviour, however work to support multi-threading has been ongoing behind the scenes. Support for multi-threaded applications such as the test case you provided will be released in the coming weeks :)

TylerJordan1 commented 11 months ago

Hi @TylerJordan1 Thanks for your interest! Currently this is expected behaviour, however work to support multi-threading has been ongoing behind the scenes. Support for multi-threaded applications such as the test case you provided will be released in the coming weeks :)

Oh, okay. Thank you for the quick response!

jkressel commented 11 months ago

@TylerJordan1 Support for multithreading has been merged in :)