wasi pthread_mutex_lock issue

[tkernelcn]

test code as below, wasi-sdk version : https://github.com/WebAssembly/wasi-libc/releases/tag/wasi-sdk-21

when .wasm running, a lot of warning infos, is it a bug? Thanks.

infos:

[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                    
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                            
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                            
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                       
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                  
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                 
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                   
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                   
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                   
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                    
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                    
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                                                                                                                                    
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                                                                                                        
[00:00:12:000 - 5676AB60]: HashMap find elem failed: map or key is NULL.                                                                                                                      

source

#include <stdio.h>
#include <pthread.h>

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
int buffer;
_Noreturn void pthread_exit(void *);

void* producer(void* arg) {
    int produced;
    for (produced = 0; produced < 10; ++produced) {
        printf("producer lock\n");
        pthread_mutex_lock(&mutex);
        buffer = produced;
        pthread_cond_signal(&cond);
        printf("producer unlock\n");
        pthread_mutex_unlock(&mutex);
    }
    printf("producer exit\n");
    pthread_exit(NULL);
    return NULL;
}

void* consumer(void* arg) {
    int consumed, got;
    for (consumed = 0; consumed < 10; ++consumed) {
        printf("consumer lock\n");
        pthread_mutex_lock(&mutex);
        while (consumed > buffer) {
            pthread_cond_wait(&cond, &mutex);   // release mutex, wait for signal, on return re-acquire mutex
        }
        got = buffer;
        printf("consumer unlock\n");
        pthread_mutex_unlock(&mutex);
        if (consumed != got) {
            printf("Bug! consumed=%d, got=%d\n", consumed, got);
        }
    }
    printf("consumer exit\n");
    pthread_exit(NULL);
    return NULL;
}

int main(int argc, char *argv[])
{
    pthread_t cons, prod;
    pthread_create(&cons, NULL, consumer, NULL);
    pthread_create(&prod, NULL, producer, NULL);

    pthread_join(cons, NULL);
    pthread_join(prod, NULL);
    return 0;
}

[tkernelcn]

new findings:

seems wasi not support:

pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;

change to below, it works.

    pthread_mutex_init(&mutex, NULL);
    pthread_cond_init(&cond, NULL);

[wenyongh]

Hi, "HashMap find elem failed: map or key is NULL" should be not an error, could you please try patch below:

diff --git a/core/iwasm/common/wasm_shared_memory.c b/core/iwasm/common/wasm_shared_memory.c
index 70e84a37..f33dc043 100644
--- a/core/iwasm/common/wasm_shared_memory.c
+++ b/core/iwasm/common/wasm_shared_memory.c
@@ -176,7 +176,7 @@ acquire_wait_info(void *address, AtomicWaitNode *wait_node)
     AtomicWaitInfo *wait_info = NULL;
     bh_list_status ret;

-    if (address)
+    if (wait_map && address)
         wait_info = (AtomicWaitInfo *)bh_hash_map_find(wait_map, address);

     if (!wait_node) {

[tkernelcn]

sure, It's not a bug, if I use below initialize， it will disappear pthread_mutex_init(&mutex, NULL); pthread_cond_init(&cond, NULL);

if I use： pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; pthread_cond_t cond = PTHREAD_COND_INITIALIZER;

even apply your patch， also have the warnings： [00:00:09:000 - 567A7918]: HashMap find elem failed: map or key is NULL.

[wenyongh]

Do you rebuild iwasm? If the warning keeps showing, could you help debug iwasm, e.g. add break point at https://github.com/bytecodealliance/wasm-micro-runtime/blob/8b37048823677d9229b7458b064a515b19744c9c/core/shared/utils/bh_hashmap.c#L134 and check which function is calling bh_hash_map_find?

[tkernelcn]

first finding at here

#0  bh_hash_map_find (map=0xf6000700, key=0x0) at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/shared/utils/bh_hashmap.c:134
#1  0x565a9b31 in get_thread_info (exec_env=0xf6c0b990, handle=0)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:475
#2  0x565aa3bb in pthread_mutex_lock_wrapper (exec_env=0xf6c0b990, mutex=0xf7faf558)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:851
#3  0x5655fdc5 in skip_push_args ()
#4  0xf6c0b990 in ?? ()
#5  0x5655e3c1 in wasm_runtime_invoke_native (exec_env=0xf6c0b990, func_ptr=0x565aa398 <pthread_mutex_lock_wrapper>, func_type=0xf6c01030, signature=0x56605962 "(*)i", attachment=0x0,
    argv=0xf6c0c0b8, argc=1, argv_ret=0xf5ffe25c) at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:4312
#6  0x565658fa in wasm_interp_call_func_native (module_inst=0xf6c127d0, exec_env=0xf6c0b990, cur_func=0xf6c14470, prev_frame=0xf6c0c030)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:933
#7  0x5657bc3d in wasm_interp_call_func_bytecode (module=0xf6c127d0, exec_env=0xf6c0b990, cur_func=0xf6c14470, prev_frame=0xf6c0c030)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:3902
#8  0x5657c5b8 in wasm_interp_call_wasm (module_inst=0xf6c127d0, exec_env=0xf6c0b990, function=0xf6c14a10, argc=1, argv=0xf5fff2ac)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_interp_classic.c:4295
#9  0x56563512 in call_indirect (exec_env=0xf6c0b990, tbl_idx=0, elem_idx=1, argc=1, argv=0xf5fff2ac, check_type_idx=false, type_idx=0)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_runtime.c:2681
#10 0x5656356d in wasm_call_indirect (exec_env=0xf6c0b990, tbl_idx=0, elem_idx=1, argc=1, argv=0xf5fff2ac)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/interpreter/wasm_runtime.c:2693
#11 0x5655e4f8 in wasm_runtime_call_indirect (exec_env=0xf6c0b990, element_index=1, argc=1, argv=0xf5fff2ac)
    at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/common/wasm_runtime_common.c:4719
#12 0x565a9c8c in pthread_start_routine (arg=0xf6c0b990) at /home/peter/git_workspace/invest_wasm/lib/wasm/wasm-micro-runtime/core/iwasm/libraries/lib-pthread/lib_pthread_wrapper.c:515

[yamt]

FYI, wamr supports two pthread implementations: https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/pthread_impls.md it seems that you are using the old one. is it intentional? iirc, the old one doesn't have PTHREAD_MUTEX_INITIALIZER at all. i wonder how you built your wasm module.

[tkernelcn]

embed iwasm build with config script:

set (WAMR_BUILD_LIBC_BUILTIN 1)
set (WAMR_BUILD_LIBC_UVWASI 0)
set (WAMR_BUILD_LIBC_WASI 1)
set (WAMR_BUILD_MULTI_MODULE 1)
set (WAMR_BUILD_MINI_LOADER 0)
set (WAMR_BUILD_SHARED_MEMORY 1)
set (WAMR_BUILD_BULK_MEMORY 1)
set (WAMR_BUILD_THREAD_MGR 1)
set (WAMR_BUILD_LIB_PTHREAD 1)
set (WAMR_BUILD_LIB_PTHREAD_SEMAPHORE 1)
set (WAMR_BUILD_LIB_WASI_THREADS 0)      #error: allocate_aux_stack():WASM_ENABLE_HEAP_AUX_STACK_ALLOCATION

so maybe not correct due to I use wasi-sdk to build app:

cmake_minimum_required(VERSION 2.8)
project(test-apps)

set(WAMR_ROOT_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../wasm-micro-runtime)

set(CMAKE_SYSTEM_PROCESSOR wasm32)
#set (CMAKE_SYSROOT                  ${WAMR_ROOT_DIR}/wamr-sdk/app/libc-builtin-sysroot)
set (CMAKE_SYSROOT                  /opt/wasi-sdk/share/wasi-sysroot)

if (NOT DEFINED WASI_SDK_DIR)
    set (WASI_SDK_DIR               "/opt/wasi-sdk")
endif ()

set (CMAKE_C_FLAGS                  "-pthread -O3")
set (CMAKE_C_COMPILER_TARGET        "wasm32-wasi")
set (CMAKE_C_COMPILER               "${WASI_SDK_DIR}/bin/clang")

set (DEFINED_SYMBOLS
"${CMAKE_SYSROOT}/share/defined-symbols.txt")

set (CMAKE_EXE_LINKER_FLAGS
     "-Wl,--no-entry,--strip-all,                       \
      -Wl,--export=__heap_base,--export=__data_end      \
      -Wl,--export=malloc -Wl,--export=free             \
      -Wl,--export=main -Wl,--export=__main_argc_argv   \
      -Wl,--allow-undefined,--no-check-features"
      #-Wl,--allow-undefined-file=${DEFINED_SYMBOLS}"
)

file (GLOB_RECURSE source_all ${CMAKE_CURRENT_SOURCE_DIR}/*.c)
add_executable(test-apps.wasm  ${source_all})
target_link_libraries(test-apps.wasm)

@yamt so is it new approach? is it correct? Thanks.

[yamt]

pthread.h in /opt/wasi-sdk/share/wasi-sysroot is for the new one. pthread.h in ${WAMR_ROOT_DIR}/wamr-sdk/app/libc-builtin-sysroot is for the old one.

otoh, WAMR_BUILD_LIB_WASI_THREADS is for the new one. WAMR_BUILD_LIB_PTHREAD and WAMR_BUILD_LIB_PTHREAD_SEMAPHORE are for the old one.

it seems that you are somehow mixing the old one and new one. it isn't expected to work because they are ABI-incompatible.

[tkernelcn]

if I use the new on configurations:

set (WAMR_BUILD_LIB_PTHREAD 0)
set (WAMR_BUILD_LIB_PTHREAD_SEMAPHORE 0) 
set (WAMR_BUILD_LIB_WASI_THREADS 1) 

the running errors:

[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_mutex_lock)                                                                                                  
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_cond_signal)                                                                                                 
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_mutex_unlock)                                                                                                
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_cond_wait)                                                                                                   
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_create)                                                                                                      
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_join)                                                                                                        
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_cond_broadcast)                                                                                              
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_mutex_init)                                                                                                  
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_cond_init)
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_mutex_destroy)
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_cond_destroy)
[00:00:00:000 - 5671A410]: warning: failed to link import function (env, pthread_exit)

if use below, no errors

set (WAMR_BUILD_LIB_PTHREAD 1)
set (WAMR_BUILD_LIB_PTHREAD_SEMAPHORE 1) 
set (WAMR_BUILD_LIB_WASI_THREADS 0) 

very strange.