search

warmcat / libwebsockets

canonical libwebsockets.org networking library
https://libwebsockets.org
Other
4.76k stars 1.48k forks source link

NULL context in lws_vhost_foreach_listen_wsi #2911

Open marwisihans opened 1 year ago

marwisihans commented 1 year ago

Hi, and thank you for your work on this library!

I'm using libwebsockets (version/tag 4.3.2) for my server, and somewhat randomly it crashes with the following stacktrace.

Thread 62 "LWS" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 569]
0x00007ffff7c91c1d in lws_vhost_foreach_listen_wsi (cx=0x0, arg=0x7fffcbf3acc0, cb=0x7ffff7ca39b3 )
    at /yocto/build/workspace/sources/libwebsockets/lib/core-net/vhost.c:1262
1262    /yocto/build/workspace/sources/libwebsockets/lib/core-net/vhost.c: No such file or directory.
(gdb) bt full
#0  0x00007ffff7c91c1d in lws_vhost_foreach_listen_wsi (cx=0x0, arg=0x7fffcbf3acc0, cb=0x7ffff7ca39b3 )
    at /yocto/build/workspace/sources/libwebsockets/lib/core-net/vhost.c:1262
        v = 0x7fffcbf3ac30
        n = 32767
#1  0x00007ffff7ca3acd in _lws_vhost_init_server_af (a=0x7fffcbf3acc0)
    at /yocto/build/workspace/sources/libwebsockets/lib/roles/http/server/server.c:102
        cx = 0x0
        pt = 0x7fffcbf3ad88
        n = 32767
        opt = 1
        limit = 1
        sockfd = 13489080
        wsi = 0x7fffcbf3aca0
        m = 0
        is = 0
        __func__ = "_lws_vhost_init_server_af"
#2  0x00007ffff7ca44ac in _lws_vhost_init_server (info=0x0, vhost=0x7fff74000c40)
    at /yocto/build/workspace/sources/libwebsockets/lib/roles/http/server/server.c:482
        a = {info = 0x0, vhost = 0x7fff74000c40, af = 2}
        __func__ = "_lws_vhost_init_server"
#3  0x00007ffff7c6d792 in lws_sul_plat_unix (sul=0xcdbab8)
    at /yocto/build/workspace/sources/libwebsockets/lib/plat/unix/unix-init.c:69
        v = 0x7fff74000c40
        pv = 0xcdbd90
        pt = 0xcdb9e0
        context = 0xcdb760
        n = 1
        m = 11
        __func__ = "lws_sul_plat_unix"
#4  0x00007ffff7c95a77 in __lws_sul_service_ripe (own=0xcdba28, own_len=2, usnow=104564988)
    at /yocto/build/workspace/sources/libwebsockets/lib/core-net/sorted-usec-list.c:161
        hit = 0xcdbab8
        lowest = 104564985
        n = 2
        pt = 0xcdb9e0
        __PRETTY_FUNCTION__ = "__lws_sul_service_ripe"
        __func__ = "__lws_sul_service_ripe"
#5  0x00007ffff7c6e40d in _lws_plat_service_tsi (context=0xcdb760, timeout_ms=2000000000, tsi=0)
    at /yocto/build/workspace/sources/libwebsockets/lib/plat/unix/unix-service.c:125
        ftp = 0x0
        next = 0x0
        vpt = 0xcdb9e0
        pt = 0xcdb9e0
        timeout_us = 2000000000000
        us = 104564988
        n = 0
        m = 0
#6  0x00007ffff7c6e685 in lws_plat_service (context=0xcdb760, timeout_ms=0)
    at /yocto/build/workspace/sources/libwebsockets/lib/plat/unix/unix-service.c:235
No locals.
#7  0x00007ffff7c95640 in lws_service (context=0xcdb760, timeout_ms=0)
    at /yocto/build/workspace/sources/libwebsockets/lib/core-net/service.c:838
        pt = 0xcdb9e0
        n = 0
#8  0x00000000006d050c in libwebsockets_task (param=0x0)
    at /yocto/build/workspace/sources/core/modules/server_libwebsockets.c:185
        n = 0
        __func__ = "libwebsockets_task"
#9  0x0000000000628e48 in _init_task (ref=0xce17f8) at /yocto/build/workspace/sources/core/modules/system/task.c:75
        taskdata = 0xce17f8
#10 0x00007ffff7e5fea4 in ?? () from /lib/libpthread.so.0
No symbol table info available.
#11 0x00007ffff767edcf in clone () from /lib/libc.so.6
No symbol table info available.
(gdb)

(gdb) frame
#1  0x00007ffff7ca3acd in _lws_vhost_init_server_af (a=0x7fffcbf3acc0)
    at /yocto/build-gtarx/workspace/sources/libwebsockets/lib/roles/http/server/server.c:102
102     in /yocto/build-gtarx/workspace/sources/libwebsockets/lib/roles/http/server/server.c
(gdb) p *a->vhost
$4 = {proxy_basic_auth_token = "\000\000\000\000\000\000\000\000\320\b\000t\377\177", '\000' , http = {
    http_proxy_address = '\000' , mount_list = 0x0, error_document_404 = 0x0, http_proxy_port = 0}, lc = {
    gutag = '\000' , list = {prev = 0x0, next = 0x0, owner = 0x0}, us_creation = 0, log_cx = 0x0}, 
  vh_being_destroyed_list = {prev = 0x0, next = 0x0, owner = 0x0}, tls_sessins = {tail = 0x0, head = 0x0, count = 0}, evlib_vh = 0x0, 
  options = 0, context = 0x0, vhost_next = 0x0, retry_policy = 0x0, listen_wsi = {tail = 0x0, head = 0x0, count = 0}, name = 0x0, 
  iface = 0x0, listen_accept_role = 0x0, listen_accept_protocol = 0x0, unix_socket_perms = 0x0, finalize = 0x0, finalize_arg = 0x0, 
  protocols = 0x0, protocol_vh_privs = 0x0, pvo = 0x0, headers = 0x0, same_vh_protocol_owner = 0x0, no_listener_vhost_list = 0x0, 
  abstract_instances_owner = {tail = 0x0, head = 0x0, count = 0}, dll_cli_active_conns_owner = {tail = 0x0, head = 0x0, count = 0}, 
  vh_awaiting_socket_owner = {tail = 0x0, head = 0x0, count = 0}, tls = {ssl_ctx = 0x0, ssl_client_ctx = 0x0, tcr = 0x0, alpn = 0x0, 
    ss = 0x0, alloc_cert_path = 0x0, key_path = 0x0, ecdh_curve = '\000' , alpn_ctx = {data = '\000' , 
      len = 0 '\000'}, use_ssl = 0, allow_non_ssl_on_ssl_port = 0, ssl_info_event_mask = 0, user_supplied_ssl_ctx = 0, skipped_certs = 0}, 
  user = 0x0, listen_port = 0, bind_iface = 0, count_protocols = 0, ka_time = 0, ka_probes = 0, ka_interval = 0, keepalive_timeout = 0, 
  timeout_secs_ah_idle = 0, connect_timeout_secs = 0, fo_listen_queue = 0, count_bound_wsi = 0, tls_session_cache_max = 0, 
  allocated_vhost_protocols = 0 '\000', created_vhost_protocols = 0 '\000', being_destroyed = 0 '\000', from_ss_policy = 0 '\000', 
  default_protocol_index = 0 '\000', raw_protocol_index = 0 '\000'}

We have tried to debug our code based on the stacktrace above, but we are unable to figure out why the context pointer of the vhost, and the vhost itself is null (seemingly uninitialized) or why it arrives where it does in the first place. We seem to crash at the same place each time, so at least it is consistent.

Below is an example based on our setup/config. I was wondering if there are any obvious issues with our configuration that eventually causes libwebsockets to reach this state?

Libwebsockets starts in "example_libwebsockets_init()".

#define example_LWS_DEFAULT_TIMEOUT_SECS 15
#define example_LWS_BUFFER_SIZE 16384
#define example_LWS_BUFFER_SIZE_LWS_RX (example_LWS_BUFFER_SIZE / 2)

#ifndef example_LIBWEBSOCKETS_INPUT_PORT
#define example_LIBWEBSOCKETS_INPUT_PORT 80
#endif

struct example_libwebsockets_context {};

static int example_libwebsockets_http_callback(struct lws *wsi, enum lws_callback_reasons reason, void *user, void *in, size_t len);
static int example_libwebsockets_websocket_callback(struct lws *wsi, enum lws_callback_reasons reason, void *user, void *in, size_t len);
static void example_libwebsockets_task(void *param);
static int example_libwebsockets_lws_vhost_init(struct lws_context ** const lws_context);
static void example_libwebsockets_lws_vhost_create_callback(void *param);
static struct lws_vhost *example_libwebsockets_lws_vhost_create_callback_helper(struct lws_context_creation_info * const info, const char * const vhost_name, const int port, const int interface_id);

static struct lws_context *example_libwebsockets_lws_context = NULL;
// static char example_libwebsockets_buffer[example_LWS_BUFFER_SIZE]; /* Generic buffer */
static struct log_module *example_log = NULL;

/** libwebsockets protocols
 */
static const struct lws_protocols example_libwebsockets_protocols[] = {
    { "http", example_libwebsockets_http_callback, sizeof (struct example_libwebsockets_context), example_LWS_BUFFER_SIZE_LWS_RX, 0, NULL, 0 },
    { "example", example_libwebsockets_websocket_callback, sizeof (struct example_libwebsockets_context), 0, 1, NULL, 0 },
    { NULL, NULL },
};

/** libwebsockets mountpoints
 *
 *  Only one mountpoint is defined; we handle different mountpoints in our own code instead
 */
static const struct lws_http_mount example_libwebsockets_http_mounts[] = {
    { .mountpoint = "/", .protocol = "http", .origin_protocol = LWSMPRO_CALLBACK, .mountpoint_len = 1 },
};

/** libwebsockets options
 */
static const struct lws_protocol_vhost_options example_libwebsockets_protocol_vhost_options[] = {
    { &example_libwebsockets_protocol_vhost_options[1], NULL, "http", "" },
    { &example_libwebsockets_protocol_vhost_options[2], &example_libwebsockets_protocol_vhost_options[2], "example", "" },
    { NULL, NULL, "default", "1" }
};

/** Policy for connection validity management
 */
static lws_retry_bo_t example_libwebsockets_retry_and_idle_policy = {
    .secs_since_valid_ping = 10,
    .secs_since_valid_hangup = 15,
};

void example_libwebsockets_init()
{
    example_log = log_init_module("lws", log_enabled);
    struct lws_context_creation_info context_creation_info;

    /* Create the libwebsockets context without any vhosts. */
    memset(&context_creation_info, 0, sizeof (context_creation_info));
    context_creation_info.options = LWS_SERVER_OPTION_EXPLICIT_VHOSTS;
    example_libwebsockets_lws_context = lws_create_context(&context_creation_info);

    if (example_libwebsockets_lws_context == NULL) {
        log_printf(example_log, log_priority_internal_error, "Failed to create lws context.");
        return;
    }

    task_create(example_libwebsockets_task, NULL, OS_TASK_DEFAULT_STACKSIZE, OS_PRIORITY_WICO, "LWS", 0);
}

static void example_libwebsockets_task(void *param)
{
    int n = 0;

    init_wait();

    if (example_libwebsockets_lws_vhost_init(&example_libwebsockets_lws_context) != 0) {
        log_printf(example_log, log_priority_internal_error, "Failed to initialize lws vhosts.");
        task_destruct(0);
        return;
    }

    /* Run main libwebsockets loop. */
    for (;;) {
        if ((n = lws_service(example_libwebsockets_lws_context, 0)) != 0) {
            log_printf(example_log, log_priority_internal_error, "lws_service failed");
        }
    }
}

/**
 * \brief Initialize vhosts.
 * 
 * \param lws_context Pointer to context pointer.
 * \return Returns 0 on success, -1 on error.
 */
static int example_libwebsockets_lws_vhost_init(struct lws_context ** const lws_context)
{
    int i;
    int n;
    int *interfaces;

    /*
     * Create vhosts for http and websockets.
     * 
     * (For each virtual interface, create a vhost.)
     */
    if ((n = networking_get_n_interfaces(NULL)) > 0) {
        interfaces = mem_alloc(n * sizeof (int));
        n = min(n, networking_get_n_interfaces(interfaces));

        for (i = 0; i < n; i++) {
            int *id = mem_alloc(sizeof(*id));
            *id = interfaces[i];
            example_libwebsockets_lws_vhost_create_callback(id);
        }
    }

    mem_free(interfaces);

  return 0;
}

/**
 * \brief Creates a vhost.
 * 
 * \param networking_interface_id Pointer to the id of the networking interface. This value will take ownership of this pointer/be freed by this callback.
 */
static void example_libwebsockets_lws_vhost_create_callback(void *param)
{
    char *ifname;
    char vhost_name[IFNAMSIZ];
    struct lws_context_creation_info context_creation_info;
    struct lws_vhost *vhost_http;
    int *networking_interface_id = param;

    /* Configure generic vhost options. */
    memset(&context_creation_info, 0, sizeof(context_creation_info));

    context_creation_info.options = LWS_SERVER_OPTION_ALLOW_LISTEN_SHARE; /* Allow vhosts to share ip tuples. */
    context_creation_info.protocols = example_libwebsockets_protocols;
    context_creation_info.pvo = example_libwebsockets_protocol_vhost_options;
    context_creation_info.mounts = example_libwebsockets_http_mounts;
    context_creation_info.retry_and_idle_policy = &example_libwebsockets_retry_and_idle_policy;
    context_creation_info.timeout_secs = example_LWS_DEFAULT_TIMEOUT_SECS;

    if (networking_interface_is_capable(networking_interface_id)) {
        context_creation_info.bind_iface = 1;
    }

    /* Get interface ifname. */
    ifname = mem_alloc(IFNAMSIZ);
    if (networking_get_interface_name(*networking_interface_id, ifname, IFNAMSIZ) == -1) {
        log_printf(example_log, log_priority_internal_error, "Could not find virtual interface ifname.");
        mem_free(ifname);
        ifname = NULL;
        goto bail;
    }
    ifname[IFNAMSIZ - 1] = '\0';
    context_creation_info.iface = ifname; /* This value has to be freed by us (i.e. not lws) when destroying the vhost. */

    /* Create vhost for http. */
    snprintf(&vhost_name[0], IFNAMSIZ, "%d", *networking_interface_id); /* The "vhost_name" will just be numbers 1,2,3... etc. We use this because netowrking_interface_id is unique, and we want our vhost_name to also be unique. */
    if ((vhost_http = example_libwebsockets_lws_vhost_create_callback_helper(&context_creation_info, vhost_name, example_LIBWEBSOCKETS_INPUT_PORT, *networking_interface_id)) == NULL) {
        log_printf(example_log, log_priority_internal_error, "Failed to create vhost (http).");
        mem_free(ifname);
        ifname = NULL;
        goto bail;
    }

    bail:
    mem_free(networking_interface_id);
    return;
}

/**
 * \brief Creats a vhost and opens respective port for it.
 * 
 * \param vhost_name Name of the vhost. This has to be unique! Basing it on networking id's is good, because those are unique.
 * \param port Port to the vhost.
 * \param interface_id Networking interface id.
 * \param tls Should the vhost created using TLS (/HTTPS)?
 * 
 * \return NULL on error. Otherwise pointer to lws vhost.
*/
static struct lws_vhost *example_libwebsockets_lws_vhost_create_callback_helper(struct lws_context_creation_info * const info, const char * const vhost_name, const int port, const int interface_id)
{
    struct lws_vhost *vhost = NULL;
    networking_traffic_request_parameters_t traffic_params;

    info->port = port;
    info->vhost_name = vhost_name; /* lws will copy this value. */

    memset(&traffic_params, 0, sizeof(traffic_params));
    traffic_params.protocol = networking_protocol_tcp;
    traffic_params.dst_port = info->port;
    if (networking_enable_interface(interface_id, traffic_params) != 0) {
        return NULL;
    }

    if ((vhost = lws_create_vhost(example_libwebsockets_lws_context, info)) == NULL) {
        if (networking_disable_interface(interface_id, traffic_params) != 0) {
            log_printf(example_log, log_priority_internal_debug0, "Failed to close networking port after failed vhost creation.");
        }
        return NULL;
    }

    return vhost;
}

static int example_libwebsockets_http_callback(struct lws *wsi, enum lws_callback_reasons reason, void *user, void *in, size_t len)
{
    /* CONFIGURATION EXAMPLE! callback definition ignored. */
    return -1;
}

static int example_libwebsockets_websocket_callback(struct lws *wsi, enum lws_callback_reasons reason, void *user, void *in, size_t len)
{
    /* CONFIGURATION EXAMPLE! callback definition ignored. */
    return -1;
}
lws-team commented 1 year ago

First move with time travelling crashes is run it under valgrind.

marwisihans commented 1 year ago

Sadly I don't have valgrind, only gdb because the platform is a custom build, built with yocto. But I will see what I can do!

lws-team commented 1 year ago

Yocto should have a valgrind recipe you can add to your image, or unpack the build of it into your rootfs if it's mutable.

marwisihans commented 1 year ago

I was able to get valgrind up and running!

I could not run with flags such as "--track-origins" because it was either way to slow and/or I could not reproduce the bug with them active. This is the output where lws was present.

root@(custom-linux):~# valgrind --vgdb=yes lws_example -D
==902== Memcheck, a memory error detector
==902== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==902== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==902== Command: lws_example -D
==902== 
==902== Thread 61 example LWS:
==902== Conditional jump or move depends on uninitialised value(s)
==902==    at 0x4FEB42: protobuf_encode_uint32 (protobuf.c:204)
==902==    by 0x6C891F: example_socket_method (example_socket.c:358)
==902==    by 0x6CA40D: example_socket_default_open (example_socket_default.c:136)
==902==    by 0x6C8BF4: example_socket_connected_callback (example_socket.c:438)
==902==    by 0x6C47EB: example_libwebsockets_raw_callback (example_libwebsockets.c:213)
==902==    by 0x4B78D36: lws_adopt_descriptor_vhost2 (adopt.c:434)
==902==    by 0x4B78EE9: lws_adopt_descriptor_vhost_via_info (adopt.c:545)
==902==    by 0x4B78E00: lws_adopt_descriptor_vhost (adopt.c:494)
==902==    by 0x4B97543: rops_handle_POLLIN_listen (ops-listen.c:147)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==    by 0x4B4D621: _lws_plat_service_tsi (unix-service.c:216)
==902== 
==902== Conditional jump or move depends on uninitialised value(s)
==902==    at 0x4FEB42: protobuf_encode_uint32 (protobuf.c:204)
==902==    by 0x6C98B7: example_socket_response_internal (example_socket.c:808)
==902==    by 0x6CA7BD: example_socket_default_method_open_session (example_socket_default.c:238)
==902==    by 0x6CA4BD: example_socket_default_method (example_socket_default.c:160)
==902==    by 0x6C968B: example_socket_handle_message (example_socket.c:762)
==902==    by 0x6C6BC4: example_libwebsockets_callback_receive (example_libwebsockets.c:1457)
==902==    by 0x6C4817: example_libwebsockets_raw_callback (example_libwebsockets.c:216)
==902==    by 0x4B75D4D: user_callback_handle_rxflow (wsi.c:498)
==902==    by 0x4B96C25: rops_handle_POLLIN_raw_skt (ops-raw-skt.c:208)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==    by 0x4B4D621: _lws_plat_service_tsi (unix-service.c:216)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B82C19: _lws_vhost_init_server_af (server.c:122)
==902==    by 0x4B834AB: _lws_vhost_init_server (server.c:482)
==902==    by 0x4B70538: lws_create_vhost (vhost.c:1010)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==    by 0x4B4D621: _lws_plat_service_tsi (unix-service.c:216)
==902==  Address 0x2c511c00 is 592 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 4
==902==    at 0x4B7688C: lws_broadcast (wsi.c:866)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==    by 0x4B4D621: _lws_plat_service_tsi (unix-service.c:216)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511cd0 is 800 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B7689B: lws_broadcast (wsi.c:874)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==    by 0x4B4D621: _lws_plat_service_tsi (unix-service.c:216)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511b78 is 456 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Conditional jump or move depends on uninitialised value(s)
==902==    at 0x4FEB42: protobuf_encode_uint32 (protobuf.c:204)
==902==    by 0x6C98B7: example_socket_response_internal (example_socket.c:808)
==902==    by 0x6CA606: example_socket_default_method_hello (example_socket_default.c:193)
==902==    by 0x6CA466: example_socket_default_method (example_socket_default.c:154)
==902==    by 0x6C968B: example_socket_handle_message (example_socket.c:762)
==902==    by 0x6C6BC4: example_libwebsockets_callback_receive (example_libwebsockets.c:1457)
==902==    by 0x6C5C72: example_libwebsockets_websocket_callback (example_libwebsockets.c:963)
==902==    by 0x4B75D4D: user_callback_handle_rxflow (wsi.c:498)
==902==    by 0x4B96429: lws_ws_frame_rest_is_payload (server-ws.c:970)
==902==    by 0x4B9656E: lws_parse_ws (server-ws.c:1052)
==902==    by 0x4B8E6D3: lws_read_h1 (ops-h1.c:270)
==902==    by 0x4B9150C: rops_handle_POLLIN_ws (ops-ws.c:1183)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B4C75E: lws_sul_plat_unix (unix-init.c:68)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511ba0 is 496 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 4
==902==    at 0x4B833DB: _lws_vhost_init_server (server.c:421)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511cc8 is 792 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 4
==902==    at 0x4B833EA: _lws_vhost_init_server (server.c:422)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511cc8 is 792 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B83403: _lws_vhost_init_server (server.c:435)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511b68 is 440 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B83424: _lws_vhost_init_server (server.c:447)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511ba8 is 504 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B82A62: _lws_vhost_init_server_af (server.c:87)
==902==    by 0x4B834AB: _lws_vhost_init_server (server.c:482)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511b70 is 448 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B82AB0: _lws_vhost_init_server_af (server.c:102)
==902==    by 0x4B834AB: _lws_vhost_init_server (server.c:482)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x2c511b70 is 448 bytes inside a block of size 854 free'd
==902==    at 0x4839900: free (vg_replace_malloc.c:540)
==902==    by 0x4B56FBB: _realloc (alloc.c:144)
==902==    by 0x4B56FF2: lws_realloc (alloc.c:154)
==902==    by 0x4B71537: __lws_vhost_destroy2 (vhost.c:1585)
==902==    by 0x4B715DF: lws_vhost_destroy (vhost.c:1631)
==902==    by 0x6C5B50: example_libwebsockets_lws_vhost_destroy_callback (example_libwebsockets.c:911)
==902==    by 0x6C5DC5: example_libwebsockets_callback_event_wait_cancelled (example_libwebsockets.c:1042)
==902==    by 0x6C4968: example_libwebsockets_http_callback (example_libwebsockets.c:301)
==902==    by 0x4B76876: lws_broadcast (wsi.c:869)
==902==    by 0x4B798DC: rops_handle_POLLIN_pipe (ops-pipe.c:111)
==902==    by 0x4B7442F: lws_service_fd_tsi (service.c:766)
==902==    by 0x4B4D214: _lws_plat_service_forced_tsi (unix-service.c:51)
==902==  Block was alloc'd at
==902==    at 0x4838694: malloc (vg_replace_malloc.c:308)
==902==    by 0x483ABE3: realloc (vg_replace_malloc.c:836)
==902==    by 0x4B56F61: _realloc (alloc.c:133)
==902==    by 0x4B5701D: lws_zalloc (alloc.c:159)
==902==    by 0x4B6F719: lws_create_vhost (vhost.c:599)
==902==    by 0x6C59C4: example_libwebsockets_lws_vhost_create_callback_helper (example_libwebsockets.c:872)
==902==    by 0x6C58BC: example_libwebsockets_lws_vhost_create_callback (example_libwebsockets.c:800)
==902==    by 0x6C5597: example_libwebsockets_lws_vhost_init (example_libwebsockets.c:715)
==902==    by 0x6C467C: example_libwebsockets_task (example_libwebsockets.c:175)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902== 
==902== Invalid read of size 8
==902==    at 0x4B70C1D: lws_vhost_foreach_listen_wsi (vhost.c:1262)
==902==    by 0x4B82ACC: _lws_vhost_init_server_af (server.c:102)
==902==    by 0x4B834AB: _lws_vhost_init_server (server.c:482)
==902==    by 0x4B4C791: lws_sul_plat_unix (unix-init.c:69)
==902==    by 0x4B74A76: __lws_sul_service_ripe (sorted-usec-list.c:161)
==902==    by 0x4B4D40C: _lws_plat_service_tsi (unix-service.c:125)
==902==    by 0x4B4D684: lws_plat_service (unix-service.c:235)
==902==    by 0x4B7463F: lws_service (service.c:838)
==902==    by 0x6C46D9: example_libwebsockets_task (example_libwebsockets.c:185)
==902==    by 0x61E16C: _init_task (task.c:75)
==902==    by 0x49A2EA3: ??? (in /lib/libpthread-2.31.so)
==902==    by 0x51C7DCE: clone (in /lib/libc-2.31.so)
==902==  Address 0x628 is not stack'd, malloc'd or (recently) free'd

The output above mentions a function "example_libwebsockets_lws_vhost_destroy_callback" which was not included in the previous issue entry. Here is the definition for it.

/**
 * \brief Finds a vhost and destroys it.
 * 
 * \param networking_interface_id Pointer to the id of an interface. This value will take ownership of this pointer/be freed by this function.
*/
static void example_libwebsockets_lws_vhost_destroy_callback(void *param)
{
    char vhost_name[IFNAMSIZ];
    char *ifname;
    struct lws_vhost *vhost = NULL;
    int *networking_interface_id = param;

    log_printf(example_log, log_priority_internal_debug3, "Removing vhost for interface: id = %d.", *networking_interface_id);

    /* Find vhost. */
    snprintf(vhost_name, IFNAMSIZ, "%d", *networking_interface_id);
    vhost = lws_get_vhost_by_name(example_libwebsockets_lws_context, vhost_name);
    if (vhost == NULL) {
        log_printf(example_log, log_priority_internal_debug0, "Unable to find vhost name given the networking interface id: %d", *networking_interface_id);
        goto bail;
    }

    /* Destroy the vhost. */
    ifname = (char *)lws_get_vhost_iface(vhost); /* Cast so we may free it. This was allocated during creation. */
    lws_vhost_destroy(vhost);
    mem_free(ifname);
    ifname = NULL;

    bail:
    mem_free(networking_interface_id);
    return;
}
marwisihans commented 1 year ago

Extra note which might be relevant: The interfaces on our platform do update/change every now and then.

Currently when an interface is updated, we ask libwebsockets to remove and re-add the vhost responsible for that interface with updated values. This happens by queuing _example_libwebsockets_lws_vhost_destroycallback() and _example_libwebsockets_lws_vhost_createcallback() and then call _lws_servicecancel(), which then will run them in that same order. Is it a problem that a vhost is removed and re-added within the same LWS_CALLBACK_EVENT_WAIT_CANCELLED event?