Corrupted generated trace with barectf 3.0.1

[eepp]

This was originally reported by Yoann Farré on the lttng-dev mailing list:

For the context, I use the tool Papyrus to build UML models and then generate C++ code. The code can be instrumented to generate CTF traces (with barectf) and follow some relevant model elements.

More precisely, I generated a trace and tried to read it with TraceCompass. The traces seems to contains a malformed event. Unfortunately it avoids to read the follow-up (if any).

I also tried to read the trace with babeltrace which clearly shows that there is a malformed event [1].

Attached trace: ctftrace04.zip

This trace shows that he's using barectf 3.0.1:

...
 * The following code was generated by barectf v3.0.1
 * on 2021-11-26T15:34:36.165563.
 *
 * For more details, see <https://barectf.org/>.
...

This requires further investigation.

[eepp]

Can you provide the platform code you're using?

[yoannfarre]

Yes. There is the code of the platform. It is the example code for a platform given on barectf website.

From my first message, I think the issue is due to the use of the same context from several threads, which leads to writing some traces from several threads in the same file.

barectf-platform-fs.h

#ifndef _BARECTF_PLATFORM_fs_H
#define _BARECTF_PLATFORM_fs_H

/*
 * Copyright (c) 2015 EfficiOS Inc. and Linux Foundation
 * Copyright (c) 2015-2020 Philippe Proulx <pproulx@efficios.com>
 *
 * Permission is hereby granted, free of charge, to any person obtaining
 * a copy of this software and associated documentation files (the
 * "Software"), to deal in the Software without restriction, including
 * without limitation the rights to use, copy, modify, merge, publish,
 * distribute, sublicense, and/or sell copies of the Software, and to
 * permit persons to whom the Software is furnished to do so, subject to
 * the following conditions:
 *
 * The above copyright notice and this permission notice shall be
 * included in all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 */

#include <stdint.h>

#ifdef __cplusplus
extern "C" {
#endif

struct barectf_default_ctx;
struct barectf_platform_fs_ctx;

struct barectf_platform_fs_ctx *barectf_platform_fs_init(
    unsigned int buf_size, const char *data_stream_file_path,
    int simulate_full_backend, unsigned int full_backend_rand_max,
    unsigned int full_backend_rand_lt);

void barectf_platform_fs_fini(struct barectf_platform_fs_ctx *ctx);

struct barectf_default_ctx *barectf_platform_fs_get_barectf_ctx(
    struct barectf_platform_fs_ctx *ctx);

#ifdef __cplusplus
}
#endif

#endif /* _BARECTF_PLATFORM_fs_H */

barectf-platform-fs.c

/*
 * Copyright (c) 2015 EfficiOS Inc. and Linux Foundation
 * Copyright (c) 2015-2020 Philippe Proulx <pproulx@efficios.com>
 *
 * Permission is hereby granted, free of charge, to any person obtaining
 * a copy of this software and associated documentation files (the
 * "Software"), to deal in the Software without restriction, including
 * without limitation the rights to use, copy, modify, merge, publish,
 * distribute, sublicense, and/or sell copies of the Software, and to
 * permit persons to whom the Software is furnished to do so, subject to
 * the following conditions:
 *
 * The above copyright notice and this permission notice shall be
 * included in all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
 * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
 * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
 * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
 * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 */

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <assert.h>
#include <time.h>

#include "barectf-platform-fs.h"
#include "barectf.h"

#ifdef __cplusplus
# define _FROM_VOID_PTR(_type, _value)  static_cast<_type *>(_value)
#else
# define _FROM_VOID_PTR(_type, _value)  ((_type *) (_value))
#endif

struct barectf_platform_fs_ctx {
    struct barectf_default_ctx ctx;
    FILE *fh;
    int simulate_full_backend;
    unsigned int full_backend_rand_lt;
    unsigned int full_backend_rand_max;
};

static uint64_t get_clock(void * const data)
{
    struct timespec ts;

    clock_gettime(CLOCK_REALTIME, &ts);
    return ts.tv_sec * 1000000000ULL + ts.tv_nsec;
}

static void write_packet(const struct barectf_platform_fs_ctx * const platform_ctx)
{
    const size_t nmemb = fwrite(barectf_packet_buf(&platform_ctx->ctx),
        barectf_packet_buf_size(&platform_ctx->ctx), 1, platform_ctx->fh);

    assert(nmemb == 1);
}

static int is_backend_full(void * const data)
{
    int is_backend_full = 0;
    const struct barectf_platform_fs_ctx * const platform_ctx =
        _FROM_VOID_PTR(const struct barectf_platform_fs_ctx, data);

    if (platform_ctx->simulate_full_backend) {
        if (rand() % platform_ctx->full_backend_rand_max <
                platform_ctx->full_backend_rand_lt) {
            is_backend_full = 1;
            goto end;
        }
    }

end:
    return is_backend_full;
}

static void open_packet(void * const data)
{
    struct barectf_platform_fs_ctx * const platform_ctx =
        _FROM_VOID_PTR(struct barectf_platform_fs_ctx, data);

    barectf_default_open_packet(&platform_ctx->ctx);
}

static void close_packet(void * const data)
{
    struct barectf_platform_fs_ctx * const platform_ctx =
        _FROM_VOID_PTR(struct barectf_platform_fs_ctx, data);

    /* Close packet now */
    barectf_default_close_packet(&platform_ctx->ctx);

    /* Write packet to file */
    write_packet(platform_ctx);
}

struct barectf_platform_fs_ctx *barectf_platform_fs_init(
    const unsigned int buf_size, const char * const data_stream_file_path,
    const int simulate_full_backend,
    const unsigned int full_backend_rand_lt,
    const unsigned int full_backend_rand_max)
{
    uint8_t *buf = NULL;
    struct barectf_platform_fs_ctx *platform_ctx;
    struct barectf_platform_callbacks cbs;

    cbs.default_clock_get_value = get_clock;
    cbs.is_backend_full = is_backend_full;
    cbs.open_packet = open_packet;
    cbs.close_packet = close_packet;
    platform_ctx = _FROM_VOID_PTR(struct barectf_platform_fs_ctx,
        malloc(sizeof(*platform_ctx)));

    if (!platform_ctx) {
        goto error;
    }

    buf = _FROM_VOID_PTR(uint8_t, malloc(buf_size));

    if (!buf) {
        goto error;
    }

    platform_ctx->fh = fopen(data_stream_file_path, "wb");

    if (!platform_ctx->fh) {
        goto error;
    }

    platform_ctx->simulate_full_backend = simulate_full_backend;
    platform_ctx->full_backend_rand_lt = full_backend_rand_lt;
    platform_ctx->full_backend_rand_max = full_backend_rand_max;
    barectf_init(&platform_ctx->ctx, buf, buf_size, cbs, platform_ctx);
    open_packet(platform_ctx);
    goto end;

error:
    free(platform_ctx);
    free(buf);

end:
    return platform_ctx;
}

void barectf_platform_fs_fini(struct barectf_platform_fs_ctx * const platform_ctx)
{
    if (barectf_packet_is_open(&platform_ctx->ctx) &&
            !barectf_packet_is_empty(&platform_ctx->ctx)) {
        close_packet(platform_ctx);
    }

    fclose(platform_ctx->fh);
    free(barectf_packet_buf(&platform_ctx->ctx));
    free(platform_ctx);
}

struct barectf_default_ctx *barectf_platform_fs_get_barectf_ctx(
    struct barectf_platform_fs_ctx * const platform_ctx)
{
    return &platform_ctx->ctx;
}

[yoannfarre]

By using one context by thread, the generated trace is correct. There is no issue with barectf, only the need to manage the thread safety (as said in the documentation). I even didn't change the platform code, only the way to init the used contexts.

Therefore I think this issue can be closed.

[eepp]

By using one context by thread, the generated trace is correct. There is no issue with barectf, only the need to manage the thread safety (as said in the documentation). I even didn't change the platform code, only the way to init the used contexts.

You are quite right.

This section of the documentation says:

The C source code which barectf generates doesn’t guarantee any concurrent access safety (thread safety, reentrancy).

Because barectf generates general ANSI C code with no dependencies, it can’t know how to synchronize accesses to barectf context structures. As of barectf 3.0, you need to protect tracing function calls for a given barectf context with your own synchronization primitives.

Note, however, that CTF is designed for fast multicore/multithread tracing: dedicate one barectf context (one data stream) per core/thread to avoid tracing function locking.

Consequently, your solution of one barectf context per thread is correct.

Please understand that the linux-fs platform exists for demonstration purposes only, and that it could be improved in many ways. The expectation with barectf is that each user writes its own platform because bare metal systems are so different from one another.

If you're tracing on Linux, please have a look at LTTng, a full-fledged user space and kernel tracing toolkit which, like barectf, produces CTF.

I'm closing this issue now, but you can still comment if needed.

[yoannfarre]

Thank you for the clarification. We have the constraint to be able to generate traces on Linux and Windows platforms (using cygwin or mingw), that is the reason why LTTng was first excluded. As far as I know, barectf is the best tool that provides this feature. Thanks again.