amd64 - ubuntu:20.04 - dccl_test_dynamic_conditions_v4 failing LeakSanitizer

[psmskelton]

While trying to trace down the source of some issues on armhf and arm64 platforms, I thought I'd check my amd64 development platform first. While doing so, I noticed the 4.0 repository is failing an lsan test, although the CI *san tests are passing.

=================================================================
==91==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 63992 byte(s) in 421 object(s) allocated from:
    #0 0x7f457801d1f8 in __interceptor_realloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:164
    #1 0x7f4577eaa737 in pbL_grow /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1401
    #2 0x7f4577eab9fd in pbL_OneofDescriptorProto /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1522
    #3 0x7f4577eac13a in pbL_DescriptorProto /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1547
    #4 0x7f4577eac4e4 in pbL_FileDescriptorProto /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1566
    #5 0x7f4577eac96b in pbL_FileDescriptorSet /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1585
    #6 0x7f4577eaf92a in pb_load /tmp/dccl/src/thirdparty/lua-protobuf/pb.h:1715
    #7 0x7f4577ebb522 in Lpb_load /tmp/dccl/src/thirdparty/lua-protobuf/pb.c:1198
    #8 0x7f45773a0584  (/usr/lib/x86_64-linux-gnu/liblua5.3.so.0+0xd584)

SUMMARY: AddressSanitizer: 63992 byte(s) leaked in 421 allocation(s).

97% tests passed, 1 tests failed out of 32

Total Test time (real) =  20.34 sec

The following tests FAILED:
         32 - dccl_test_dynamic_conditions_v4 (Failed)

Horribly mangled Dockerfile for easy testing:

FROM ubuntu:20.04
SHELL ["/bin/bash", "-c"]
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update \
  && apt-get install -y \
    git \
    cmake \
    libprotobuf-dev \
    libprotoc-dev \
    protobuf-compiler \
    libb64-dev \
    liblua5.3-dev \
    libcrypto++-dev \
    python3-pip \
  && rm -rf /var/lib/apt/lists/*

RUN pip3 install --force-reinstall "protobuf==3.20.0"

WORKDIR /tmp/dccl
RUN git clone https://github.com/GobySoft/dccl.git --branch 4.0 --single-branch .
WORKDIR /tmp/dccl/build
RUN cmake -D enable_b64=ON -D enable_lua=ON -D enable_cryptography=ON -D SANITIZE_ADDRESS=ON -D enable_testing=ON ..
RUN make -j install
WORKDIR /tmp/dccl/python
RUN python3 setup.py install
WORKDIR /tmp/dccl/build
RUN ctest --output-on-failure > ctest_outputs.log || true
WORKDIR /deployment
CMD ["/bin/bash"]

---

EDIT 1: Fixed cmake call typo.

[psmskelton]

This and #116 are solved by bringing the src/thirdparty/lua-protobuf library up-to-date. Specifically, https://github.com/starwing/lua-protobuf/issues/156 and https://github.com/starwing/lua-protobuf/pull/181 both identified the leak in 2022 caused by a missing delete on the oneof_decl.

@tsaubergine Any ideas how the CI missed this?

[tsaubergine]

The CI doesn't run the unit tests on arm builds as these are cross-compile builds. Now that CircleCI supports ARM builds natively I can consider running the unit tests for arm64 automatically: https://circleci.com/docs/using-arm/

[tsaubergine]

Oh, this is on amd64. Hmm...quick look suggests this should run the same test: https://app.circleci.com/pipelines/github/GobySoft/dccl/221/workflows/6225951e-b4a0-4fea-9340-4f67989182c2/jobs/4164

[tsaubergine]

Yes I can easily replicate what you see on my laptop, but it doesn't fail on CircleCI ... that's annoying...

[tsaubergine]

Ah, apparently I was suppressing leaks from liblua5.3.so: https://github.com/GobySoft/dccl/blob/4.0/.circleci/config.yml#L198

I probably thought this was a false positive in Lua but actually comes from lua-protobuf

[psmskelton]

Ahhh, yes, that would explain the whole CI missing it part :)