Memory Leak?

[tolga-talkme]

Hi We are using concurrentqueue in our trading application for transfering data between threads, and we are very pleased about the performance of this queue. However there is a significant memory leak problem in the library, We are transferring all market data from 1 producer to multiple consumers(approx 10-15 consumers) and when we do that the memory usage increases 100mb in every 1 seconds, so at the end of the day we are consuming at least 200gb of memory. When we remove the concurrentqueue and replace it with intel tbb concurrent queue library, memory leak dissapears. Here is how we use the concurrentqueue:

#pragma once

#include <memory>
#include "IdGenerator.h"
#include "../Globals/Enumarations.h"
#include "../Containers/concurrentqueue.h"

template<class T = char*, class L = moodycamel::ConcurrentQueue<T>>

class Subscriber: private IdGenerator {
public:
    Subscriber() : id(++idGenerater) {
    }
    virtual ~Subscriber(){
    }

    void add(const T& message) {
        messageList.enqueue(message);
    }
    bool tryPop(T& item) {
        return messageList.try_dequeue(item);
    }
    L& getList() {
        return messageList;
    }
    unsigned short getId()const {
        return id;
    }
    void clear()
    {
    }
protected:
    const short id;
    L messageList;
};

template<class T, class L> using SubscriberPtr = std::shared_ptr<Subscriber<T, L>>;

[cameron314]

Why was this closed immediately after opening? Are the consumers keeping up with the producers? Are there many temporary threads enqueueing elements? Is this reproducible with the latest version of the queue?

[cameron314]

Also, 100 MB per second is 360 GB per hour, but only 200 GB are in use at the end of the day. Are you sure this is a leak and not just the peak memory usage? (The queue recycles memory internally but never returns it to libc/the OS until destruction).