Slowdown bug with GraphVisual::update

[sebjameswml]

spwilson 3:04 PM Hi Seb - I'm finding that updating the data in a graph visual on each step of a sim leads to a massive slow down (up to a minute after about 1000 updates) at the end of a simulation. Should I be calling some kind of clear function after each update?

sebjames 3:24 PM How are you doing the update? are you using GraphView::append? 3:25 Like the example in morphologica/examples/graph4.cpp?

spwilson 3:26 PM no i'm not doing append, I'm overwriting the entire vector -- bad?!

sebjames 3:26 PM So you're calling gv->update()?

spwilson 3:26 PM yep

sebjames 3:26 PM Ok, that's slower

spwilson 3:27 PM seems like a cumulative thing, so longer I run it the slower the shutdown time (doesn't effect runtime speed) 3:27 I'll try append though

sebjames 3:27 PM gv->append only appends the primitive objects required to add on to your preexisting graph. See graph4.cpp 3:28 What you're describing sounds a bit like the terrible slowdown problem updating HexGridVisuals, so there may be a bug (edited)

spwilson 3:28 PM yes thats what i thought

sebjames 3:28 PM in the update() method, which makes it slower than it should be 3:29 I'll check - I can remember what that problem was - it was a failure to clear out the OpenGL indices and vertices 3:29 Yes! Same bug

spwilson 3:29 PM but that would slow at runtime, and get progressively worse. I don't notice runtime slowdown (printout of current time seems to increment about as long), just massive at the end, i.e., when cleqring the data

sebjames 3:30 PM It's a one line fix 3:30 I'll do it now and push

spwilson 3:30 PM ah yes, cool! 3:30 thanks

sebjames 3:36 PM Hmm, I was mistaken. When you call GraphVisual::update, it updates the data and the scalings and then it calls VisualDataModel::reinit which DOES clear out vertexPositions, vertexNormals, vertexColors and indices. So I don't know why you're getting a slow down.

[sebjameswml]

@stuartwilson (ping)

[claszlo123]

Hi Seb,

I'm having an issue I believe to be similar. I'm dynamically updating a graph of real-time simulation data and I call reinit() on every loop. Everything visually works and plots as expected, however there appears to be a memory leak associated with calling reinit(). Removing it from the loop stops the leak (but it is required to dynamically change our axes).

It appears the issue above may be related to the memory leak I'm seeing with reinit()? CPU usage does not appear to be affected but RAM linearly increases on runtime. Let me know your thoughts.

[sebjameswml]

Hello, Thanks for reporting that you're seeing a similar problem - and thanks for trying out the code! How did you determine that there's a leak? Valgrind? Do you know if it's a leak in system memory or to do with the gl buffers? I'm wondering if I can detect a leak with the morphologica example program graph2.cpp which updates a GraphVisual using GraphVisual::update(), which calls reinit(). I'll run that through valgrind in a minute. Do you have any code you can share? I'd be happy to try and reproduce the leak. Seb

[sebjameswml]

I've reproduced a memory leak (I can just watch the process's memory increase with time; no need for valgrind) with this program (which will soon be committed as examples/graph6.cpp). I suspect this is the same leak that you're experiencing. I have an idea that the problem is that the reinit() call inside GraphVisual::update() is failing to clear out text objects, so they're silting up.

// This is a graph which updates on each step. To test for a bug, but also to show how
// a graph can be completely redrawn each time, if required.
#include <morph/Visual.h>
#include <morph/GraphVisual.h>
#include <morph/vVector.h>
#include <morph/MathConst.h>

int main()
{
    morph::Visual v(1024, 768, "Continuous redrawing of GraphVisual");

    auto gv = new morph::GraphVisual<double> (v.shaderprog, v.tshaderprog, {0,0,0});

    morph::vVector<double> x;
    x.linspace (-morph::mathconst<double>::pi, morph::mathconst<double>::pi, 100);

    double dx = 0.0;

    gv->setdata (x, (x+dx).sin());
    gv->finalize();

    v.addVisualModel (gv);

    while (v.readyToFinish == false) {
        dx += 0.01;
        glfwWaitEventsTimeout (0.01667); // 16.67 ms ~ 60 Hz
        gv->update (x, (x+dx).sin(), 0);
        v.render();
    }

    return 0;
}

[sebjameswml]

Hi @claszlo123 , see if commit fe629c2 resolves this for you. It does so for the test program above (which is now in the examples directory and compiles as examples/graph6)

[claszlo123]

Looks like it did, thank you! Apologies for the slow response I was trying to put together a proper report to show the memory leak, I also just found it by watching memory usage increase with time. Looks like everything is working great now.

[sebjameswml]

Yes. I actually did run with valgrind, but as the memory was getting deallocated by the GL library on program exit, valgrind didn't complain about it! Glad it works. This was a good fix and plugs a hole that has been sitting around for a while.

[sebjameswml]

I'm going to close this issue, as I think it is now resolved. I think @stuartwilson used a workaround to avoid the memory leak issue when he ran into it and with a helpful report from @claszlo123 the memory leak is now sorted. Btw, @claszlo123 - I'd be interested to hear about your project, so please let me know if you have anything you can share, especially any papers or software that you release publicly.

[claszlo123]

@sebjameswml My team and I just concluded our project-- it was a senior design project for our Mechanical Engineering degrees at Purdue University. We've got all the details at https://realfeelbraking.weebly.com/ along with a product demo if you're interested. The basic idea is that it's a system that connects to the brake pedal of an automotive racing simulator to provide adjustable feedback based on the vehicle being driven. It eliminates the need to exchange springs/bump rubbers to change the "feel" of the brake to the user.

In the beginning of the semester we needed a way to take in data from our hall effect sensor and visualize it on screen primarily for development purposes. I found Morphologica online and it seemed to suit our needs perfectly. We left the functionality in our program so that users of the product can get live feedback about brake travel and commanded torque from the motor. I can share the code with you once we get everything wrapped up and organized and even show a video of the user interface we developed (you won't be able to run the code because it will crash if not connected to the simulator). We're very appreciative of the work you did on Morphologica to help us with our project!

[sebjameswml]

Congratulations on finishing your project! It looks like you did an amazing job on the simulated brake system - a future product for serious race sim gamers, surely? I'd love to see a video of the live brake travel feedback in action. Really glad you found my code useful!