Axis change detection / caching?

[jgarvin]

I have a data set I want to scatter plot, but a lot of the data points are redundant in the sense that many of them are so close together in plot space that they end up occupying the same pixel in screen space. This means a lot of redundant useless rendering is going on, so I attempted to write some code to preprocess my data that would remove all of the redundant points that would go to the same pixel.

With ImPlot::PlotToPixels it's possible to write the code that will generate the new data set. I make a bitset with 1 bit per pixel, iterate all my data points converting them to pixel coordinates, and if the associated bit is not set set it and add the point to my data set. That works dandy.

The problem is in figuring out when I need to rerun the processing step. Because I control the generation of the original data set I know when it changes already, but as far as I can tell the API gives me no easy way to check when the axes change. I can get the limits of each axis and the pixel size and make sure that hasn't changed but I can't tell if log scale, opposite, or any of the other axis options have been toggled. ImPlot::PlotToPixels appears to use ImPlotAxis objects that I could compare for changes, but this type is fully hidden inside implot.cpp.

Is there a recommended away to do this?

[jgarvin]

Right after writing this realized I can check if Pixels2Plot returns the same value for the corners. Maybe there's a better way though?

[jgarvin]

Here is my best attempt... it still doesn't seem to work quite correctly, most points don't get plotted, and those that do flicker in and out as I move the axes around. I'm not sure what I'm doing wrong here I assume I'm misunderstanding the coordinate system somehow?

The original unfiltered dataset is deltas.data. To to detect if the coordinate system has changed I check ImPlot::PixelsToPlot giving the same values for the diagonal corners as I have previously stashed on the EventInfo object. The original dataset only has Y values, so X values are just the index of each Y value.

We iterate over each original point, and if it is in the bounds of the current range of the plot convert to pixel coordinates then check if the bit we associate with that pixel has already been set. If not we add it to the new hopefully smaller dataset.

    const std::pair<std::vector<uint64_t>&, std::vector<uint64_t>&> get_fast_transaction_deltas(
        EventInfo& info
        )
    {
        auto& deltas = get_transaction_deltas(info);
        auto pixel_top_left = ImPlot::GetPlotPos();
        auto pixel_lower_right = ImPlot::GetPlotPos();
        pixel_lower_right.x += ImPlot::GetPlotSize().x;
        pixel_lower_right.y += ImPlot::GetPlotSize().y;
        ImPlotPoint top_left = ImPlot::PixelsToPlot(pixel_top_left);
        ImPlotPoint lower_right = ImPlot::PixelsToPlot(pixel_lower_right);
        if(info.fast_deltas_x.size()
           && info.fast_top_left.x == top_left.x
           && info.fast_top_left.y == top_left.y
           && info.fast_lower_right.x == lower_right.x
           && info.fast_lower_right.y == lower_right.y
            )
        {
            return {info.fast_deltas_x, info.fast_deltas_y};
        }

        info.fast_deltas_x.clear();
        info.fast_deltas_y.clear();
        Bitset pixels(ImPlot::GetPlotSize().x * ImPlot::GetPlotSize().y);
        auto limits = ImPlot::GetPlotLimits();
        for(std::size_t i = 0; i < deltas.data.size(); ++i) {
            if(i > limits.X.Max) {
                break;
            }
            if(!limits.Contains(i, deltas.data[i])) {
                continue;
            }
            ImVec2 v = ImPlot::PlotToPixels(i, deltas.data[i]);
            auto x = v.x - ImPlot::GetPlotPos().x;
            auto y = v.y - ImPlot::GetPlotPos().y;

            // In practice without this we seem to go one pixel out of
            // bounds
            if(ImPlot::GetPlotSize().x - x <= 1) {
                x = ImPlot::GetPlotSize().x - 1;
            }
            if(ImPlot::GetPlotSize().y - y <= 1) {
                y = ImPlot::GetPlotSize().y - 1;
            }

            std::size_t bit_index = x + y*ImPlot::GetPlotSize().x;

            assert(bit_index < pixels.size());
            if(!pixels.test(bit_index)) {
                info.fast_deltas_x.emplace_back(i);
                info.fast_deltas_y.emplace_back(deltas.data[i]);
                pixels.set(bit_index);
            }
        }

        info.fast_top_left = top_left;
        info.fast_lower_right = lower_right;
        return {info.fast_deltas_x, info.fast_deltas_y};
    }

[jgarvin]

In the end I did get it to work, I had a bug outside this code making things more confusing. On Linux, with SDL backend using software rendering of opengl (llvm pipe) it's actually faster to run this every frame than actually trying to render 1.7 million points. A million points will render at <1fps, with this I'm getting about 25fps (distilled down to ~20k points). No idea if this would still be true with a real GPU.

    const std::pair<std::vector<uint64_t>&, std::vector<uint64_t>&> get_fast_transaction_deltas(
        EventInfo& info
        )
    {
        auto& deltas = get_transaction_deltas(info);
        auto pixel_top_left = ImPlot::GetPlotPos();
        auto pixel_lower_right = ImPlot::GetPlotPos();
        pixel_lower_right.x += ImPlot::GetPlotSize().x;
        pixel_lower_right.y += ImPlot::GetPlotSize().y;
        ImPlotPoint top_left = ImPlot::PixelsToPlot(pixel_top_left);
        ImPlotPoint lower_right = ImPlot::PixelsToPlot(pixel_lower_right);
        if(info.fast_deltas_x.size()
           && info.fast_top_left.x == top_left.x
           && info.fast_top_left.y == top_left.y
           && info.fast_lower_right.x == lower_right.x
           && info.fast_lower_right.y == lower_right.y
           )
        {
                return {info.fast_deltas_x, info.fast_deltas_y};
        }

        info.fast_deltas_x.clear();
        info.fast_deltas_y.clear();
        Bitset pixels(ImPlot::GetPlotSize().x * ImPlot::GetPlotSize().y);
        auto limits = ImPlot::GetPlotLimits();
        auto x_start = std::max<double>(0, limits.X.Min);
        auto x_stop = std::min<double>(limits.X.Max, deltas.data.size());
        for(std::size_t i = x_start; i < x_stop; ++i) {
            if(!limits.Contains(i, deltas.data[i])) {
                continue;
            }
            ImVec2 v = ImPlot::PlotToPixels(i, deltas.data[i]);
            auto x = std::floor(v.x - ImPlot::GetPlotPos().x);
            auto y = std::floor(v.y - ImPlot::GetPlotPos().y);

            // In practice we seem to go one pixel out of bounds
            // sometimes.
            if(x < 0
               || x >= ImPlot::GetPlotSize().x
               || y < 0
               || y >= ImPlot::GetPlotSize().y
               )
            {
                continue;
            }

            std::size_t bit_index = x + y*ImPlot::GetPlotSize().x;

            if(!pixels.test(bit_index)) {
                info.fast_deltas_x.emplace_back(i);
                info.fast_deltas_y.emplace_back(deltas.data[i]);
                pixels.set(bit_index);
            }
        }

        info.fast_top_left = top_left;
        info.fast_lower_right = lower_right;
        return {info.fast_deltas_x, info.fast_deltas_y};
    }

[epezent]

We can look into adding an API to detect state changes to plots/axes. A function that returns fllags with the current state changes (a la ImGuiTableColumnFlags) is probably the right way of doing it.