Handling Multiple Plots and Different draw calls per one chart

[9prady9]

We can discuss the different possible directions in which we want to implement the current issue. Please post the URLS to other plotting libraries for reference.

[9prady9]

@bkloppenborg @pavanky @shehzan10 @umar456 @syurkevi I have come up with following changes to Plot class/object to handle 2d and 3d plots (line/scatter) with unified interface that will handle multiple plots situation as well. But, all the plots should have same range for their data. Anything outside given range will be clipped from the rendered window.

class Plot {
    // nPlots defaults to single plot per Plot object
    Plot(fg::dtype type, const unsigned nPlots=1);

    // default parameter for index indicates it is the very first plot
    void setProperties(const unsigned index,
                       const unsigned nPoints,
                       const fg::Color c,
                       const fg::PlotType ptype=fg::LINE,
                       const fg::MarkerType mtype=fg::NONE);
}

example code using this object would look like below

Window wnd(...);
Plot p(f32, 3);
p.setProperties(0, 30, FG_RED, FG_LINE, FG_CROSS);
p.setProperties(1, 24, FG_GREEN, FG_SCATTER, FG_PLUS);
p.setProperties(2, 56, FG_BLUE);

/* update the vbo pointed by the Plot object with corresponding data
   the user of forge has to make sure they are updating the vbo with
  right size of data. For example, in the above example the data
  copied should be of total size 110(30 + 24+56).
*/
while(!wnd.close())
    wnd.draw(p);

Let me know what you guys think about it.

[9prady9]

Based on @bkloppenborg 's feedback(offline), i drew the next draft on how forge API should look like after the new feature additions. It is given below.

We discussed further on what other customization one might expect from plotting library. Based on our today's discussion will try to come up with next draft that includes the improvements (given below) that we discussed:

• Using images in charts
• Potentially use image as colors for a surface plot.
• Ability to render any number of objects in a given chart (can be only 2d or 3d)
• Rendering order should follow order of setup[plot|histogram|scatter] calls.

Thank you @bkloppenborg for the invaluable feedback.

[9prady9]

Based on yesterday's discussion, i made the following changes to draft to achieve the below goals:

• Ability to render any number of objects in a given chart (can be only 2d or 3d)
• Rendering order should follow order of setup[plot|histogram|scatter] calls.
• Using images in charts
• Enable usage of image as colors for a surface plot.
• Enable plots to have per point color and transparency variation

Based on what we need to do, i modified the Forge API to the below. Window can display two types of objects

• Images - Simple image display with colormaps that preserves aspect ratio when requested - it will be same as what it is now , hence no changes.
• Charts - Can be of type 2d (displays two dimension charts) or 3d (displays three dimensional charts). All charts will have grid layout that aligns with the ticks (we don't have this now).

class Chart {
    public:
        Chart(const ChartType type); // Can take either FG_2D | FG_3D
        void setAxesTitles(string x, string y, string z=string(""));
        void setAxesRanges(float xmin, float xmax, float ymin, float ymax, float zmin=-1, float zmax=1);

        // All below three functions return a unique identifier
        uint image(const uint W, const uint H, const dtype dataType, const ChannelFormat frmt, const float alpha=0.5);
        uint plot(const uint nPoints, const dtype dataType, const string legend=string(""), const Color c=fg::RED, const PlotType pt=fg::LINE, const MarkerType mt=fg::NONE, const float alpha=0.5);
        uint histogram(const uint nbins, const dtype dataType, const string legend=string(""), const Color c=fg::BLUE, const float alpha=0.5);

        // Getters for plotting objects resource ids(VBOs, PBOs)
        uint getImageResourceId(const uint uniqueId); // -> Returns PBO identifier

        // Below two functions set the output variables to VBO identifiers
        // Unless the user updates the colors, alphas VBO,
        // they are defaulted to to constant value(Color c) passed in during plot/Histogram creation
        void getPlotResourceIds(uint* vertices, uint* colors, uint* alphas, const uint uniquePlotId);
        void getHistogramResourceIds(uint* bins, uint* colors, uint* alphas, const uint uniquePlotId);
};

For 3d charts, i guess image function and associated functions will transform into a noop.

An example of how to use this Chart object to render multiple plots in single chart would be as follows

Window wnd("ArrayFire Demo", 1280, 720);

Image i(640, 480, f32, FG_GRAYSCALE);
Chart c(FG_2D);
c.setAxesTitles("x label", "y label");
c.setAxesRanges(-1, 1, -1, 1);
int background = c.image(512, 512, f32, FG_RGB);
int hist       = c.histogram(256, u32, "Image Histogram", FG_GREEN);
int plot       = c.plot(256, uint,  "Histogram Peaks", FG_BLUE, FG_LINE, FG_CROSS);
// rendering order: image(background) -> hist -> plot(front)

// get resource id's to have them updated in render loop
uint imageId = c.getImageResourceId(background);
uint hvid, hcid, haid;
c.getHistogramResourceIds(&hvid, &hcid, &haid, hist);
uint pvid, pcid, paid;
c.getPlotResourceIds(&pvid, &pcid, &paid, plot);

while(!wnd.close()) {
    wnd.grid(2,1);
    wnd(0, 0).draw(i, "Input Image");
    wnd(1, 0).draw(c, "Image Analysis");
    wnd.show();
    // update data using below functions,
    // which handle update VBOs or PBOs as required.
    updateImage(imageId);
    updateHistogramData(hvid, hcid, haid);
    updatePlotData(pvid, pcid, paid);
}

I believe changes related to Surface plot are independent of these, hence didn't mix them up with Chart here.

[pavanky]

This is getting overly complicated. For starters I see no use case for histogram + image + plots together. The only use case I can think of is to plot points on an image and we should probably have a special function for that.

[9prady9]

No, it is not actually and there are use cases which @bkloppenborg showed yesterday. @bkloppenborg Can you please post the URLs here so that pavan can take a look at them.

[9prady9]

Also, just to be clear. Whichever changes (approved ones) discussed in this issue will be implemented in Forge after 3.2 release of ArrayFire in all likelihood. So, this is not for ArrayFire 3.2.

[pavanky]

This API is differentiating between an Image drawn in a chart and an image drawn directly to a window. We should not have two ways to do the same thing.

[bkloppenborg]

Indeed, there are a tremendous number of use cases where users plot multiple things on the same chart. My input here was that we should simplify the API such that the user may put as many items on a chart as they please and that the items be drawn in the order in which they are received. All major plotting libraries (gnuPlot, matplotlib, IDL's plotting functions, QCustomPlot, etc.) implement this functionality.

From a user's perspective you do something like this:

chart A = chart()
A.scatter(x, y, name="chart 1", colors=color ...)
A.line(x, y, ...)
A.x_axis.label = "Time"
A.y_axis.label = "FLOPS"
A.title = "Some Awesome Graph"
A.show()

There are also many situations in which people plot images, contour plots, lines, and scatter plots on the same figure (e.g. a satellite image with topographic indication and a hiking plot with text labels) overlain. Clearly, we don't need to implement all of this functionality, just make the API sufficiently general such that we do not preclude this type of functionality.

P.S. I'm with @pavanky , images should be charts, but perhaps without axes.

[9prady9]

One example where two different usages of image makes sense is #11 . In the feature requested in #11, user doesn't need axes while rendering the images in question with lines connecting the features across images.

As far as the API goes, it is not huge addition in terms of LOC, the example code only increased by 6 to 8 lines to handle multiple plots situation.

Regarding why there are two images (one is af::Chart::image() & another is fg::Image object): On the implementation side of things, image will be one single object with axes(and other chart related features) that can be turned on and off. Image object in Forge API is merely a convenience to display images directly instead of going through chart where the user doesn't need to handle auxiliary calls related to axes, ranges etc.