WebGL significantly slower than RGL viewer

jfmoyen commented 9 months ago

Greetings,

Sorry for what is probably a naive question. I'm using ggrgl to generate a 3D interactive plot. So something like

p <- ggplot(data) + geom_sphere_3d(aes(x,y,z))

Viewed in the "built in" RGL viewer, with open3d();p (*) the viewer works very well and the graph can be rotated etc. smoothly.

Exporting to an htmlwidget, either in RStudio or external browser, via rglwidget()

gives a much slower interaction for the same graph. The difference is not noticeable under ca. 100 points (spheres), 500 points make the html version feel sluggish, 1000 points make the html practically unusable. Profiling in browser reveals 500 ms execution time for each mouse drag event, specifically here:

(drawScene > drawBackground > getParameter)

I guess, for time being I can live with a smaller number of points (I use html only for exporting) but it would be nice to be able to play with the whole dataset and be able to share it with colleagues who are not R users...

Thanks !

(*) in fact, this seems to work only if I have previousy called devoutrgl::rgldev() , which must be doing something else... Not sure whether this is related or not...

R 4.1.3, windows 10 64b with a cr*p load of memory but an ageing processor, rgl 1.2.8

dmurdoch commented 9 months ago

ggrgl isn't my package. Try plot3d(rnorm(1000), rnorm(1000), rnorm(1000), type="s", col="red") which is an rgl function. On my laptop that's not really slower in the browser than in R. If I set alpha = 0.5, R slows to a crawl, while the browser is slower, but still quite usable.

jfmoyen commented 9 months ago

ggrgl isn't my package.

Yes, I fully realize that. However, as far as I can tell, the issue appears when using rgl::rglwidget . At first glance it is hard to find out whether the issue lies with the way rglwidget converts an rgl object into webGL, or with the rgl object supplied by ggrgl.

Try plot3d(rnorm(1000), rnorm(1000), rnorm(1000), type="s", col="red")

Thanks. This does actually make the point clearer.

This graph works flawlessly in the RGL window (the one created by open3d). Interaction is smooth and immediate (if there is a way to time the redraw event of an RGL window, I'm happy to give you numbers). In RStudio or in my browser, this is slow and nearly unusable. Each pointermove event takes ca. 300 ms, so we are down to 3 fps or so, which is horrible.

So, I can think of several explanations:

webGL cannot be used for more than ca. 1000 objects (I don't think so, 1000 is not such a high number these days);
for some reason, the version of webGL on my computer is slow/elderly and should be updated (???);
webGL is intrinsically slower than RGL and that's how it is;
the RGL -> webGL conversion results in something innefficient and results in a webGL that is not optimized;
.... ?

Thanks for your time !

dmurdoch commented 9 months ago

I was using a different version than 1.2.8. I do see rglwidget being noticeably slower in 1.2.8.

Regarding the explanations:

webGL is fine with 1000 objects, but the way it sees objects is not the way you think. Each sphere is a few hundred triangles, so 1000 spheres is a few hundred thousand objects. Switching to points will make the display a lot faster.
I think it's hard to change WebGL, though you might see a difference if you update your graphics drivers. It's also possible that your graphics hardware is slow, but see the next point...
No, webGL is often faster than the R version, just not in this case for some reason. It uses modern methods, whereas the driver in R uses old OpenGL 1.2.
Yes, in this case something is going wrong in the conversion. You could install a slightly older rgl and things will probably be faster.

jfmoyen commented 9 months ago

Ok, Thanks. Well, the last point (conversion issue) seems to be in your department :-)

Meanwhile I'll try to move back to some older RGL. Thanks !

dmurdoch commented 9 months ago

Actually, my test results appear to depend on the browser. If I run the WebGL code in Firefox, things are fine. In RStudio and Chrome, they are slow. (RStudio's interface is Chromium based, so they may be using the same rendering code.) Safari is really slow. So all I can suggest is that you should use Firefox, not some other browser.

jfmoyen commented 9 months ago

So all I can suggest is that you should use Firefox, not some other browser.

Hardly so here, I still have lags of some 500 ms on each redraw event (during drawBackground specifically, as before). Ah, well...

dmurdoch commented 9 months ago

My previous message about timings was probably incorrect, in that I was doing some tests in RStudio, and others in Firefox. As far as I can see, Firefox is fine in several recent rgl versions, and RStudio and Chrome are equally slow in all of those. You say Firefox is slow for you, but you're on Windows and I'm on a Mac, so it's not too surprising you're seeing different results.

Looking more closely at your first message, I find the profile strange. getParameter isn't an rgl function, it's a WebGL function. It's not called directly by drawBackground at all. It is only called when rendering a texture (e.g. a background PNG image?), but my sample code wouldn't draw one of those.

jfmoyen commented 9 months ago

Looking more closely at your first message, I find the profile strange. getParameter isn't an rgl function, it's a WebGL function. It's not called directly by drawBackground at all. It is only called when rendering a texture (e.g. a background PNG image?), but my sample code wouldn't draw one of those.

I think you can mostly ignore my first message, as it was done with ggrgl (that, amongst other things, converts text to png textures). Here is a more useful log using you example (in Vivaldi, which is chrome-based):

it still evidences the same behaviour (ca. 300 ms, even 600 ms in this specific case, on each pointermove event), and here too, the getParameter seems to be the main offender ... whatever the reason for it to be there !

The code is

open3d()
plot3d(rnorm(1000), rnorm(1000), rnorm(1000), type="s", col="red")
rglwidget()

then from Rstudio, save as html

The resulting html is attached.

bench.zip

If I try instead htmlwidgets::saveWidget(rglwidget(), "D:/bench2.html")

I end up with the file also attached, and the same laggy result in the same place:

bench2.zip

Don't know if that helps...

dmurdoch commented 9 months ago

If you run this code instead, do you get the same issue?

open3d()
spheres3d(rnorm(1000), rnorm(1000), rnorm(1000), col="red", radius = 0.3)
rglwidget()

I'm seeing some signs that the problems are coming in drawing the text for the axes and labels; this doesn't draw any.

jfmoyen commented 9 months ago

No difference, sadly, and still the same getParameter call taking up most of the time...

tomicapretto commented 9 months ago

Hi!

I have a similar issue than @jfmoyen. But in this case I'm not sure if it's myself just pretending too much. See the following example:

library(rgl)

length_out <- 101
x <- seq(-3.5, 3.5, length.out = length_out)
y <- seq(-3.5, 3.5, length.out = length_out)
z <- mvtnorm::dmvnorm(expand.grid(x = x, y = y), c(0, 0), diag(2))

mfrow3d(1, 2, sharedMouse = TRUE)
bg3d(col="grey90")

# Surface
persp3d(
  x = x,
  y = y,
  z = z,
  color = "grey50",
  xlab = "x",
  ylab = "y",
  zlab = "",
  alpha = 0.4,
  lit = FALSE,
  polygon_offset = 1,
  box = FALSE,
  axes = FALSE
)

next3d()
bg3d(col="grey90")
persp3d(
  x = x,
  y = y,
  z = z,
  color = "grey50",
  xlab = "x",
  ylab = "y",
  zlab = "",
  alpha = 0.4,
  lit = FALSE,
  polygon_offset = 1,
  box = FALSE,
  axes = FALSE
)

rglwidget()

In other words, when I use length_out = 101 the experience is not smooth at all. If I use something like length_out = 30 it's way smoother. The reason why I'm sharing this here is that I noticed this only when plotting more than two plots. If I have a single plot, I can use length_out = 101 and the experience is still perfectly smooth.

library(rgl)

length_out <- 101
x <- seq(-3.5, 3.5, length.out = length_out)
y <- seq(-3.5, 3.5, length.out = length_out)
z <- mvtnorm::dmvnorm(expand.grid(x = x, y = y), c(0, 0), diag(2))
bg3d(col="grey90")

# Surface
persp3d(
  x = x,
  y = y,
  z = z,
  color = "grey50",
  xlab = "",
  ylab = "",
  zlab = "",
  alpha = 0.4,
  lit = FALSE,
  polygon_offset = 1,
  box = FALSE,
  axes = FALSE
)

Just in case it matters, I'm using rgl version 1.2.8 on Ubuntu 22.04.3 LTS. Thanks a lot in advance!

dmurdoch commented 9 months ago

Generally speaking transparency makes things slow. The reason is that rgl needs to sort the triangles that make up the surface before drawing every frame. Having two separate transparent objects in the same scene will make it even slower, because it has to do more context switches between drawing parts of one surface, then parts of the other, etc. There are certainly optimizations that are possible (e.g. things can be out of order if they don't overlap), but rgl doesn't do those.

tomicapretto commented 9 months ago

@dmurdoch thanks for the insight!

tomicapretto commented 9 months ago

Just wanted to add that this problem only happens in the browser. When I use the RGL device it's very smooth and I can't notice any lag.

tomicapretto commented 6 months ago

Hi @dmurdoch I'm coming back to this issue because I'm about to publish the work I've been doing based on {rgl} and I wanted to make sure the result is as smooth as possible.

I still want to point this issue about the performance difference between WebGL and RGL. See the attached video. The code is the one I shared on https://github.com/dmurdoch/rgl/issues/400#issuecomment-1873367627.

I understand why the transparency can be an issue, but, why does it work so well with the RGL device and so poorly with the other one?

I would like to help here as much as I can, I'm happy to dig into the codebase, provide more detailed reports, or anything that can be useful.

Screencast from 05-04-24 22:40:54.webm

Thanks very much!

dmurdoch commented 6 months ago

Yours would be easy to optimize following my Jan 1 comment: the two subscenes don't overlap at all, so the sorting that rgl does is not necessary. The problem is for rgl to recognize that.

I've put together a patch that makes the calculations quite similar to the rgl device, and it's just as fast. However, the quality in cases of overlap will be worse. Try installing the "Fasttransparency" branch to see it in action.

If you want to play around with the code, the modifications in that branch are the things to fiddle with. If you come up with something that performs reasonably well when the transparent objects overlap, I'll think about merging your code into the main branch.

tomicapretto commented 6 months ago

Thanks!!

dmurdoch commented 6 months ago

I just updated the patch. You probably want the new one, not the original.

dmurdoch commented 6 months ago

I've decided to bring this into the main branch, and set it as the default behaviour. There will be cases where it does a worse job than before, but it will often be a lot faster.

dmurdoch / rgl

WebGL significantly slower than RGL viewer #400