Huge memory consumption difference between 0.15.2 and 0.16 version

[AnthonyGermain]

Hi Tyler, I am super excited about the new version of rayshader and I was testing it starting yesterday when I encounter big memory issues. I tried to make exhaustive testing so it will be as helpful as possible and you can find the code and screenshot here : https://drive.google.com/open?id=1cNctPfXA385mNjMOVbZlsw_GTeC96hfe

So just to sum-up my experimentation, I was running R 3.6 and rayshader 0.15.2. I upgraded to 0.16 and then faced memory issues. I then thought it could be because I didn't run R 4.0 so I upgraded to R 4.0.

I still add the issue. I then downgraded back to rayshader 0.15.2 for R3.6. So then I could switch between the two versions just by selecting R3.6 or R 4.0 in rstudio. not the best as I have 2 versions of R but practical considering I switch back and forth to run multiples test.

Starting with the easy part, I used your example code :

par(mfrow = c(1, 1)) plot_gg(pp, width = 5, height = 4, scale = 300, raytrace = FALSE, windowsize = c(1200, 960), fov = 70, zoom = 0.4, theta = 330, phi = 20, max_error = 0.01, verbose = TRUE)

98.8% reduction: Number of triangles reduced from 3600000 to 41446. Error: 0.009996

Sys.sleep(0.2) render_highquality(samples = 400, aperture=30, light = FALSE, ambient = TRUE,focal_distance = 1700, obj_material = rayrender::dielectric(attenuation = c(1,1,0.3)/200), ground_material = rayrender::diffuse(checkercolor = "grey80",sigma=90,checkerperiod = 100))

It doesn't work for me in the latest version. It produces this weird result.

It puzzled me so I tested with the triangle reduction on 0.15.2 vs 0.16.

I made it work on 0.15.2 (definitely slower!)

but still the same issue on 0.16 ()

--- I'm wondering if this is related or not to the main issue which is memory.

So my main issue, you can find all the data in the link at the beginning if you want to test it yourself, my laptop has 16GB of ram and I used to be able to run full resolution and half resolution of Ohrid (inspired by your awesome tutorial https://t.co/os92oLm4I7?amp=1 )

with the new version, I cannot even run the full res, it crashed my laptop, I manage to have it crash nicely enough I could get some profiling info for you (requesting more than 32GB) I can run the half res version but it is almost using all my available ram which looks like almost 10 times more than the old version.

half res, performance 0.15.2 :

half res, performance 0.16.0 :

half res, performance 0.16.0 rayrender 0.10.1 :

full res, performance 0.15.2 :

full res, performance 0.16.0 (render didn't finish, crashed before), rayrender 0.10.1

PS: I updated rayrender this morning as you released it to see if it changed things, it looks like it is improving the speed but the memory issue seems to remain.

Hopefully, this will be helpful to you, I'd be happy to run more testing if necessary.

[tylermorganwall]

What is your setting for max_error when triangulate = TRUE for the large memory cases? If that value is too small you'll get no benefit from triangulation, as you're just going to draw the full model with the less-memory efficient rgl.triangles() (instead of rgl.surface()).

For the first case, it looks like the image isn't being saved. What happens when you call the function save_obj() with a filename? Does an image of the plot appear in your working directory, along with the obj file and a mtl file?

[AnthonyGermain]

I'm actually not using triangulate = TRUE but keeping the default false. I tried Triangulate = TRUE with the default max error and it literally blocked my laptop as it requires a crazy amount of ram (more than my 16Gb or even 32GB, I tried with a VM in the cloud)

So i did tried again with max.error = 2! and I managed to at least render the map, so progress :)

this version still uses a lot more memory than the previous one but looks like with this setup I stay under my total ram capacity. I noticed after closing rgl, if i run gc() i free a significant amount of memory, I'm wondering if forcing calling gc() during all your different steps couldn't improve the ram situation. Just a guess.

i called save_obj and I did generate 3 files, the obj, mlt and png.

testObj1.zip

still i tried again render_highquality and obtained the same black image.

[AnthonyGermain]

This may lead nowhere but triangulate seems to make things worst on a simple matrix %>% sphere_shade (at least in term of memory).

without triangulate :

with triangulate, amx error 2 :

[AnthonyGermain]

using max.error 2 and above produces incredible results, I would put the default value of max error maybe higher or people using large matrix should just be careful.

the dielectric effect is not solved unfortunately. I'll continue testing :)

[tylermorganwall]

If triangulate = FALSE, plot_3d() should use the same amount of memory it used in 0.15.2, since it defaults to the prior behavior. I do see the increase in memory use you're referring to on my system when triangulate = TRUE, but that's expected since the triangulating process itself is memory intensive, and passing individual triangles to rgl.triangle() uses more memory than passing a surface via rgl.surface(). Most of that memory use is temporary, however--the garbage collector should automatically release those objects once they aren't needed and the R session needs more to allocate memory. Generally, you never need to call gc() manually when programming in R.

For the default for max error, the best setting for max_error is entirely data dependent--no default will work for all datasets. I'd just recommend playing with max_error until you find a value that works for your data.

I downloaded your obj file and it loaded fine--try setting the cache_filename argument to save the obj file in your working directory and see if it works then.

[AnthonyGermain]

Yeah that's my whole questioning because I wasn't using triangulate at first so I was expecting the same behavior than the previous version. Could you tell me how you ran it with my obj file, I did try to set up the cache_filename argument and i could see the obj created, (around 8Mb) but still the rendering failed and gave my the same kind of dark image.

[tylermorganwall]

I tested it using rayrender. What do you get when you run the following code? Replace the path with your local directory. I get the following:

library(rayrender)

generate_ground() %>%
  add_object(obj_model("path/to/testObj1/testObject.obj", texture = TRUE,scale_obj = 1/400)) %>%
  render_scene(lookfrom = c(0,10,10))

[AnthonyGermain]

I don't get an output when I run this, I get this :

I am running the latest version, I tried reinstalling it and also with the older version 0.10.

[AnthonyGermain]

In case it helps, I played with your neon example in the past with rayrender, and testing it again right now this seems still to be fine.

the issue is quite puzzling.

[tylermorganwall]

Can you uninstall rayrender (remove.packages("rayrender")), confirm it is uninstalled by checking the directory (or directories) listed when you call the function .libPaths(), and then reinstall it from github and try again?

[AnthonyGermain]

I got the same result, I actually deleted the folder the other times I reinstalled it. This time I did the remove package and the only file that remained was the rayrender.dll.

My full session :

rayrendersession.txt

[tylermorganwall]

Everything looks good in that compilation log--not sure what's going on in your installation. Whenever I've seen model corrupted like you've shown, it's been due to a faulty install or incomplete uninstall (definitely remove the dll file if it's still around). Particularly on Windows, where I often have to resort to manual cleaning to fully remove the package.

Sorry I can't be of more help!

[tylermorganwall]

If you aren't able to figure this out, the next version I upload to the CRAN will have these features and I'm fairly positive the pre-compiled binary they supply won't have these issues.

[AnthonyGermain]

Hopefully it does, I actually tried again completely erasing the folder and i got the same result. I also tried by installing the binary version of cran (0.9.1) Maybe it was expected that it wouldn't work with this old version but I got the same result. Weirdly the high rendering works, it seems to be situated with this dielectric material.

[tylermorganwall]

What you're seeing isn't any type of material--it's the result of NaNs encountered during pathtracing. Have you ever set any custom options for compilation? You are compiling with SSE extensions enabled -mfpmath=sse -msse2 and with -mstackrealign, which aren't specified by rayrender and may be the source of your problems. I just have -mtune=generic on my Windows 10 system where those options would be.

[AnthonyGermain]

I didn't set any custom options for compilation, do you know how I could change those options ?

[tylermorganwall]

Check the Makeconf file (should be located in ${R_HOME}/etc/Makeconf, type $R_HOME in R studios Terminal to see where that is) and remove those arguments from the CXX11* rows and see if it compiles without it.

[AnthonyGermain]

So i tried removing the options, then resinstalled rayrender. It Failed.

session2.txt

I then tried adding your -mtune=generic option. It failed too.

session3.txt

my i386 Makeconf file :

Makeconf.txt

My x64 MakeConf File :

Makeconf.txt

Weirdly I wanted to see if with ubuntu it would go better, I was testing using ggcloud to speed up some high_quality rendering but it wasn't as fast as expected. I have the same bug under ubuntu, which is weird.

session 4 linux.txt

[tylermorganwall]

@AnthonyGermain Are you still having these issues with the latest version of rayshader?

[AnthonyGermain]

Hey Tyler, we can close it, I tested the latest version and the example is running fine, I'm able to get the dielectric example running.