Closed joaoponte closed 4 years ago
I used the logging DEBUG and got these messages in Jupyter and terminal, respectively.
Running the 8_shading_with_textures.ipynb
[INFO] (MainThread) Configure RnD.SharpOptiX library... [INFO] (MainThread) ...done. [INFO] (MainThread) Preparing empty scene... [INFO] (MainThread) Empty scene ready. [INFO] (MainThread) Use start() to start raytracing. [INFO] (MainThread) Set min_accumulation_step to 4 [INFO] (MainThread) Set max_accumulation_frames to 512 [INFO] (MainThread) Set light_shading to Hard [INFO] (MainThread) Configured material plastic. [INFO] (MainThread) Configured material glass. [INFO] (MainThread) Configured material beads. [INFO] (MainThread) Create Parallelograms plane, 1 primitives... [INFO] (MainThread) ...done, handle: 1 [INFO] (MainThread) Create ParticleSetTextured sphere, 1 primitives... [INFO] (MainThread) ...done, handle: 2 [INFO] (MainThread) Create ParticleSetTextured particles, 300 primitives... [INFO] (MainThread) ...done, handle: 3 [INFO] (MainThread) Camera cam1 handle: 1. [INFO] (MainThread) Light light1 handle: -1153832004. [INFO] (MainThread) Light light2 handle: 566500016. [INFO] (MainThread) Background constant gray level updated. [INFO] (MainThread) Ambient color updated. [INFO] (MainThread) Add postprocessing stage: Gamma. [INFO] (MainThread) Camera cam1 handle: 1. [INFO] (MainThread) Light light1 handle: -1153832004. [INFO] (MainThread) Light light2 handle: 566500016. [INFO] (MainThread) Background constant gray level updated. [INFO] (MainThread) Ambient color updated. [INFO] (MainThread) Add postprocessing stage: Gamma.
And freezes.
Terminal running the 1_scatter_plot_3d.py
[INFO] (MainThread) Configure RnD.SharpOptiX library... [INFO] (MainThread) ...done. [INFO] (MainThread) Preparing empty scene... [INFO] (MainThread) Empty scene ready. [INFO] (MainThread) Use start() to start raytracing. [INFO] (MainThread) Set max_accumulation_frames to 30 [INFO] (MainThread) Background constant gray level updated. [INFO] (MainThread) Create ParticleSet particles, 22505 primitives... [INFO] (MainThread) ...done, handle: 1 [INFO] (MainThread) Create Parallelepipeds cubes, 22516 primitives... [INFO] (MainThread) ...done, handle: 2 [INFO] (MainThread) Coordinate system mode set to: Box. [INFO] (MainThread) Default scene initialization. [INFO] (MainThread) Camera default handle: 1. [INFO] (MainThread) Initialization done. [INFO] (MainThread) RT loop ready. [INFO] (Thread-1 ) Callbacks registered. [INFO] (Thread-1 ) Tkinter widgets ready. [INFO] (Thread-1 ) Couple scene to the output window... [INFO] (Thread-1 ) Start UI event loop... [INFO] (Thread-1 ) Resize to: 1802 x 542 [INFO] (MainThread) Raytracing started. [INFO] (MainThread) Camera default updated. [INFO] (MainThread) Light light1 handle: -1153832004. done [INFO] (Dummy-2 ) Compute, delta 1 frames. [INFO] (Dummy-3 ) RT completed, result 0. [INFO] (Dummy-4 ) Compute, delta 1 frames. [INFO] (Dummy-5 ) RT completed, result 0. [INFO] (Dummy-6 ) Compute, delta 1 frames. [INFO] (Dummy-7 ) RT completed, result 0. (...) [INFO] (Dummy-3 ) Compute, delta 1 frames. [INFO] (Dummy-4 ) RT completed, result 0. [INFO] (Dummy-9 ) Compute, delta 1 frames. [INFO] (Dummy-2 ) RT completed, result 0. [INFO] (Dummy-6 ) Compute, delta 1 frames. [INFO] (Dummy-8 ) RT accumulation finished. [INFO] (Dummy-11 ) RT completed, result 1. [INFO] (Dummy-9 ) Compute, delta 1 frames. [INFO] (Dummy-9 ) RT completed, result 2. [INFO] (Dummy-3 ) Compute, delta 1 frames. [INFO] (Dummy-4 ) RT completed, result 0. [INFO] (Dummy-6 ) Compute, delta 1 frames. [INFO] (Dummy-3 ) RT completed, result 0. [INFO] (Dummy-4 ) Compute, delta 1 frames. [INFO] (Dummy-8 ) RT completed, result 0. [INFO] (Dummy-2 ) Compute, delta 1 frames. (...) [INFO] (Dummy-13 ) Compute, delta 1 frames. [INFO] (Dummy-9 ) RT completed, result 0. [INFO] (Dummy-10 ) Compute, delta 1 frames. [INFO] (Dummy-4 ) RT accumulation finished. [INFO] (Dummy-5 ) RT completed, result 1. [INFO] (Dummy-9 ) Compute, delta 1 frames. [INFO] (Dummy-9 ) RT completed, result 2.
And freezes even without any interaction with the mouse. I removed a lot of dummy lines, but preserved the lines with two mouse interactions I did at that moment, but I tested without interaction and froze anyways.
Can you let me know your NVIDIA driver release?
Of course! The installed nvidia driver is the 430.26
On Tue, Sep 24, 2019, 12:09 PM Robert Sulej notifications@github.com wrote:
Can you let me know your NVIDIA driver release?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rnd-team-dev/plotoptix/issues/6?email_source=notifications&email_token=AA2XFJ4Z4ZUKWZWZAK2NBUDQLIUTRA5CNFSM4I2AGTZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7OWIPQ#issuecomment-534602814, or mute the thread https://github.com/notifications/unsubscribe-auth/AA2XFJY33S7OGSILQW44TSTQLIUTRANCNFSM4I2AGTZQ .
OK, great. There is a good chance that changing the driver will solve the problem. You can either try the newest (435 or 436) or downgrade to 419 (Windows) or 418 (Linux). I experienced issues with 430 and OptiX' internal threads synchronization, and since it appears only on some system configurations, I could not solve this with NVIDIA guys.
Hey @robertsulej, sorry about the waiting time!
I tested with the 435.21 driver and I faced the same error.. The funny thing is that when I tried to install the 418 driver, the "Graphics Drivers" PPA pointed to the 430.50 driver, lol
Do you have another idea to test or other source of logs to send to you? Is it maybe the CUDA 10.1?
Oh, that's bad that 435 didn't work. I'll test it in Linux on the problematic system. I was happy to see that 436 worked well in Windows but did not look at 435+Linux.
The link to 418 is in the archive: https://www.nvidia.com/Download/driverResults.aspx/149219/en-us
CUDA is not referenced until you create video encoder, so the CUDA release number does not matter here. The freeze which I know happens in the OptiX core function, which never returns. I am travelling now, when I'm back I'll see how to confirm that that this is that problem.
Thanks for the link! I didn't test the 418 yet because it is a different process to update and I must require a authorization to do that (my pc is shared with others).. But I tested the 415 via PPA and didn't work as well.. I will focus on the 418 and later I send you the results!
I understand, so it is indeed in the nvidia-driver package.. Maybe a buffer that is overflowing, because it works really well during some seconds.. That is what makes everything kind of strange..
Ok, don't worry about that. Enjoy your travel and after that we continue trying to solve this
Just a quick comment: 418 is minimum required by OptiX 6. Part of OptiX code was moved to driver libraries and it won't work <418. Thanks for tests!
I've just uploaded v0.5.1 to PyPI. It should report timeout on the terminal and not freeze the window when stucked in the internal library function.
I also tested 435 in Linux. It works well on my machine which was problematic for 430.
I am also working on moving to OptiX 7 where things arranged differently and there is much less of black-boxed parts on the OptiX side. Hopefully this will work for everyone ;)
Thanks for the update! Unfortunately, the plot continues freezing after few secs.. Also, I didn't see any information on the terminal after freeze! I got only a line with "done", which already was before the update.
Maybe it is not an OptiX problem, but anything else.. I will try a clean env and test it again
Tested with a clean conda env and got the same result
I mentioned that It should give me some information and not freeze the window. It is interesting, because every time I test, I need to send a kill sign to close the thread, because neither the terminal window nor the render responds anymore.
Maybe it is a clue..
Default timeout is quite long, 30s. You can set it lower with optix.set_param(rt_timeout=x_millisec)
called before optix.start()
. Message "done" gets printed from another thread so it can arrive before timeout message.
Does it freeze on every example script?
Oh, this is really useful to know! But I waited way more than 30s and the message didn't come :/
I tested so far all terminal examples and some of the jupyter notebooks and all freezes without returning any error message.
Is there any other test you want to run or information that you want to know?
Thanks for all the tests you've done so far!
Until now I did not see a system where all of them are freezing... From your DEBUG output it looks like the OptiX library problem though. It may be difficult to debug at a distance. I'll try to find a way to pin the problem, but it might be that it won't get solved before moving to OptiX 7.
No problem, I really want to help to solve this ;) I see a huge potential on your lib! And thank you for all your support!
Indeed, It's definitely hard to debug these kind of problems at a distance.. I'll wait for the OptiX 7 implementation and, in the mean time, I will try to test my system with the OptiX 6 and 7 to find what happened.
Same issue here after following install instructions on Ubuntu 18.04 with 435.21
The new release is out. Shader compilation changed and issues like this one should be gone. This was a significant rewrite, if new issues appear, please, let me know!
Thank you very much for the update!
I tested on my personal computer and all went well. However, in my problematic system, the update doesn't worked as well..
Same problem, the plot froze after some seconds and I needed to kill the process manually. I used a Ubuntu 18.04, NVIDIA drivers version 440.64 and CUDA 10.2.
In the terminal, the program outputs this: [0]: Titan X (Pascal) done
Tanks so much for testing!
Uh... that's not a best news... I'll investigate. Could you, please, try running with changed log level and post the output? The log level is a constructor parameter, like this:
rt = TkOptiX(log_level="DEBUG")
Of course!
This is what the python wrote in the terminal:
[0]: TITAN X (Pascal) [INFO] (MainThread) Configure RnD.SharpOptiX library... [INFO] (MainThread) ...done. [INFO] (MainThread) Preparing empty scene... [INFO] (MainThread) - ray-tracer initialization [ 4][ KNOBS]: All knobs on default.
[ 4][ DISK CACHE]: Opened database: "/var/tmp/OptixCache_joao/cache7.db" [ 4][ DISK CACHE]: Cache data size: "39.8 KiB" [INFO] (MainThread) Empty scene ready. [INFO] (MainThread) Use start() to start raytracing. [INFO] (MainThread) Set max_accumulation_frames to 30 [INFO] (MainThread) Background constant gray level updated. [INFO] (MainThread) Create ParticleSet particles, 22731 primitives... [INFO] (MainThread) ...done, handle: 1 [INFO] (MainThread) Create Parallelepipeds cubes, 22304 primitives... [INFO] (MainThread) ...done, handle: 2 [INFO] (MainThread) Coordinate system mode set to: Box. [INFO] (MainThread) Default scene initialization. [INFO] (MainThread) Camera default handle: 1. [INFO] (MainThread) Initialization done. [INFO] (MainThread) RT loop ready. [INFO] (Thread-1 ) Callbacks registered. [INFO] (Thread-1 ) Tkinter widgets ready. [INFO] (Thread-1 ) Couple scene to the output window... [INFO] (Thread-1 ) Start UI event loop... [INFO] (Thread-1 ) Resize to: 842 x 527 [INFO] (MainThread) Raytracing started. [ 4][COMPILE FEEDBACK]: Info: Pipeline parameter "params" size is 264 bytes Info: Module uses 4 payload values. Pipeline configuration: 4. Info: Module uses 6 attribute values. Pipeline configuration: 6. Info: Entry function "closesthitocclusion" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 2 basic block(s), 8 instruction(s) Info: Entry function "closesthitocclusion_transparency" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 7 basic block(s), 77 instruction(s) Info: Entry function "anyhitocclusion" with semantic type ANYHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 2 basic block(s), 9 instruction(s) Info: Entry function "closesthitradianceflat" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 40 basic block(s), 493 instruction(s) Info: Entry function "closesthitradiancecos" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 40 basic block(s), 517 instruction(s) Info: Entry function "closesthitradianceblurred_dot" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 44 basic block(s), 548 instruction(s) Info: Entry function "closesthitradiance__diffuse_masked" with semantic type CLOSESTHIT has 1 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 81 basic block(s), 1238 instruction(s) Info: Entry function "closesthitradiancediffuse" with semantic type CLOSESTHIT has 1 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 77 basic block(s), 1180 instruction(s) Info: Entry function "closesthitradianceglass" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 123 basic block(s), 1223 instruction(s) Info: Entry function "closesthitradiancereflective" with semantic type CLOSESTHIT has 2 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 129 basic block(s), 1996 instruction(s) Info: Entry function "closesthitradiance__light_soft" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 46 basic block(s), 547 instruction(s) Info: Entry function "closesthitradiance__light_hard" with semantic type CLOSESTHIT has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 44 basic block(s), 528 instruction(s) Info: 6 non-entry function(s) have 17 basic block(s), 232 instruction(s)
[ 4][COMPILE FEEDBACK]: Info: Pipeline parameter "params" size is 264 bytes Info: Module uses 0 payload values. Pipeline configuration: 4. Info: Module uses 6 attribute values. Pipeline configuration: 6. Info: Entry function "intersectionparticles" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 9 basic block(s), 106 instruction(s) Info: Entry function "intersectionparticles_entry" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 9 basic block(s), 118 instruction(s) Info: Entry function "intersectionparticles_tex" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 23 basic block(s), 380 instruction(s) Info: Entry function "intersectionparticles_displaced" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 31 basic block(s), 439 instruction(s) Info: 6 non-entry function(s) have 17 basic block(s), 232 instruction(s)
[ 4][COMPILE FEEDBACK]: Info: Pipeline parameter "params" size is 264 bytes Info: Module uses 0 payload values. Pipeline configuration: 4. Info: Module uses 6 attribute values. Pipeline configuration: 6. Info: Entry function "intersectionparallelograms" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 4 basic block(s), 122 instruction(s) Info: Entry function "intersectionparallelograms_front" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 5 basic block(s), 124 instruction(s) Info: Entry function "intersectionparallelepipeds" with semantic type INTERSECTION has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 29 basic block(s), 520 instruction(s) Info: 6 non-entry function(s) have 17 basic block(s), 232 instruction(s)
[ 4][COMPILE FEEDBACK]: Info: Pipeline parameter "params" size is 264 bytes Info: Module uses 4 payload values. Pipeline configuration: 4. Info: Module uses 0 attribute values. Pipeline configuration: 6. Info: Entry function "raygenpinhole_camera" with semantic type RAYGEN has 3 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 117 basic block(s), 1946 instruction(s) Info: Entry function "raygendof_camera" with semantic type RAYGEN has 3 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 130 basic block(s), 1961 instruction(s) Info: 6 non-entry function(s) have 17 basic block(s), 232 instruction(s)
[ 4][COMPILE FEEDBACK]: Info: Pipeline parameter "params" size is 264 bytes Info: Module uses 2 payload values. Pipeline configuration: 4. Info: Module uses 0 attribute values. Pipeline configuration: 6. Info: Entry function "missradiance" with semantic type MISS has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 4 basic block(s), 46 instruction(s) Info: Entry function "missradiance_ambient" with semantic type MISS has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 7 basic block(s), 62 instruction(s) Info: Entry function "missradiance_ambient_and_vol" with semantic type MISS has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 12 basic block(s), 158 instruction(s) Info: Entry function "missradiance_texturecart" with semantic type MISS has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 7 basic block(s), 88 instruction(s) Info: Entry function "missradiance_envtexture" with semantic type MISS has 0 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 12 basic block(s), 144 instruction(s) Info: 6 non-entry function(s) have 17 basic block(s), 232 instruction(s)
[ 4][COMPILE FEEDBACK]: Info: Pipeline has 5 module(s), 14 entry function(s), 5 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 410 basic block(s) in entry functions, 6385 instruction(s) in entry functions, 30 non-entry function(s), 85 basic block(s) in non-entry functions, 1160 instruction(s) in non-entry functions
[INFO] (MainThread) Camera default updated. [INFO] (MainThread) Light light1 handle: -1153832004. [INFO] (Dummy-2 ) Compute, delta 1 frames. [ 4][COMPILE FEEDBACK]: Info: Pipeline has 5 module(s), 18 entry function(s), 5 trace call(s), 0 continuation callable call(s), 0 direct callable call(s), 476 basic block(s) in entry functions, 7152 instruction(s) in entry functions, 30 non-entry function(s), 85 basic block(s) in non-entry functions, 1160 instruction(s) in non-entry functions
done [INFO] (Dummy-3 ) RT completed, result RtResult.Success. [INFO] (Dummy-4 ) Compute, delta 1 frames. [INFO] (Dummy-5 ) RT completed, result RtResult.Success. [INFO] (Dummy-6 ) Compute, delta 1 frames. (...) [INFO] (Dummy-10 ) RT completed, result RtResult.Success. [INFO] (Dummy-3 ) Compute, delta 1 frames. [INFO] (Dummy-5 ) RT completed, result RtResult.Success. Killed
Thanks! Compilation looks OK. Something else is locking the ray-tracing/refreshing loop. I'll search for the problem.
Just a small update ;) There is no urgency on this, since I use your software in my laptop. S it is all good.
I did the upgrade of all releases, including the new one - 0.8.1 - and the plot still freezes at the same point in my system with the Titan X (Pascal). With all updates in my system, the SO is the Ubuntu 18.04, driver version is 440.82 and CUDA version is 10.2.
With my (g)old 980M (NVIDIA driver version 446.14 and CUDA 10.2) in my laptop, everything works fine.
Hi Joao, Sorry for no progress! Still can't find a system where I could reproduce this freeze and chase the bug. Uh....
Hi Robert, There is actually no problem at all This is a strange bug and I'm just a beginner in C/C++ to even try to solve this, lol
Hi Robert!
Sorry for the delay to test the new releases. I lost the connection to my computer in my lab for all these months, but now I went back to normal!
I think that since 0.8.2 the render worked well, but I'm sure that 0.9 works without any problem! So I think that this issue is now solved ;)
Thank you very much!
Thats great to hear, thanks! There were many low-level changes and a switch to OptiX 7.1, hope we have this freezing sorted out.
I'm facing a problem with plotoptix in a Ubuntu 18.04 with CUDA 10.2, FFMPEG 4.2 and Mono 5.20.1 in my system and CUDA 10.1, FFMPEG 4.2 and pythonnet 2.4 in a conda environment.
All seems to work well running the code on a terminal, but after few seconds the plot freezes and I'm forced to kill the thread to close the window.
In Jupyterlab, for example, the plot window open but freezes almost immediately after opening.
Is there anything I'm doing wrong here? lol I did the Windows step on my personal laptop and all worked pretty amazingly ;)
Here is some information about the system and conda environment.
System:
If needed, I can provide more information, just sent me a command example. I'm just a beginner on the linux world.