Crash on F12: access violation

[philstopford]

Hello,

I have a scene that was previously rendering OK with V-ray / Blender (and renders fine with Cycles). Today, it seems to be having trouble :

V-Ray For Blender: V-Ray For Blender: Thread [0] stopping.Thread [1] stopping.

V-Ray For Blender: Total sync time 35.362 sec. Error : EXCEPTION_ACCESS_VIOLATION Address : 0x00007FF7E9AA8139 Module : C:\Program Files\Chaos Group\Blender 2.79 (With V-Ray Additions)\blender.exe

I've zipped up the .vrscene files that were generated, in case they are useful. I'm not sure how to debug this further, or to provide more useful information.

Let me know if the vrscenes are helpful and I'll email you a link to the package.

[bdancer]

I'll need a blend-file not vrscenes.

[philstopford]

Working on it - trying to find the minimum case.... Bear with me.

[philstopford]

This seems to be intermittent, oddly. Trying again often causes the render to go through without any change. Not sure whether this makes it more or less interesting to you.

[philstopford]

Are there some additional debug flags I could try? This seems to be happening randomly across a number of scenes and I'm trying to understand what might be contributing.

[bdancer]

Blender has "-d" flag, not sure how useful it'll be.

[philstopford]

It looks like blender is dying in the fcurve evaluation. I wonder if there's some kind of rate limit or race involved. I don't see this failure rendering with Cycles, so I don't think I can open a blender bug for this.

I've attached two different logs from two attempts with the same scene in the same V-ray/blender build on the same machine. They fail at different points, but always in the fcurve eval sequence.

log.txt log2.txt

[philstopford]

Just wondered if there was more I could do to help with this particular issue.

[philstopford]

I sent a link to a Google drive folder for, admittedly large, content that triggers this. I hope this might be useful.

[philstopford]

Just following up on this again. If there's a preferred location other than Google Drive to submit content for cases like this, please let me know.

[bdancer]

@philstopford, we are looking into this issue, I can't say much right now.

[philstopford]

More power to your elbow - I hope you are able to resolve it. If you have the content, can I remove it from the Google Drive to save space?

Please do let me know if I can help in any way. I'm grateful for your support and hard work.

[bdancer]

@philstopford sure, I've downloaded the files.

[bdancer]

@philstopford should work fine in tomorrow nightly.

[philstopford]

Seems to be working, building from the current code. However, during render :

ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen! ERROR: Python context internal state bug. this should not happen!

Uncertain where that is coming from and how serious it might be.

[bdancer]

@philstopford those messages could appear, shouldn't do any harm just restart Blender.

[philstopford]

I can provoke a crash : 1) Set up scene with distributed rendering, with two network nodes (I disabled rendering on the host - I only have two render licenses) 2) F12 a few frames to check all works. 3) Deselect one of the network nodes. 4) F12

Error : EXCEPTION_ACCESS_VIOLATION Address : 0x00007FF705003113 Module : C:\Program Files\Chaos Group\Blender 2.79 (With V-Ray Additions)\blender.exe

The -d log shows this at the end:

eval fcurve 'pose.bones["hand.r"].constraints["rotFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["arm.l"].constraints["scaleIK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["arm.l"].constraints["scaleFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["arm.r"].constraints["rotFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["arm.r"].constraints["scaleFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["arm.l"].constraints["rotFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["c_eyelid_base.l"].constraints["Copy Rotation"].influence' - 0.100000 => 1/2, 0 eval fcurve 'pose.bones["c_eyelid_base.r"].constraints["Copy Rotation"].influence' - 0.100000 => 1/2, 0 eval fcurve 'pose.bones["leg_i

[bdancer]

@philstopford weird...

[bdancer]

@philstopford there is a "lock" button at the right of the "Render" button - could you turn it on and check if this will help?

[philstopford]

That does seem to help. I've been playing with this on and off this morning. With the lock set, I got no crash. With the lock off, it crashed on F12. No idea what's the root cause, though.

[philstopford]

OK. I'm still getting a crash here if I render (F12) a few times (e.g. F12, wait, stop, F12, wait, stop, F12, wait, stop).

The crashlog still shows something like :

eval fcurve 'pose.bones["arm.l"].constraints["rotFK"].influence' - 1.000000 => 1/3, 1 eval fcurve 'pose.bones["c_eyelid_base.l"].constraints["Copy Rotation"].influence' - 0.100000 => 1/2

This is all local rendering - no DR.

[bdancer]

I've changed export a bit, hope this'll help...

[philstopford]

It seems to be better, although I'm seeing Deadline requeuing tasks from exceptions thrown by blender.exe in conjunction with V-ray rendering. An example of the log output is below.

2018-10-31 07:38:07: 0: STDOUT: [2018/Oct/31|07:38:03] Unknown property "metalness" in object "BRDFVRayMtl@NTpouldrons|L_characters_xurge_srgr_Ranger_Armor_M3_nohelmet_noammo_simon_blend|L___characters_xurge_characters_blend|Lpedestrians_blend|NStandard_Material" 2018-10-31 07:38:07: 0: STDOUT: [2018/Oct/31|07:38:03] The adaptive option for the light cache is no longer supported and will be ignored. 2018-10-31 07:38:07: 0: STDOUT: V-Ray For Blender: vfb_free() 2018-10-31 07:38:09: 0: STDOUT: Error : EXCEPTION_ACCESS_VIOLATION 2018-10-31 07:38:09: 0: STDOUT: Address : 0x00007FF603053EF3 2018-10-31 07:38:09: 0: STDOUT: Module : C:\Program Files\Chaos Group\Blender 2.79 (With V-Ray Additions)\blender.exe 2018-10-31 07:38:09: 0: INFO: Process exit code: 11 2018-10-31 07:38:09: 0: Done executing plugin command of type 'Render Task'

======================================================= Details

Date: 10/31/2018 07:39:02 Frames: 311-315 Elapsed Time: 00:05:35:59 Job Submit Date: 10/27/2018 15:39:29 Job User: phils Average RAM Usage: 22644121600 (88%) Peak RAM Usage: 25465253888 (99%) Average CPU Usage: 80% Peak CPU Usage: 100% Used CPU Clocks (x10^6 cycles): 669023424 Total CPU Clocks (x10^6 cycles): 836279260

[bdancer]

So what should I do to recreate this issue? Setup Deadine with Blender? On that scene? What if you'll submit just the exported vrscene?

[philstopford]

The large scene I sent previously should reproduce this from what I can tell. All I did was set Deadline up to render (via blender) with two nodes and the failures show up in the log. It's not every task (of 5 frames) that causes a problem, and it generally seems to be happening at the end of the task, which causes Deadline to fail the entire task and re-queue it.

[bdancer]

@philstopford so it's happening only via Deadline? What command line options Deadline supplies? What if you run Blender with the same options locally? Will it crash?

[philstopford]

This is the full command:

"C:\Program Files\Chaos Group\Blender 2.79 (With V-Ray Additions)\blender.exe" -b "D:\blender\SStation\nofusion_interiorWIP_r3.blend" -t 0 -s 375 -e 379 -a

I'll try it here in a commandline, but it will take a few hours to see what happens

[philstopford]

Running that through the command line, I see the exact same fail, so it's unrelated to Deadline. It always seems to be associated with the vfb_free() in this case, after all of the rendering is done:

[2018/Oct/31|17:52:56] The adaptive option for the light cache is no longer supported and will be ignored. V-Ray For Blender: vfb_free() Error: Not freed memory blocks: 138920, total unfreed memory 6.597916 MB Error : EXCEPTION_ACCESS_VIOLATION Address : 0x00007FF61D965783 Module : C:\Program Files\Chaos Group\Blender 2.79 (With V-Ray Additions)\blender.exe

[bdancer]

@philstopford well, weird, but I can't make it crash...

[philstopford]

Any additional switches I can set to give more insight?

[bdancer]

@philstopford no idea sry, I need a way to make it crash here, otherwise I can't do much...

[philstopford]

Uploaded some smaller test content that is failing the same way for me at the end of each batch of frames in a task (not just under Deadline Slave).. It's in the Google Drive and is showing as shared with you - let me know if that's not the case.

[bdancer]

@philstopford tnx! Checking...

[bdancer]

@philstopford I've tried running it lots of time... no crash... very weird...

[philstopford]

Ahah. I've found the cause for the end of render task crash. Sending it to the right place. This can be closed.

[philstopford]

Actually, no, there's an interaction here specifically for V-ray/blender and the FLIP fluids addon.

If you have access to the FLIP fluids package, render with that addon. You'll get a crash. If you render with the non-Vray blender, there is no crash.

[philstopford]

Resolved with FLIP fluids change to their init.py file.