Command Line Renderer

[bhouston]

Is your feature request related to a problem? Please describe.

It would be nice to have a standardized way of rendering an image via a command line tool. This would allow one to automate renders in a standardized fashion.

Describe the solution you'd like

> three-gpu-pathtracer-renderer --ibl=myBackground.hdr --asset=./models/helmet.glb \
--iterations=500 --output-width=1920 --output-height=1080 --output-image=./outputs/helmet.jpg

Probably want to have an underlying class with the same parameters. The command line utility is just exposes those class parameters on the command line.

Probably can use puppeteer in headless mode. It is relatively simple to do.

[FabianRueckert]

Hey. I tried loading a page with puppeteer, but I get errors that some shaders seem to fail to compile:

For example when loading https://gkjohnson.github.io/three-gpu-pathtracer/example/bundle/materialBall.html

LOG Warning: meshopt_decoder is using experimental SIMD support
WAR [.WebGL-0x2c900034b100]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels
WAR [.WebGL-0x2c900034b100]GL Driver Message (OpenGL, Performance, GL_CLOSE_PATH_NV, High): GPU stall due to ReadPixels
ERR THREE.WebGLProgram: Shader Error 0 - VALIDATE_STATUS false

Program Info Log: 

WAR WebGL: INVALID_OPERATION: useProgram: program not valid
ERR THREE.WebGLProgram: Shader Error 1282 - VALIDATE_STATUS false

Program Info Log: 

WAR WebGL: INVALID_OPERATION: useProgram: program not valid
ERR THREE.WebGLProgram: Shader Error 1282 - VALIDATE_STATUS false

Program Info Log: 

WAR WebGL: INVALID_OPERATION: useProgram: program not valid
WAR WebGL: INVALID_OPERATION: useProgram: program not valid
WAR WebGL: INVALID_OPERATION: useProgram: program not valid
WAR WebGL: INVALID_OPERATION: useProgram: program not valid
WAR WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost
LOG THREE.WebGLRenderer: Context Lost.`

Unfortunately it looks like gl.getShaderInfoLog() returns an empty string, so I don't know what could be causing it. Does anyone have an idea? @gkjohnson

Here is the test page I made where you can load various pages with a puppeteer backend and take screenshots and display the logs: https://thread-dear-novel.glitch.me/ Here is the code if you want to play around with it yourself: https://glitch.com/edit/#!/thread-dear-novel

You can see that puppeteer supports WebGL2 correctly when passing in https://webglreport.com/?v=2 for example. The ThreeJS examples also render correctly.

[gkjohnson]

Unfortunately it looks like gl.getShaderInfoLog() returns an empty string, so I don't know what could be causing it.

This has been one of the frustrating things about debugging some of these device and platform-specific issues. It seems that ANGLE chokes somewhere and for some reason doesn't report any error. My approach has basically been to do a binary search of the code to find the issue by iteratively commenting out pieces of the shader until something works and backing out which portion of the code is a problem. But admittedly that's a huge PITA.

That glitch is a pretty cool thing to see if we could get it working, though. @FabianRueckert if you have some interest in helping here I'd appreciate it. Here's what I would look in to:

• Does puppeteer work when you run locally? (mac, windows?)
• Can we confirm that puppeteer is using GPU-accelerated rendering rather than software? I used puppeteer years ago and recall running into an issue related to this when running it headless.
• Is there someone from Chromium / ANGLE that can help us understand why we're getting no error output when the shader fails?
• Worst case we do some manual shader binary searching but hopefully theres a better solution. Running locally would make this debugging easier, at least.

[bhouston]

Puppeteer disables GPU acceleration by default when you are in headless. Thus you have to enable GPU acceleration manually in that case. Basically there are a bunch of flags. Please check the output of chrome://gpu when in headless and compare to your Chrome/Chromium on desktop that works and ensure it is using hardware acceleration and not SwiftShader emulation.

[bhouston]

@gkjohnson asked "Is there someone from Chromium / ANGLE that can help us understand why we're getting no error output when the shader fails?"

Well, there is the famous Mr. Angle @null77, but I am not sure he has time for this question.

[bhouston]

Also we have to ensure any machine we run Puppeteer on, there is a real GPU available and hopefully at least a decent one. This can be quite hard to do inside of a Docker container on Kubernetes and other such platforms. I've seen people struggle for months to try to get this to work reliably across NVIDIA drivers versions, Kubernetes versions and NVIDIA-docker versions -- basically it is possible but it is fragile and error prone.

I would recommend just getting it to work locally on a machine with a real GPU. It is infinitely easier.

[FabianRueckert]

Thanks @bhouston. I got it to work locally by enabling GPU acceleration using puppeteer --use-gl=egl flag.

I've also digged into the shader sources and was able to compile the shaders even with software rendering. @gkjohnson

The changes I needed to make to make the shaders compile with puppeteer software rendering were:

• The "continue" here caused it to fail compilation. Negating the expression and putting everything below in the if-block fixed it. https://github.com/gkjohnson/three-gpu-pathtracer/blob/main/src/materials/PhysicalPathTracingMaterial.js#L201

• In three-mesh-bvh https://github.com/gkjohnson/three-mesh-bvh/blob/master/src/gpu/shaderFunctions.js#L169 It didn't like that line. Changing it to for ( uint i = offset; i < offset + count; i++ ) { worked.

• It didn't like while ( ptr > - 1 && ptr < 60 ) { Replacing with for (int ptr = 0; ptr > -1 && ptr < 60; ptr+=0) { worked.

• And it didn't like the continue here https://github.com/gkjohnson/three-mesh-bvh/blob/master/src/gpu/shaderFunctions.js#L232 .

With software rendering and modified shaders:

Same scene with hardware rendering (also with and without the shader modifications):

As you can see from the above screenshot it looks like there are some issues with the environment map being rendered as black. It's a bit tricky to debug because the shader is behaving somewhat undeterministic on puppeteer software rendering. Seem like some driver issues. But if we manage to fix that it could work without the need for a real GPU.

[gkjohnson]

Oh wow - thanks for digging into that! It definitely looks like some weird inexplicable compiler issues... Glad to hear that --use-gpu=egl works, though. Does that work on the glitch link, as well?

But if we manage to fix that it could work without the need for a real GPU.

I think changes like this are too brittle and unpredictable to keep chasing down and are definitely prone to breaking again. I had to track something similar down to fix ios and m1 renders, as well (#187), and it would be extremely easy to fix in one place and wind up breaking in another. For now I wondering if it's best to make an issue for SwiftShader (bug tracker) / puppeteer and point them to this issue.

[FabianRueckert]

The flag doesn't seem do anything for glitch.com. Probably their servers don't have a GPU.

Agree, it will be very hard to maintain such workarounds. I've created a ticket in the SwiftShader bug tracker here.

[c0d1f1ed]

https://gkjohnson.github.io/three-gpu-pathtracer/example/bundle/materialBall.html now renders correctly with Chrome Canary 106.0.5232.0 using SwiftShader. It should reach Stable in a matter of weeks. Thanks for reaching out to us about this issue!

[gkjohnson]

Amazing! Thanks @c0d1f1ed. This should help us with implementing CI, as well.