Blender 2.79 frozen when export too hard models

[ghost]

When export to glTF these models, such this... https://www.cgtrader.com/3d-models/sports/game/chess-set-1549cdc3-f6f0-4625-b1e1-54a541f2d992

Blender freeze. Here are 2 million polygons.

[UX3D-nopper]

We know this issue and the goal is to integrate critical runtime features of the exporter into Blender. Also, can you pelase do the following: Export again and just wait a looooooooooooooooooooooooooooong time. Before this, please open the Blender window console. Maybe you can provide me the output, as I do not have access to the 3D model.

[ghost]

I waited 15 minutes, and exports all happened without stopping.

[UX3D-nopper]

Thats the reason, why I would like to have the console output. Normally, debug and profiling is enabled. Also, with this trace, I can see where the exporter did get stuck.

Also, if you could provide me excatly this model does help.

[UX3D-nopper]

Also, maybe Blender did run out of memory and just crash internally.

[ghost]

Although my rendering is not very fast, 200,000 polygons can hardly endure (the same sponza), not to mention the millions where only 3 fps from strength. This is real-time path tracing. (comment moved, I wrote in wrong issue)

[TheOnlyJoey]

Can confirm, exporting 1mil polygon model, waiting for 30 min, still not doing anything. Will see if i can get a log together.

[ncoder]

I'm having trouble with my model, > 200k verts. It does work, it's just very slow.

Do you have any pointers about how to collect profiling data? I'm new to blender, but otherwise a competent programmer. I could give you a few stacktraces, I just don't know where to begin to debug a blender plugin.

[McNopper]

Great, that you would like to help on this. Before export, please open the system console of Blender. This can be done by menu entry "Window->Toggle System Console". During export, there are profiling messages, which indicates, how much time a specific section needs.

For developing, I am using Eclipse plus the PyDev plug-in: https://wiki.blender.org/index.php/Dev:Doc/Tools/Debugging/Python_Eclipse

We are aware of some bottlenecks and we hope to get some of the Python code into Blender and executed as a C++ method.

But please go ahead, maybe you find something which can already be improved.

[McNopper]

Can you guys please try out this fix:

https://github.com/KhronosGroup/glTF-Blender-Exporter/commit/810b522e7f26810f86d39778e445eba336b524a9

A typo could cause a crash for large models.

[ncoder]

For my use cases, most of the slow code is the "extract_primitives" function.

It looks like there is code to try to de-dupe vertices, based on your choice of exported data. That sort of micro-processing will be hard to optimise in general. It may not be a big deal in the end, as you seem to limit the search on a per-triangle basis. (forgive me if my interpretation is wrong). Consider having the ability to switch that off.

I'm not a python expert, but I don't expect a hash lookup like this: attributes['NORMAL'] = xxx to be fast in any case. A simple optimisation would be to alias those to local variables.

If you are able to get rid of the processing, and to more of a copy operation, then you can also skip the incremental allocations of the arrays;. that should save you some time.

I found another implementation of an exporter that uses a few more tricks:

https://github.com/Kupoman/blendergltf/blob/master/blendergltf.py#L569-L671

This pattern looks like it would be more performant: vdata = buf.add_accessor(view, 0, vertex_size, Buffer.FLOAT, num_verts, Buffer.VEC3) ... for j in range(3): vdata[(i 3) + j] = co[j] ndata[(i 3) + j] = normal[j]

the array access like above, assuming it's not hiding some sort of complex getter/setter, should be fast enough to be used in this type of workload.

In a few minutes I can paste the results of my timings.

[ncoder]

Read blend: C:\Users\ncoder\Downloads\Large Cruiser\Starship-blend\Starship.blend Info: Read library: 'C:\Users\ncoder\extern\glTF-Blender-Exporter\pbr_node\glTF2.blend', '//......\extern\glTF-Blender-Exporter\pbr_node\glTF2.blend', parent '' Info: Read library: 'C:\Users\ncoder\extern\glTF-Blender-Exporter\pbr_node\glTF2.blend', '//......\extern\glTF-Blender-Exporter\pbr_node\glTF2.blend', parent ''

INFO: Starting glTF 2.0 export PROFILE: Delta time: 0.0 (asset) PROFILE: Delta time: 0.0 (images) PROFILE: Delta time: 0.0 (textures) PROFILE: Delta time: 0.0010001659393310547 (materials) PROFILE: Delta time: 0.0 (cameras) INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 59094 Vertices: 59094 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 9474 Vertices: 9474 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 252 Vertices: 252 DEBUG: Adding primitive without splitting. Indices: 25674 Vertices: 25674 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive with splitting. Indices: 65535 Vertices: 65535 DEBUG: Adding primitive with splitting. Indices: 4734 Vertices: 4734 DEBUG: Creating temporary primitive for splitting DEBUG: Adding primitive with splitting. Indices: 3 Vertices: 3 INFO: Primitives created: 3 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 252 Vertices: 252 DEBUG: Adding primitive without splitting. Indices: 25674 Vertices: 25674 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 6270 Vertices: 6270 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 30 Vertices: 30 DEBUG: Adding primitive with splitting. Indices: 65535 Vertices: 65535 DEBUG: Adding primitive with splitting. Indices: 1602 Vertices: 1602 DEBUG: Creating temporary primitive for splitting DEBUG: Adding primitive with splitting. Indices: 3 Vertices: 3 INFO: Primitives created: 4 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 3144 Vertices: 3144 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 11880 Vertices: 11880 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 72 Vertices: 72 DEBUG: Adding primitive without splitting. Indices: 49416 Vertices: 49416 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 30 Vertices: 30 DEBUG: Adding primitive without splitting. Indices: 282 Vertices: 282 DEBUG: Adding primitive with splitting. Indices: 65535 Vertices: 65535 DEBUG: Adding primitive with splitting. Indices: 64026 Vertices: 64026 DEBUG: Creating temporary primitive for splitting DEBUG: Adding primitive with splitting. Indices: 3 Vertices: 3 INFO: Primitives created: 5 INFO: Extracting primitive DEBUG: Adding primitive with splitting. Indices: 65535 Vertices: 65535 DEBUG: Adding primitive with splitting. Indices: 37728 Vertices: 37728 DEBUG: Creating temporary primitive for splitting DEBUG: Adding primitive with splitting. Indices: 3 Vertices: 3 DEBUG: Adding primitive without splitting. Indices: 426 Vertices: 426 INFO: Primitives created: 4 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 171 Vertices: 171 DEBUG: Adding primitive without splitting. Indices: 52197 Vertices: 52197 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 216 Vertices: 216 DEBUG: Adding primitive without splitting. Indices: 27720 Vertices: 27720 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 3144 Vertices: 3144 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 47043 Vertices: 47043 INFO: Primitives created: 1 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 78 Vertices: 78 DEBUG: Adding primitive without splitting. Indices: 1368 Vertices: 1368 DEBUG: Adding primitive with splitting. Indices: 65535 Vertices: 65535 DEBUG: Adding primitive with splitting. Indices: 8424 Vertices: 8424 DEBUG: Creating temporary primitive for splitting DEBUG: Adding primitive with splitting. Indices: 3 Vertices: 3 INFO: Primitives created: 5 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 768 Vertices: 768 DEBUG: Adding primitive without splitting. Indices: 25848 Vertices: 25848 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 108 Vertices: 108 DEBUG: Adding primitive without splitting. Indices: 44664 Vertices: 44664 INFO: Primitives created: 2 INFO: Extracting primitive DEBUG: Adding primitive without splitting. Indices: 4806 Vertices: 4806 INFO: Primitives created: 1 PROFILE: Delta time: 1348.4236624240875 (meshes) PROFILE: Delta time: 0.007995843887329102 (nodes) PROFILE: Delta time: 0.0 (animations) PROFILE: Delta time: 0.0 (scenes) PROFILE: Delta time: 0.0 (scene) INFO: Finished glTF 2.0 export

22 minutes for 348k vertices...

[UX3D-nopper]

Thank you very much for your investigations.

I agree, that the approach from the link is faster but it does not work, as soon as flat shaded surfaces and/or specific modifiers are used: The exported result does not look like the scene from Blender. We had several issues because of this.

Can you please provide a trace of the same file, where the indices are integers and not shorts?

But I will have another look at the code and investigate, if organizing the data in a different way does speed up things.

[ncoder]

Indeed, it's much easier to make fast code if it doesn't need to be correct!

Integers and not shorts: I think you answered why I may be missing the backside of my model after this export.

[UX3D-nopper]

You are missing the backside in your viewer, but the primitives are in the glTF file, right? If not, this would be a bug.

[Moguri]

The blendergltf addon supports flat shaded surfaces. I double-checked this last night using BlenderPanda, which makes use of blendergltf. Exporting a UV sphere with enough subdiv operations to reach about two million faces took about two and a half minutes in one of our recent tests.

If blendergltf is exporting models incorrectly, we welcome bug reports. One of our key goals is performance, but not at the cost of correctness.

[UX3D-nopper]

Can you guys please try out the fix mentioned here: https://github.com/KhronosGroup/glTF-Blender-Exporter/issues/85

[UX3D-nopper]

Seems to be fixed/does not appear on our side anymore.