CyrilWaechter commented 2 years ago

Someone sent me a file which crash when importing it in Blender + BIM Add-on.

Crash have occured with:

IfcOpenShell freshly compiled (96902bf38a23226bc0c2c5137202ac67dcf65937) on Linux
blenderbim-220210 on Windows

From informations below, is there a way to find out which element makes process crash ? Original file is pretty large.

Error message from blender.crash.txt

# Blender 3.0.1, Commit date: 2022-01-25 17:19, Hash dc2d18018171
bpy.ops.bim.save_drawing_style(index="2")  # Operator
bpy.ops.object.select_all(action='SELECT')  # Operator
bpy.ops.object.delete(use_global=False, confirm=False)  # Operator
Deleted 3 object(s)  # Info
bpy.ops.import_ifc.bim()  # Operator
bpy.context.space_data.recent_folders_active = 0  # Property
bpy.ops.import_ifc.bim()  # Operator
bpy.context.space_data.system_bookmarks_active = 3  # Property

# backtrace
blender(BLI_system_backtrace+0x33) [0x5619b0328593]
blender(+0xebbb2e) [0x5619adccbb2e]
/usr/lib/libc.so.6(+0x42560) [0x7f768ae9b560]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_ZNK8IfcParse12ArgumentListcvSt6vectorIS1_IdSaIdEESaIS3_EEEv+0x9) [0x7f764056b8f9]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_ZNK8IfcParse12ArgumentList4typeEv+0x1b) [0x7f764056572b]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_Z9traverse_PN7IfcUtil12IfcBaseClassERSt3setIS1_St4lessIS1_ESaIS1_EER18traversal_recorderii+0x1a5) [0x7f764056db05]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_Z9traverse_PN7IfcUtil12IfcBaseClassERSt3setIS1_St4lessIS1_ESaIS1_EER18traversal_recorderii+0x1c2) [0x7f764056db22]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_Z9traverse_PN7IfcUtil12IfcBaseClassERSt3setIS1_St4lessIS1_ESaIS1_EER18traversal_recorderii+0x2b6) [0x7f764056dc16]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_ZN8IfcParse8traverseEPN7IfcUtil12IfcBaseClassEi+0x103) [0x7f764056de93]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_ZN7IfcGeom12KernelIfc2x321convert_openings_fastEPKN6Ifc2x310IfcProductERKN5boost10shared_ptrI12aggregate_ofINS1_18IfcRelVoidsElementEEEERKSt6vectorINS_26IfcRepresentationShapeItemESaISE_EERK7gp_TrsfRSG_+0x25c) [0x7f7642ae8ddc]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(_ZN7IfcGeom12KernelIfc2x342create_brep_for_representation_and_productERKNS_16IteratorSettingsEPN6Ifc2x317IfcRepresentationEPNS4_10IfcProductE+0x187a) [0x7f7642af396a]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(+0x50bf2e5) [0x7f7642b152e5]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(+0x50bf59f) [0x7f7642b1559f]
blender(_ZNSt13__future_base13_State_baseV29_M_do_setEPSt8functionIFSt10unique_ptrINS_12_Result_baseENS3_8_DeleterEEvEEPb+0x29) [0x5619af04af09]
/usr/lib/libc.so.6(+0x92788) [0x7f768aeeb788]
/home/cyril/.config/blender/3.0/scripts/modules/ifcopenshell/_ifcopenshell_wrapper.so(+0x50c0b88) [0x7f7642b16b88]
/usr/lib/libstdc++.so.6(+0xd54d4) [0x7f768b2224d4]
/usr/lib/libc.so.6(+0x8d5c2) [0x7f768aee65c2]
/usr/lib/libc.so.6(clone+0x44) [0x7f768af6b584]

# Python backtrace

Console log

Starting import process :: 0.00
Loading file :: 0.00
Calculate unit scale :: 0.00
Calculate model offset :: 0.00
Set units :: 0.00
Create project :: 0.00
Process element filter :: 0.21
Process context filter :: 0.00
Create collections :: 0.44
Create opening collection :: 0.00
Create materials :: 0.00
Create styles :: 0.05
Create annotation :: 0.00
Parsing native elements :: 3.08
Done creating geometry
Create native elements :: 0.00
250 (0%) elements processed in 0.67s ...
500 (1%) elements processed in 1.16s ...
750 (1%) elements processed in 0.78s ...
1000 (2%) elements processed in 0.81s ...
1250 (2%) elements processed in 0.65s ...
1500 (3%) elements processed in 0.77s ...
1750 (4%) elements processed in 0.76s ...
2000 (4%) elements processed in 0.75s ...
2250 (5%) elements processed in 1.86s ...
2500 (6%) elements processed in 0.90s ...
2750 (6%) elements processed in 2.12s ...
3000 (7%) elements processed in 1.07s ...
3250 (7%) elements processed in 1.41s ...
3500 (8%) elements processed in 2.51s ...
3750 (8%) elements processed in 3.32s ...
4000 (9%) elements processed in 4.29s ...
4250 (10%) elements processed in 2.57s ...
4500 (10%) elements processed in 1.58s ...
4750 (11%) elements processed in 1.46s ...
5000 (12%) elements processed in 1.20s ...
5250 (13%) elements processed in 1.32s ...
5500 (14%) elements processed in 1.48s ...
5750 (14%) elements processed in 1.31s ...
6000 (14%) elements processed in 1.58s ...
6250 (15%) elements processed in 1.68s ...
6500 (16%) elements processed in 1.87s ...
6750 (16%) elements processed in 1.60s ...
7000 (17%) elements processed in 1.92s ...
7250 (18%) elements processed in 1.69s ...
7500 (19%) elements processed in 2.26s ...
7750 (21%) elements processed in 1.59s ...
8000 (23%) elements processed in 1.73s ...
8250 (25%) elements processed in 1.85s ...
8500 (27%) elements processed in 1.88s ...
8750 (29%) elements processed in 2.00s ...
9000 (31%) elements processed in 2.11s ...
9250 (34%) elements processed in 2.16s ...
9500 (36%) elements processed in 2.11s ...
9750 (38%) elements processed in 2.13s ...
10000 (40%) elements processed in 2.26s ...
10250 (43%) elements processed in 2.46s ...
10500 (45%) elements processed in 2.33s ...
10750 (46%) elements processed in 2.31s ...
11000 (46%) elements processed in 2.46s ...
11250 (46%) elements processed in 2.50s ...
11500 (47%) elements processed in 2.51s ...
11750 (49%) elements processed in 2.55s ...
12000 (50%) elements processed in 2.55s ...
12250 (51%) elements processed in 2.47s ...
12500 (52%) elements processed in 2.60s ...
12750 (54%) elements processed in 2.76s ...
13000 (55%) elements processed in 2.61s ...
13250 (55%) elements processed in 2.63s ...
13500 (56%) elements processed in 2.71s ...
13750 (56%) elements processed in 2.78s ...
14000 (56%) elements processed in 3.08s ...
14250 (56%) elements processed in 2.86s ...
14500 (56%) elements processed in 3.17s ...
14750 (56%) elements processed in 3.09s ...
15000 (56%) elements processed in 2.95s ...
15250 (57%) elements processed in 3.04s ...
15500 (57%) elements processed in 3.07s ...
15750 (58%) elements processed in 3.39s ...
16000 (58%) elements processed in 3.14s ...
16250 (59%) elements processed in 3.20s ...
16500 (60%) elements processed in 3.26s ...
16750 (60%) elements processed in 3.42s ...
17000 (61%) elements processed in 3.54s ...
17250 (61%) elements processed in 3.41s ...
17500 (61%) elements processed in 3.70s ...
17750 (61%) elements processed in 3.72s ...
18000 (61%) elements processed in 3.78s ...
18250 (61%) elements processed in 3.82s ...
18500 (62%) elements processed in 3.61s ...
18750 (63%) elements processed in 3.64s ...
19000 (63%) elements processed in 3.84s ...
19250 (64%) elements processed in 4.03s ...
19500 (64%) elements processed in 3.56s ...
19750 (64%) elements processed in 2.91s ...
20000 (64%) elements processed in 2.24s ...
20250 (64%) elements processed in 3.00s ...
20500 (64%) elements processed in 2.60s ...
20750 (64%) elements processed in 4.21s ...
21000 (66%) elements processed in 4.29s ...
21250 (69%) elements processed in 4.14s ...
21500 (69%) elements processed in 4.33s ...
21750 (70%) elements processed in 3.56s ...
22000 (72%) elements processed in 3.69s ...
22250 (74%) elements processed in 3.87s ...
22500 (78%) elements processed in 4.60s ...
22750 (83%) elements processed in 3.78s ...
23000 (87%) elements processed in 3.99s ...
23250 (91%) elements processed in 4.09s ...
23500 (95%) elements processed in 4.58s ...
Writing: /tmp/blender.crash.txt
Erreur de segmentation (core dumped)

aothms commented 2 years ago

From informations below, is there a way to find out which element makes process crash

No, not directly. But from the callstack, it's apparent that the crash is on this line.

https://github.com/IfcOpenShell/IfcOpenShell/blob/v0.7.0/src/ifcgeom/IfcGeomFunctions.cpp#L1355

If you run IfcConvert with -vv it will probably tell you the host element. The error is related to a representation of an opening in this element. Maybe something invalid in the file. You might also want to try with with

python3 -m ifcopenshell.validate <file.ifc>

Moult commented 2 years ago

Another approach, is to import the file using advanced mode ("Enable Advanced Mode" when browsing for your iFC). This opens the IFC but does not import geometry.

Then in the IFC Debug panel press "Test All Shapes" this runs create_shape on every element and prints out the element it was creating on the console. When it crashes, you will see the element it was working on printed just before.

Might take a while for 23500 objects but it may help.

CyrilWaechter commented 2 years ago

Using IfcConvert -vv on file works. It does not crash so it doesn't help.

"Test All Shapes" in Blender + BIM Add-on task also complete without crash.

ifcopenshell.validate spots following 3 anomalies but could it really cause the crash ?

QEA-REZ-ARC-CHE_Patched.ifc
In #10418013=IfcLocalPlacement(#9929322,#10418012)
() not valid for <inverse PlacesObject: set [1:1] of <entity IfcProduct> for <attribute ObjectPlacement?: <entity IfcObjectPlacement>>>
In #7842838=IfcLocalPlacement(#4531518,#7842837)
() not valid for <inverse PlacesObject: set [1:1] of <entity IfcProduct> for <attribute ObjectPlacement?: <entity IfcObjectPlacement>>>
In #10665222=IfcLocalPlacement(#9929322,#10665221)
() not valid for <inverse PlacesObject: set [1:1] of <entity IfcProduct> for <attribute ObjectPlacement?: <entity IfcObjectPlacement>>>

Sorry for late answer.

aothms commented 2 years ago

following 3 anomalies but could it really cause the crash ?

No, those are just placements without a related product, which is apparently forbidden by the schema, but wouldn't cause any issues.

Does the crash happen consistently?

So then there's two options I see:

Could it be related to multi-threading? From the element in the stack trace (std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*)) it appears you're using multi-threading. Could you add -j <num_threads> to the IfcConvert execution?
Would BlenderBIM somehow have modified the ifc graph?

Moult commented 2 years ago

@aothms if he's just loading a file and pressing the "Test All Shapes" button it shouldn't modify the graph.

aothms commented 2 years ago

But that's also the scenario that didn't crash:

"Test All Shapes" in Blender + BIM Add-on task also complete without crash.

CyrilWaechter commented 2 years ago

Does the crash happen consistently?

It was but yesterday I was able to import the file into blender 3.1 + BIM Add-on (ifcopenshell recompiled on 2985bba157da40030b4b6e2159481ae08cbd603f) on Linux. So I tried on Windows again with 220321 and got a different error:

23250 (94%) elements processed in 4.96s ...
Error   : EXCEPTION_ACCESS_VIOLATION
Address : 0x00007FFEC6E71A50
Module  : VCRUNTIME140.dll
Thread  : 00003d18
Writing: C:\Users\Cyril\AppData\Local\Temp\blender.crash.txt
Error loading symbols C:\Program Files\Blender Foundation\Blender 3.1\blender.pdb
        error:0xc0000005
        size = 109604864
        base=0x00007FF6C2D30000
Error   : EXCEPTION_ACCESS_VIOLATION
Address : 0x00007FFEC6E97DB9
Module  : tbbmalloc.dll
Thread  : 00003d18
Writing: C:\Users\Cyril\AppData\Local\Temp\blender.crash.txt
Error loading symbols C:\Program Files\Blender Foundation\Blender 3.1\blender.pdb
        error:0xc0000005
        size = 109604864
        base=0x00007FF6C2D30000

220321 might not be shipped with last ifcopenshell version which might explain that it succeeded on Linux and not with on Windows.

Moult commented 2 years ago

Can you also try (in the debug panel) to delete all HDF5 cache before testing? Potentially a previous import failed, the HDF5 cache got messed up, and that is causing further attempts to also fail?

CyrilWaechter commented 2 years ago

Can you also try (in the debug panel) to delete all HDF5 cache before testing? Potentially a previous import failed, the HDF5 cache got messed up, and that is causing further attempts to also fail?

Tested: deleting cache does not change the result.

Moult commented 2 years ago

There are Windows builds available for that commit, so if you download the IfcOpenBot Windows build and replace the IfcOpenShell pyd / dll or whatever file it uses but keep everything else, you can see if it is to do with an outdated IfcOpenBot build.

I've also just bumped the build in the Makefile so another alternative is to wait for the next BlenderBIM Add-on daily Github release :)

CyrilWaechter commented 2 years ago

I've also just bumped the build in the Makefile so another alternative is to wait for the next BlenderBIM Add-on daily Github release :)

It crashed the first time but worked the second time. The same occurred on the computer of the one who reported me this issue.

Also percentages do not seems accurate any more considering there is many 99% / 100% but only few 1% 2% etc… for the same amount of elements (250):

Starting import process :: 0.00
Loading file :: 0.00
Calculate unit scale :: 0.00
Calculate model offset :: 0.00
Set units :: 0.00
Create project :: 0.00
Process element filter :: 0.36
Process context filter :: 0.00
Create collections :: 0.78
Create opening collection :: 0.00
Create materials :: 0.01
Create styles :: 0.08
Create annotation :: 0.00
Parsing native elements :: 6.12
Done creating geometry
Create native elements :: 0.00
250 (0%) elements processed in 0.61s ...
500 (1%) elements processed in 0.88s ...
750 (1%) elements processed in 0.86s ...
1000 (2%) elements processed in 0.65s ...
1250 (3%) elements processed in 0.63s ...
1500 (4%) elements processed in 0.95s ...
1750 (4%) elements processed in 0.72s ...
2000 (5%) elements processed in 0.81s ...
2250 (6%) elements processed in 1.32s ...
2500 (6%) elements processed in 0.95s ...
2750 (6%) elements processed in 0.99s ...
3000 (7%) elements processed in 1.00s ...
3250 (7%) elements processed in 1.02s ...
3500 (8%) elements processed in 2.07s ...
3750 (8%) elements processed in 3.13s ...
4000 (9%) elements processed in 4.04s ...
4250 (10%) elements processed in 2.12s ...
4500 (10%) elements processed in 2.04s ...
4750 (11%) elements processed in 1.46s ...
5000 (13%) elements processed in 1.15s ...
5250 (14%) elements processed in 1.36s ...
5500 (14%) elements processed in 1.42s ...
5750 (14%) elements processed in 1.46s ...
6000 (15%) elements processed in 1.79s ...
6250 (16%) elements processed in 1.58s ...
6500 (16%) elements processed in 1.79s ...
6750 (16%) elements processed in 1.54s ...
7000 (18%) elements processed in 1.96s ...
7250 (18%) elements processed in 1.59s ...
7500 (20%) elements processed in 1.66s ...
7750 (22%) elements processed in 1.80s ...
8000 (27%) elements processed in 1.73s ...
8250 (35%) elements processed in 1.95s ...
8500 (45%) elements processed in 1.99s ...
8750 (46%) elements processed in 1.91s ...
9000 (46%) elements processed in 2.43s ...
9250 (46%) elements processed in 2.22s ...
9500 (47%) elements processed in 2.08s ...
9750 (49%) elements processed in 2.14s ...
10000 (50%) elements processed in 2.42s ...
10250 (51%) elements processed in 2.16s ...
10500 (52%) elements processed in 2.22s ...
10750 (54%) elements processed in 2.29s ...
11000 (55%) elements processed in 2.94s ...
11250 (56%) elements processed in 2.44s ...
11500 (56%) elements processed in 2.80s ...
11750 (56%) elements processed in 3.12s ...
12000 (56%) elements processed in 3.11s ...
12250 (56%) elements processed in 3.14s ...
12500 (56%) elements processed in 3.25s ...
12750 (56%) elements processed in 3.36s ...
13000 (56%) elements processed in 3.45s ...
13250 (56%) elements processed in 2.83s ...
13500 (56%) elements processed in 3.55s ...
13750 (56%) elements processed in 3.71s ...
14000 (56%) elements processed in 3.28s ...
14250 (56%) elements processed in 3.14s ...
14500 (57%) elements processed in 3.18s ...
14750 (57%) elements processed in 3.81s ...
15000 (58%) elements processed in 3.84s ...
15250 (59%) elements processed in 3.81s ...
15500 (59%) elements processed in 3.31s ...
15750 (60%) elements processed in 3.29s ...
16000 (61%) elements processed in 3.30s ...
16250 (61%) elements processed in 3.50s ...
16500 (61%) elements processed in 4.26s ...
16750 (61%) elements processed in 4.50s ...
17000 (61%) elements processed in 3.78s ...
17250 (62%) elements processed in 3.70s ...
17500 (63%) elements processed in 3.97s ...
17750 (63%) elements processed in 3.90s ...
18000 (63%) elements processed in 3.85s ...
18250 (64%) elements processed in 3.80s ...
18500 (64%) elements processed in 3.71s ...
18750 (64%) elements processed in 3.89s ...
19000 (64%) elements processed in 3.70s ...
19250 (64%) elements processed in 3.72s ...
19500 (64%) elements processed in 3.33s ...
19750 (64%) elements processed in 2.86s ...
20000 (64%) elements processed in 2.10s ...
20250 (65%) elements processed in 3.75s ...
20500 (66%) elements processed in 2.44s ...
20750 (69%) elements processed in 3.78s ...
21000 (70%) elements processed in 4.61s ...
21250 (73%) elements processed in 4.40s ...
21500 (79%) elements processed in 3.72s ...
21750 (85%) elements processed in 4.06s ...
22000 (88%) elements processed in 4.03s ...
22250 (92%) elements processed in 4.14s ...
22500 (97%) elements processed in 5.12s ...
22750 (98%) elements processed in 4.66s ...
23000 (98%) elements processed in 3.93s ...
23250 (98%) elements processed in 4.27s ...
23500 (99%) elements processed in 5.46s ...
23750 (99%) elements processed in 3.58s ...
24000 (99%) elements processed in 3.97s ...
24250 (99%) elements processed in 3.57s ...
24500 (99%) elements processed in 3.08s ...
24750 (99%) elements processed in 3.53s ...
25000 (99%) elements processed in 3.61s ...
25250 (99%) elements processed in 3.71s ...
25500 (99%) elements processed in 3.72s ...
25750 (99%) elements processed in 3.82s ...
26000 (99%) elements processed in 3.65s ...
26250 (99%) elements processed in 3.48s ...
26500 (99%) elements processed in 3.98s ...
26750 (99%) elements processed in 3.62s ...
27000 (99%) elements processed in 3.73s ...
27250 (99%) elements processed in 3.39s ...
27500 (99%) elements processed in 3.53s ...
27750 (99%) elements processed in 4.03s ...
28000 (99%) elements processed in 3.76s ...
28250 (99%) elements processed in 3.67s ...
28500 (99%) elements processed in 4.03s ...
28750 (100%) elements processed in 4.29s ...
29000 (100%) elements processed in 4.74s ...
29250 (100%) elements processed in 4.38s ...
29500 (100%) elements processed in 3.71s ...
29750 (100%) elements processed in 3.91s ...
30000 (100%) elements processed in 4.76s ...
30250 (100%) elements processed in 4.25s ...
30500 (100%) elements processed in 3.91s ...
30750 (100%) elements processed in 3.77s ...
31000 (100%) elements processed in 4.08s ...
31250 (100%) elements processed in 4.02s ...
31500 (100%) elements processed in 4.16s ...
31750 (100%) elements processed in 4.65s ...
32000 (100%) elements processed in 5.31s ...
32250 (100%) elements processed in 5.30s ...
32500 (100%) elements processed in 5.60s ...
32750 (100%) elements processed in 5.38s ...
33000 (100%) elements processed in 5.41s ...
33250 (100%) elements processed in 5.45s ...
33500 (100%) elements processed in 5.48s ...
33750 (100%) elements processed in 5.52s ...
34000 (100%) elements processed in 5.53s ...
34250 (100%) elements processed in 5.60s ...
34500 (100%) elements processed in 5.61s ...
34750 (100%) elements processed in 5.67s ...
35000 (100%) elements processed in 5.70s ...
35250 (100%) elements processed in 5.76s ...
35500 (100%) elements processed in 5.72s ...
35750 (100%) elements processed in 5.82s ...
36000 (100%) elements processed in 5.94s ...
36250 (100%) elements processed in 5.88s ...
36500 (100%) elements processed in 5.60s ...
36750 (100%) elements processed in 5.95s ...
37000 (100%) elements processed in 5.99s ...
37250 (100%) elements processed in 6.00s ...
37500 (100%) elements processed in 6.06s ...
37750 (100%) elements processed in 6.09s ...
38000 (100%) elements processed in 6.11s ...
38250 (100%) elements processed in 6.13s ...
38500 (100%) elements processed in 5.50s ...
38750 (100%) elements processed in 4.56s ...
39000 (100%) elements processed in 4.71s ...
39250 (100%) elements processed in 4.77s ...
39500 (100%) elements processed in 4.99s ...
39750 (100%) elements processed in 4.94s ...
40000 (100%) elements processed in 4.78s ...
40250 (100%) elements processed in 4.91s ...
40500 (100%) elements processed in 5.18s ...
40750 (100%) elements processed in 4.91s ...
41000 (100%) elements processed in 5.07s ...
41250 (100%) elements processed in 3.89s ...
41500 (100%) elements processed in 4.95s ...
41750 (100%) elements processed in 4.93s ...
42000 (100%) elements processed in 6.55s ...
42250 (100%) elements processed in 6.07s ...
Done creating geometry
250 elements processed in 3.40s ...
500 elements processed in 3.37s ...
750 elements processed in 3.34s ...
1000 elements processed in 3.38s ...
1250 elements processed in 3.39s ...
1500 elements processed in 3.40s ...
Done creating geometry
Create elements :: 664.38
Create grids :: 0.72
250 (100%) elements processed in 5.93s ...
Done creating geometry
Create spatial elements :: 8.97
Create structural items :: 0.00
Create type products :: 5.15
Place objects in collections :: 90.42
Add project to scene :: 0.10
Setting default context :: 0.42
Import finished in 777.52 seconds

Moult commented 2 years ago

Yeah reporting progress is harder than I thought :) Currently the percentage doesn't correlate to the number of Blender elements processed - it correlates to the number of IFC iterator elements processed. I'll see what I can do to do an average of the two.

CyrilWaechter commented 2 years ago

Could it be related to multi-threading? From the element in the stack trace (std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*)) it appears you're using multi-threading. Could you add -j <num_threads> to the IfcConvert execution?

I tried but it did not fail. But considering on Windows import crashed the first time on 2 different computers and worked on second import it might point that this is coming from multithreading as you thought. I think success on second import is explained by cached processed geometries.

aothms commented 2 years ago

Strange. So can we then isolate a minimal python script without blenderbim logic? @Moult can you fill us in on the settings, would this look good to you otherwise?

import ifcopenshell
import ifcopenshell.geom

ifcopenshell.ifcopenshell_wrapper.turn_on_detailed_logging()

ss = ifcopenshell.geom.settings()

f = ifcopenshell.open(...)

for elem in ifcopenshell.geom.iterate(ss, f, num_threads=X):
    print(elem.guid)

Moult commented 2 years ago

Here is a minimal script which does most of what the importer does. (There are other non iterator-things that it does too but they are generally quite docile or emit pretty self explanatory Python traces if they fail)

Notable differences between a super minimal script is that it 1) uses set_context_ids, 2) processes 3D first, and falls back to 2D, 3) strict tolerance is enabled 4) hdf5 caching is used.

import ifcopenshell
import ifcopenshell.api
import ifcopenshell.geom
import multiprocessing

class BlenderImporter:
    def __init__(self):
        self.file = ifcopenshell.open('/home/dion/Downloads/project.ifc')
        self.cache_path = "cache.h5"
        self.should_use_cpu_multiprocessing = True
        self.deflection_tolerance = 0.001
        self.angular_tolerance = 0.5

    def execute(self):
        self.created_guids = set()

        self.settings = ifcopenshell.geom.settings()
        self.settings.set_deflection_tolerance(self.deflection_tolerance)
        self.settings.set_angular_tolerance(self.angular_tolerance)
        self.settings.set(self.settings.STRICT_TOLERANCE, True)
        self.settings_2d = ifcopenshell.geom.settings()
        self.settings_2d.set(self.settings_2d.INCLUDE_CURVES, True)

        self.process_element_filter()
        self.process_context_filter()
        self.create_elements(self.elements)

    def process_element_filter(self):
        # Blender users can typically customise a filtered set of elements to import.
        # For simplicity, let's select everything:
        self.elements = set(self.file.by_type("IfcProduct"))

    def process_context_filter(self):
        # Facetation is to accommodate broken Revit files
        # See https://forums.buildingsmart.org/t/suggestions-on-how-to-improve-clarity-of-representation-context-usage-in-documentation/3663/6?u=moult
        self.body_contexts = [
            c.id()
            for c in self.file.by_type("IfcGeometricRepresentationSubContext")
            if c.ContextIdentifier in ["Body", "Facetation"]
        ]
        # Ideally, all representations should be in a subcontext, but some BIM programs don't do this correctly
        self.body_contexts.extend(
            [
                c.id()
                for c in self.file.by_type("IfcGeometricRepresentationContext", include_subtypes=False)
                if c.ContextType == "Model"
            ]
        )
        if self.body_contexts:
            self.settings.set_context_ids(self.body_contexts)
        # Annotation is to accommodate broken Revit files
        # See https://github.com/Autodesk/revit-ifc/issues/187
        self.plan_contexts = [
            c.id()
            for c in self.file.by_type("IfcGeometricRepresentationContext")
            if c.ContextType in ["Plan", "Annotation"]
        ]
        if self.plan_contexts:
            self.settings_2d.set_context_ids(self.plan_contexts)

    def create_elements(self, elements):
        # Based on my experience in viewing BIM models, representations are prioritised as follows:
        # 1. 3D Body, 2. 2D Plans, 3. Point clouds, 4. No representation
        # If an element has a representation that doesn't follow 1, 2, or 3, it will not show by default.
        # The user can load them later if they want to view them.
        products = self.create_products(elements)
        elements -= products
        products = self.create_products(elements, is_curve=True)
        elements -= products
        # Creating point clouds and non geometric objects are specific to Blender so no code is shown.
        pass

    def create_products(self, products, is_curve=False):
        results = set()
        if not products:
            return results
        settings = self.settings_2d if is_curve else self.settings
        if self.should_use_cpu_multiprocessing:
            iterator = ifcopenshell.geom.iterator(
                settings, self.file, multiprocessing.cpu_count(), include=products
            )
        else:
            iterator = ifcopenshell.geom.iterator(settings, self.file, include=products)
        cache = self.get_cache()
        if cache:
            iterator.set_cache(cache)
        valid_file = iterator.initialize()
        if not valid_file:
            return results
        total = 0
        while True:
            total += 1
            if total % 250 == 0:
                print(f"{total} elements processed")
            shape = iterator.get()
            if shape:
                product = self.file.by_id(shape.guid)
                if self.body_contexts:
                    print("Created", product)
                    self.created_guids.add(shape.guid)
                    results.add(product)
                else:
                    if shape.context not in ["Body", "Facetation"] and shape.guid in self.created_guids:
                        # We only load a single context, and we prioritise the Body context. See #1290.
                        pass
                    else:
                        print("Created", product)
                        self.created_guids.add(shape.guid)
                        results.add(product)
            if not iterator.next():
                break
        print("Done creating geometry")
        return results

    def get_cache(self):
        cache_settings = ifcopenshell.geom.settings()
        cache_settings.set(cache_settings.STRICT_TOLERANCE, True)
        try:
            return ifcopenshell.geom.serializers.hdf5(self.cache_path, cache_settings)
        except:
            return

BlenderImporter().execute()

CyrilWaechter commented 2 years ago

I got no failure with above script on Linux using ifcopenshell f70ab06.

Moult commented 2 years ago

Cheers, is there anything else you need @aothms ?

aothms commented 2 years ago

Uhm, I'm still at loss how I can look into this?

Is the summary @CyrilWaechter, ifcopenshell f70ab06 crashes when used in blenderbim, but not when using @Moult's script? Is it the exact same interpreter?

Moult commented 7 months ago

Closing as I'm not sure if this can be progressed any further, and a lot has changed since 2 years ago.

IfcOpenShell / IfcOpenShell

Crash core dumped on ifc import #2081

Error message from blender.crash.txt

Console log