LPub crashes when exporting png images (Native Renderer)

technicbasics commented 1 year ago

Subject

When exporting the building instructions to png images, the program crashes after about a third 99/330 pages. The following errors are displayed --> see pictures in the zip file After reopening the program, I selected this page manually, the correct page was displayed in the header, however, the start page remained present Only after a page refresh the page is displayed correctly. After another 5 page turns, the program unexpectedly crashes again.

Environment

Version of LPub3D - 2.4.6 r100 64-bit mobile
Operating system - Windows 11, Home Edition 64bit
Processor 11th Gen Intel(R) Core(TM) i9-11900H @ 2.50GHz 2.50 GHz
RAM 16,0 GB
Grafik NVIDIA RTX 3600, 6GB

Configuration

[Note: Provide details and content below as needed with appropriate redactions. To produce the Windows registry extract for an installed distribution, go to the command console and enter the command line shown. The extract file will be placed on your desktop.]

LPub3D installation path
- Windows: `C:\LPub3D\LPub3D_x86_64-2.4.6.100.3211_20230330\LPub3D_x86_64
Application data path
- Windows: %USERPROFILE%\AppData\Local\LPUb3D Software\LPub3D\

Logs

see .zip file

Screenshots

See .zip file LPub3DLog.zip

trevorsandy commented 1 year ago

Thank you for reporting this behaviour.

On review of the log file, it looks like the abnormal end starts after an invalid OpenGL context in the LDView interface module which is triggered after page 99

2023-03-30T19:03:49.185 INFO Page 99 exported. Elapsed time: 4 minutes 29.414 seconds. 
2023-03-30T19:03:49.515 NOTICE void __cdecl Gui::exportAs(const class QString &) @ln 1544 "Exporting .png images 99 of 330, size(in pixels) W 3508 x H 2480, orientation Landscape, DPI 300, pixel ratio 1" 
2023-03-30T19:03:52.708 INFO LDV loaded 'Default' preference set 
2023-03-30T19:03:53.172 ERROR ERROR - The OpenGL context is not valid! 
...

Later on, for the same page/assembly render, there is a failure to generate the STEP's pointer monochrome image (used to determine the position of the pointer arrow). Consequently the pointer arrow meta command was not generated successfully - this may also be a contributor to the abnormal end but I'm not sure.

2023-03-30T19:03:53.197 ERROR Could not write to Native CSI PNG object file: [D:/Lego/Bauanleitungen/Hans/Pisten Bully 2/LPub3D/tmp/pointerMono.png]. Reason: Image is empty. 
2023-03-30T19:03:53.197 TRACE Native PNG Export for command:  
2023-03-30T19:04:44.694 INFO LDV loaded 'Default' preference set 
2023-03-30T19:04:44.699 INFO Native Renderer CSI Arguments: InputFileName: D:/Lego/Bauanleitungen/Hans/Pisten Bully 2/LPub3D/tmp/csi.ldr, OutputFileName: D:/Lego/Bauanleitungen/Hans/Pisten Bully 2/LPub3D/tmp/pointerMono.png, ImageFileName: D:\Lego\Bauanleitungen\Hans\Pisten Bully 2\LPub3D\assem\main_1_submodel-222_36_submodel-257-0_1_0_0_0_1_0_0_0_1_120_3507_300_DPI_0.85_45_23_45.png, ExportMode: PNG, TransBackground: True, HighlightNewParts: False, UseImageSize: False, LineWidth: 1, Resolution: 300, ImageWidth: 3507, ImageHeight: 2480, PageWidth: 3507, PageHeight: 2480, PreferencesFoV: 45, OptionsFoV: 45, CameraFoVy: 60, CameraZNear: 25, CameraZFar: 50000, CameraDDF: 8, CameraDistance (Scale: 0.85): 5043107, CameraName: Default, CameraProjection: Perspective, UsingViewpoint: False, CameraLatitude: 23, CameraLongitude: 45, ZoomExtents: False, CameraTarget: X(0) Y(0) Z(0) , StudStyle: 6 High Contrast
2023-03-30T19:04:44.700 ERROR Could not open Loader for ViewerStepKey: '285;9;120', FileName: 'csi.ldr', [Use File] 
2023-03-30T19:04:44.713 ERROR Render momo annotation image for part [32062.dat] failed. 
2023-03-30T19:04:44.713 ERROR Could not generate meta for line []

I have concluded that this behaviour is very likely related to the unique configuration of your instruction document at this page. So to save time, it would be good to have your model file(s) to reproduce this behaviour.

As your instruction document is not intended for public release, can you DM me via my email address trevor.sandy@gmail.com a zipped archive of the following items :

1. The model working files including any external files relevant to generating the suspect page
1. The contents of the 'LPub3D' working folder
1. The dmp file at C:\Users\Johann\AppData\Local\Temp\LPub3D.dmp

Let me know when you have transmitted the email so I can be sure to check for it.

Cheers,

technicbasics commented 1 year ago

You got a mail. :-)

Regards Johann

trevorsandy commented 1 year ago

Got it! Many thanks.

I 'll take a look.

Cheers,

technicbasics commented 1 year ago

With pleasure. Small information to the file: It currently contains 318 submodels. Of these, 4 are more or less main submodels, since I draw the model in modular form, otherwise it becomes too large. 3 of these 4 models are 99% finalized --> submodels 2, 40 and 94. The fourth model is 222.

trevorsandy commented 1 year ago

~Ok, it looks like the configuration of your document may be triggering an image memory limit.~

I'm getting this error after about 70 pages on my system:

QImage: out of memory, returning null image

Without the debugger and IDE, I imagine the page count would be higher.

After the OOM message, the behaviour becomes unstable suggesting an out-of-memory scenario.

I'm taking a look at what can be done on my side.

~However, you can can also break the instruction document into multiple books to help you get to the end of your editing.~

Cheers,

trevorsandy commented 1 year ago

After a deeper look, I believe I've found the source of the AbEnd.

I've checked in a correction which you can evaluate using the latest DevOps build (r106) .

Along the way, I found several other items that I expect to treat before publishing the next release build.

I was able to render up to page 164 before I terminated the run.

If you continue to encounter the reported behaviour with the latest build, I'll reopen this ticket ?

Cheers,

technicbasics commented 1 year ago

After the OOM message, the behaviour becomes unstable suggesting an out-of-memory scenario.

I have also observed that with the memory... after the start of the program this occupies approx. 5GB RAM, with exporting the occupied memory rises in the further, after approx. 90 sides are occupied with me then in approximately 15GB and then the program crashes, after this crash approx. 5GB are released, after the termination of the program the remainder of the occupied memory... (see the attached picture) 2023-04-01 14_27_29-Task-Manager

Before the program crashes, some more such pages are generated... Pistenbully 600 Polar_300_DPI_page_95

I also noticed that I managed 90 pages on the first try, 170 on the second, 250 on the third and all pages on the fourth. The pages that were created in the previous attempts were also created much faster in the subsequent attempts. Are the pages buffered somewhere?

I have now changed the graphics settings to force LPub3D to run on the NVIDIA card. I think this has caused an improvement, but can't say for sure yet. I also started an attempt (after exporting all pages as png) to create a pdf directly. This also worked without crashing the program, but at the very end this message was displayed... 2023-04-01 14_44_07-Pistenbully 600 Polar mpd - LPub3D v2 4 6 r100 (Release)

I will try the r106 asap but not sure if I can give feedback today....

trevorsandy commented 1 year ago

I have also observed that with the memory...

Indeed, I found that there was a fall-through bug in the render call that effectively instantiated the LDView interface for generating content such as the HTML parts list. As garbage collection would not occur until the end of image generation, we had a excessive memory consumption scenario that evolved into an out-of-memory scenario due to the size of the instruction document.

Indications of this behaviour was indicated in the log by the following lines:

2023-03-30T19:00:05.237 NOTICE void __cdecl Gui::exportAs(const class QString &) @ln 1544 "Exporting .png images 2 of 330, size(in pixels) W 3508 x H 2480, orientation Landscape, DPI 300, pixel ratio 1" 
2023-03-30T19:00:05.707 INFO LDV loaded 'Default' preference set 
2023-03-30T19:00:14.554 INFO LDV loaded 'Default' preference set 
2023-03-30T19:00:15.452 INFO LDV loaded 'Default' preference set

Anyway, after treating the fall-through, memory consumption was within the expected limit so I believe this was the source of the AbEnd.

Before the program crashes, some more such pages are generated...

Indeed, this is an indication that no more images can be generated. In debug I saw the message QImage: out of memory, returning null image at this point.

The pages that were created in the previous attempts were also created much faster in the subsequent attempts. Are the pages buffered somewhere ?

Yes, the PLI, CLI and SMI images are generated for each page as the document cycles, If there are no modifications to a page at subsequent model file page process or export, the images are not regenerated. This can be changed, if you wish, in the export dialogue by checking the clear cache flag.

ImageGenCrash_01

This also worked without crashing the program, but at the very end this message was displayed...

This behaviour was added in #639. Before 639, you would receive only the completion dialogue. If there are no obvious errors in the exported output, WARNINGS can be safely ignored.

Cheers,

trevorsandy / lpub3d