search

sn4k3 / UVtools

MSLA/DLP, file analysis, calibration, repair, conversion and manipulation
GNU Affero General Public License v3.0
1.21k stars 104 forks source link

[Bug] OpenCV: Failed to allocate n bytes #758

Closed alfonslm closed 1 year ago

alfonslm commented 1 year ago

System

UVtools v3.15.1 X64
Operative system: Microsoft Windows 10.0.19045 X64
Processor: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
Processor cores: 40
Memory RAM: 16.15 / 31.89 GB
Runtime: win10-x64
Framework: .NET 6.0.18
AvaloniaUI: 0.10.21
OpenCV: 4.7.0

Sreens, resolution, working area, usable area:
1: 1280 x 1024 @ 100%
    WA: 1280 x 984    UA: 1280 x 984
2: 1920 x 1080 @ 100% (Primary) (On this)
    WA: 1920 x 1040    UA: 1920 x 1040
3: 1920 x 1080 @ 100%
    WA: 1920 x 1040    UA: 1920 x 1040

Path:       C:\Program Files\UVtools\
Executable: C:\Program Files\UVtools\UVtools.exe
Loaded file: TarrasqueTitan_Base.pm5s [Version: 518] [Class: PhotonWorkshopFile]

Printer and Slicer

Description of the bug

My guess for what happened is that a RAM spike crashed the operation after about 10 min when the process was around 90% done.

System.AggregateException: One or more errors occurred. (OpenCV: Failed to allocate 168036484 bytes) (External component has thrown an exception.) (OpenCV: Failed to allocate 41868288 bytes) (External component has thrown an exception.)
 ---> Emgu.CV.Util.CvException: OpenCV: Failed to allocate 168036484 bytes
   at Emgu.CV.CvInvoke.cveConnectedComponentsWithStats(IntPtr image, IntPtr labels, IntPtr stats, IntPtr centroids, LineType connectivity, DepthType type, ConnectedComponentsAlgorithmsTypes cllType)
   at Emgu.CV.CvInvoke.ConnectedComponentsWithStats(IInputArray image, IOutputArray labels, IOutputArray stats, IOutputArray centroids, LineType connectivity, DepthType labelType, ConnectedComponentsAlgorithmsTypes cclType)
   at UVtools.Core.Managers.IssueManager.<>c__DisplayClass18_2.<DetectIssues>b__9(Int64 layerIndexInt) in UVtools.Core\Managers\IssueManager.cs:line 649
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
   at System.Threading.Tasks.Parallel.ForWorker64[TLocal](Int64 fromInclusive, Int64 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException)
   at System.Threading.Tasks.Parallel.ForWorker64[TLocal](Int64 fromInclusive, Int64 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.For(Int64 fromInclusive, Int64 toExclusive, ParallelOptions parallelOptions, Action`1 body)
   at UVtools.Core.Managers.IssueManager.DetectIssues(IssuesDetectionConfiguration config, OperationProgress progress) in UVtools.Core\Managers\IssueManager.cs:line 237
   at UVtools.WPF.MainWindow.<>c__DisplayClass222_0.<ComputeIssues>b__0() in UVtools.WPF\MainWindow.Issues.cs:line 644
 ---> (Inner Exception #1) System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
   at Emgu.CV.CvInvoke.cveConnectedComponentsWithStats(IntPtr image, IntPtr labels, IntPtr stats, IntPtr centroids, LineType connectivity, DepthType type, ConnectedComponentsAlgorithmsTypes cllType)
   at Emgu.CV.CvInvoke.ConnectedComponentsWithStats(IInputArray image, IOutputArray labels, IOutputArray stats, IOutputArray centroids, LineType connectivity, DepthType labelType, ConnectedComponentsAlgorithmsTypes cclType)
   at UVtools.Core.Managers.IssueManager.<>c__DisplayClass18_2.<DetectIssues>b__9(Int64 layerIndexInt) in UVtools.Core\Managers\IssueManager.cs:line 649
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

 ---> (Inner Exception #2) Emgu.CV.Util.CvException: OpenCV: Failed to allocate 41868288 bytes
   at Emgu.CV.CvInvoke.cveSubtract(IntPtr src1, IntPtr src2, IntPtr dst, IntPtr mask, DepthType dtype)
   at Emgu.CV.CvInvoke.Subtract(IInputArray src1, IInputArray src2, IOutputArray dst, IInputArray mask, DepthType dtype)
   at UVtools.Core.Managers.IssueManager.<>c__DisplayClass18_2.<DetectIssues>b__9(Int64 layerIndexInt) in UVtools.Core\Managers\IssueManager.cs:line 649
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

 ---> (Inner Exception #3) System.Runtime.InteropServices.SEHException (0x80004005): External component has thrown an exception.
   at Emgu.CV.CvInvoke.cveConnectedComponentsWithStats(IntPtr image, IntPtr labels, IntPtr stats, IntPtr centroids, LineType connectivity, DepthType type, ConnectedComponentsAlgorithmsTypes cllType)
   at Emgu.CV.CvInvoke.ConnectedComponentsWithStats(IInputArray image, IOutputArray labels, IOutputArray stats, IOutputArray centroids, LineType connectivity, DepthType labelType, ConnectedComponentsAlgorithmsTypes cclType)
   at UVtools.Core.Managers.IssueManager.<>c__DisplayClass18_2.<DetectIssues>b__9(Int64 layerIndexInt) in UVtools.Core\Managers\IssueManager.cs:line 649
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass20_0`1.<ForWorker64>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

How to reproduce

Run low on ram

Files

No response

github-actions[bot] commented 1 year ago

This is your first time submitting an issue with UVtools 🥳Please review your issue and ensure that the submit template was followed, the information is complete, and not related to any other open issue. It will be reviewed shortly. Debugging is very important and make the program better. Thanks for contributing and making the software better! 🙌

alfonslm commented 1 year ago

What I would like to see as an option to avoid this would be to allow the use of a cash to stop the program from crashing especially if it's just for 200mb. The second thing could be to give a warning that the system is running low on ram since it seems like at least in this case that the ram usage increase slightly later in the process and that might be enough to avoid a crash.

An easy thing to start with might be to include a button to enable file compression if an out of memory happens.

sn4k3 commented 1 year ago

Unless you have SWAP disabled / capped, windows manage this quite well, however, is not guarantee that SWAP solve the lack of RAM. Your bottleneck is that you have 40 threads, but only 32GB RAM with half of that usable. By default, UVtools use all power you have and don't care about the core/ram ratio, so it will use +/-40 threads in parallel, that are decompressions/compressions/operations of large bitmaps and take ton of RAM. So, the more cores it uses more RAM you need to sustain that.

To cap that go to Settings and set the "degree of paralelism" under Tasks to a lower value, like 3/4 or ! Other option would be investing in more RAM, but set the degree to a lower value won't kill much of the performance.

alfonslm commented 1 year ago

That lowered the usage with about 10gb. But getting an error message with that information could be helpful.