Closed MightyBOBcnc closed 3 years ago
I am evaluating Meshroom and AliceVision and stumbled upon this showstopper at the start.
Meshroom version: 2019.2 OS: Ubuntu 18.04 x86_64 Nvidia driver: 418.78 Nvidia hardware: GeForce RTX 2080 Ti CUDA version: 10.1
$ nvidia-smi
Fri Aug 16 14:44:48 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.87.00 Driver Version: 418.87.00 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 208... On | 00000000:04:00.0 On | N/A |
| 0% 49C P8 24W / 300W | 558MiB / 10981MiB | 15% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 2695 G /usr/lib/xorg/Xorg 426MiB |
...
+-----------------------------------------------------------------------------+
Program called with the following parameters:
* downscale = 2
* exportIntermediateResults = 0
* imagesFolder = "/home/norru/Projects/ivt/meshroom_test/MeshroomCache/PrepareDenseScene/83b2802cbb853d07b8c1ceafa6b660cc88394795"
* input = "/home/norru/Projects/ivt/meshroom_test/MeshroomCache/StructureFromMotion/cb48e0481690b2bac4d0c797a55bf6831c701f59/sfm.abc"
* maxViewAngle = 70
* minViewAngle = 2
* nbGPUs = 0
* output = "/home/norru/Projects/ivt/meshroom_test/MeshroomCache/DepthMap/cd9460337ef57214ceb1104a65d6a25fa4933b2f"
* rangeSize = 3
* rangeStart = 0
* refineGammaC = 15.5
* refineGammaP = 8
* refineMaxTCams = 6
* refineNDepthsToRefine = 31
* refineNSamplesHalf = 150
* refineNiters = 100
* refineSigma = 15
* refineUseTcOrRcPixSize = 0
* refineWSH = 3
* sgmGammaC = 5.5
* sgmGammaP = 8
* sgmMaxTCams = 10
* sgmWSH = 4
* verboseLevel = "info"
[14:38:05.437844][warning] Cannot get available memory information for CUDA gpu device 0.
[14:38:04.512995][warning] CUDA-Enabled GPU.
Device information:
- id: 0
- name: GeForce RTX 2080 Ti
- compute capability: 7.5
- total device memory: 10981 MB
- device memory available: 0 MB
- per-block shared memory: 49152
- warp size: 32
- max threads per block: 1024
- max threads per SM(X): 1024
- max block sizes: {1024,1024,64}
- max grid sizes: {2147483647,65535,65535}
- max 2D array texture: {131072,65536}
- max 3D array texture: {16384,16384,16384}
- max 2D linear texture: {131072,65000,2097120}
- max 2D layered texture: {32768,32768,2048}
- number of SM(x)s: 68
- registers per SM(x): 65536
- registers per block: 65536
- concurrent kernels: yes
- mapping host memory: yes
- unified addressing: yes
- texture alignment: 512 byte
- pitch alignment: 32 byte
[14:38:05.438540][info] Supported CUDA-Enabled GPU detected.
[14:38:05.443070][info] Found 1 image dimension(s):
[14:38:05.443097][info] - [5472x3648]
[14:38:05.463809][info] Overall maximum dimension: [5472x3648]
[14:38:05.463898][info] Create depth maps.
CUDAError: unknown error
file: /opt/AliceVision_git/src/aliceVision/depthMap/cuda/planeSweeping/plane_sweeping_cuda.cu
function: ps_listCUDADevices
line: 205
[14:38:05.464366][info] # GPU devices: 1, # CPU threads: 24
[14:38:05.464401][info] Plane sweeping parameters:
- scale: 2
- step: 4
[14:38:05.464442][info] PlaneSweepingCuda:
- _nImgsInGPUAtTime: 2
- scales: 2
- subPixel: Yes
- varianceWSH: 4
terminate called after throwing an instance of 'std::runtime_error'
what(): Device alloc 2D array failed: unknown error
Aborted (core dumped)
@norru Please open a dedicated issue, as you pbl is specific to the 2019.2 release binaries on linux. So I will remove your comment from this PR about windows.
@MightyBOBcnc : Could you set the verbose to "trace" and send the log with the 2019.2 release? Is it the same error with 2019.1 and 2019.2?
@fabiencastan The log I included in the issue report is already set to trace. That's the full trace log for the DepthMap step in 2019.2.
The last line in the log is:
[16:22:33.796523][debug] copy image from disk to GPU Elapsed time: 00 minutes 00 seconds 526 miliseconds
and then the DepthMap fails silently <5 seconds after that without writing anything else to the log.
I have taken a look in the Windows Event Viewer and I have found that aliceVision_depthMapEstimation.exe appears to be crashing. There is 1 "Error" level events and 1 "Information" level events that appear to be connected to the crash:
Faulting application name: aliceVision_depthMapEstimation.exe, version: 2.0.0.0, time stamp: 0x5d4c28ec
Faulting module name: ucrtbase.dll, version: 10.0.18362.267, time stamp: 0x080a13f7
Exception code: 0xc0000409
Fault offset: 0x000000000006d33e
Faulting process id: 0x2994
Faulting application start time: 0x01d55455fac373a5
Faulting application path: C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_depthMapEstimation.exe
Faulting module path: C:\WINDOWS\System32\ucrtbase.dll
Report Id: 647ba96f-9a0c-41dc-9d60-1d8bb34a6c55
Faulting package full name:
Faulting package-relative application ID:
and
Fault bucket 2130295549337383112, type 5
Event Name: BEX64
Response: Not available
Cab Id: 0
Problem signature:
P1: aliceVision_depthMapEstimation.exe
P2: 2.0.0.0
P3: 5d4c28ec
P4: ucrtbase.dll
P5: 10.0.18362.267
P6: 080a13f7
P7: 000000000006d33e
P8: c0000409
P9: 0000000000000007
P10:
Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERD183.tmp.dmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERD2BD.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERD2ED.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERD2ED.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WERD30D.tmp.txt
These files may be available here:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_aliceVision_dept_5d7d516957a4f08eb65d9541212c5f4bc291bae0_58acabf4_910262ed-c0de-4365-b694-2de1727bcb6f
Analysis symbol:
Rechecking for solution: 0
Report Id: 647ba96f-9a0c-41dc-9d60-1d8bb34a6c55
Report Status: 268435456
Hashed bucket: 70d6fe45de100cb98d905483ecd9b0c8
Cab Guid: 0
Here is the associated Windows error report:
Version=1
EventType=BEX64
EventTime=132104492504375502
ReportType=2
Consent=1
UploadTime=132104492508719711
ReportStatus=268435456
ReportIdentifier=910262ed-c0de-4365-b694-2de1727bcb6f
IntegratorReportIdentifier=647ba96f-9a0c-41dc-9d60-1d8bb34a6c55
Wow64Host=34404
NsAppName=aliceVision_depthMapEstimation.exe
AppSessionGuid=00002994-0001-001b-a573-c3fa5554d501
TargetAppId=W:0006a1becd8e4b1a401d3fe3b618506c34f200000904!0000ca899cbbfc87b878012355e3a6c0ae8102c9cc3d!aliceVision_depthMapEstimation.exe
TargetAppVer=2019//08//08:13:51:40!0!aliceVision_depthMapEstimation.exe
BootId=4294967295
ServiceSplit=3720940304
TargetAsId=253
IsFatal=1
EtwNonCollectReason=1
Response.BucketId=70d6fe45de100cb98d905483ecd9b0c8
Response.BucketTable=5
Response.LegacyBucketId=2130295549337383112
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=aliceVision_depthMapEstimation.exe
Sig[1].Name=Application Version
Sig[1].Value=2.0.0.0
Sig[2].Name=Application Timestamp
Sig[2].Value=5d4c28ec
Sig[3].Name=Fault Module Name
Sig[3].Value=ucrtbase.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=10.0.18362.267
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=080a13f7
Sig[6].Name=Exception Offset
Sig[6].Value=000000000006d33e
Sig[7].Name=Exception Code
Sig[7].Value=c0000409
Sig[8].Name=Exception Data
Sig[8].Value=0000000000000007
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=10.0.18362.2.0.0.768.101
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=7785
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=77857e0ccf5abe9e7c6d80eb88fb7ebb
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=33e6
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=33e61f5127b8023b9f017798aa17382e
UI[2]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_depthMapEstimation.exe
LoadedModule[0]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_depthMapEstimation.exe
LoadedModule[1]=C:\WINDOWS\SYSTEM32\ntdll.dll
LoadedModule[2]=C:\WINDOWS\System32\KERNEL32.DLL
LoadedModule[3]=C:\WINDOWS\System32\KERNELBASE.dll
LoadedModule[4]=C:\WINDOWS\System32\ucrtbase.dll
LoadedModule[5]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_mvsUtils.dll
LoadedModule[6]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_program_options-vc140-mt-x64-1_70.dll
LoadedModule[7]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_sfmDataIO.dll
LoadedModule[8]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_depthMap.dll
LoadedModule[9]=C:\WINDOWS\SYSTEM32\MSVCP140.dll
LoadedModule[10]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_system.dll
LoadedModule[11]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_gpu.dll
LoadedModule[12]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_log-vc140-mt-x64-1_70.dll
LoadedModule[13]=C:\WINDOWS\System32\ADVAPI32.dll
LoadedModule[14]=C:\WINDOWS\System32\WS2_32.dll
LoadedModule[15]=C:\WINDOWS\System32\msvcrt.dll
LoadedModule[16]=C:\WINDOWS\System32\sechost.dll
LoadedModule[17]=C:\WINDOWS\System32\RPCRT4.dll
LoadedModule[18]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\VCRUNTIME140.dll
LoadedModule[19]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_filesystem-vc140-mt-x64-1_70.dll
LoadedModule[20]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\OpenImageIO.dll
LoadedModule[21]=C:\WINDOWS\System32\SHELL32.dll
LoadedModule[22]=C:\WINDOWS\System32\cfgmgr32.dll
LoadedModule[23]=C:\WINDOWS\System32\shcore.dll
LoadedModule[24]=C:\WINDOWS\System32\combase.dll
LoadedModule[25]=C:\WINDOWS\System32\bcryptPrimitives.dll
LoadedModule[26]=C:\WINDOWS\System32\windows.storage.dll
LoadedModule[27]=C:\WINDOWS\System32\msvcp_win.dll
LoadedModule[28]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_mvsData.dll
LoadedModule[29]=C:\WINDOWS\System32\profapi.dll
LoadedModule[30]=C:\WINDOWS\System32\powrprof.dll
LoadedModule[31]=C:\WINDOWS\System32\UMPDC.dll
LoadedModule[32]=C:\WINDOWS\System32\shlwapi.dll
LoadedModule[33]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\zlib1.dll
LoadedModule[34]=C:\WINDOWS\System32\GDI32.dll
LoadedModule[35]=C:\WINDOWS\System32\win32u.dll
LoadedModule[36]=C:\WINDOWS\System32\gdi32full.dll
LoadedModule[37]=C:\WINDOWS\System32\USER32.dll
LoadedModule[38]=C:\WINDOWS\System32\kernel.appcore.dll
LoadedModule[39]=C:\WINDOWS\System32\cryptsp.dll
LoadedModule[40]=C:\WINDOWS\System32\ole32.dll
LoadedModule[41]=C:\WINDOWS\SYSTEM32\VCOMP140.DLL
LoadedModule[42]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\Alembic.dll
LoadedModule[43]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\cudart64_80.dll
LoadedModule[44]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_sfmData.dll
LoadedModule[45]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_regex-vc140-mt-x64-1_70.dll
LoadedModule[46]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_thread-vc140-mt-x64-1_70.dll
LoadedModule[47]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_multiview.dll
LoadedModule[48]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_date_time-vc140-mt-x64-1_70.dll
LoadedModule[49]=C:\WINDOWS\SYSTEM32\Secur32.dll
LoadedModule[50]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_feature.dll
LoadedModule[51]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\IlmImf-2_3.dll
LoadedModule[52]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\Imath-2_3.dll
LoadedModule[53]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_image.dll
LoadedModule[54]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\Iex-2_3.dll
LoadedModule[55]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\Half-2_3.dll
LoadedModule[56]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\libpng16.dll
LoadedModule[57]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\jpeg62.dll
LoadedModule[58]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\raw_r.dll
LoadedModule[59]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\tiff.dll
LoadedModule[60]=C:\WINDOWS\SYSTEM32\dbgeng.dll
LoadedModule[61]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\hdf5.dll
LoadedModule[62]=C:\WINDOWS\System32\bcrypt.dll
LoadedModule[63]=C:\WINDOWS\System32\OLEAUT32.dll
LoadedModule[64]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\IlmThread-2_3.dll
LoadedModule[65]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_numeric.dll
LoadedModule[66]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\glog.dll
LoadedModule[67]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\ceres.dll
LoadedModule[68]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\CCTag.dll
LoadedModule[69]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\lcms2.dll
LoadedModule[70]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\jasper.dll
LoadedModule[71]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\opencv_features2d343.dll
LoadedModule[72]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\opencv_core343.dll
LoadedModule[73]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\lzma.dll
LoadedModule[74]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\popsift.dll
LoadedModule[75]=C:\WINDOWS\SYSTEM32\dbgmodel.dll
LoadedModule[76]=C:\WINDOWS\SYSTEM32\XmlLite.dll
LoadedModule[77]=C:\WINDOWS\SYSTEM32\SSPICLI.DLL
LoadedModule[78]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\gflags.dll
LoadedModule[79]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\openblas.dll
LoadedModule[80]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\tbb.dll
LoadedModule[81]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\boost_serialization-vc140-mt-x64-1_70.dll
LoadedModule[82]=C:\WINDOWS\SYSTEM32\OPENGL32.dll
LoadedModule[83]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\CONCRT140.dll
LoadedModule[84]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\opencv_imgproc343.dll
LoadedModule[85]=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\opencv_flann343.dll
LoadedModule[86]=C:\WINDOWS\SYSTEM32\GLU32.dll
LoadedModule[87]=C:\WINDOWS\SYSTEM32\dxcore.dll
LoadedModule[88]=C:\WINDOWS\SYSTEM32\dbghelp.dll
LoadedModule[89]=C:\WINDOWS\System32\IMM32.DLL
LoadedModule[90]=C:\Windows\system32\guard64.dll
LoadedModule[91]=C:\WINDOWS\system32\version.dll
LoadedModule[92]=C:\WINDOWS\system32\fltlib.dll
LoadedModule[93]=C:\WINDOWS\system32\nvcuda.dll
LoadedModule[94]=C:\WINDOWS\System32\SETUPAPI.dll
LoadedModule[95]=C:\WINDOWS\SYSTEM32\nvfatbinaryLoader.dll
LoadedModule[96]=C:\WINDOWS\system32\uxtheme.dll
LoadedModule[97]=C:\WINDOWS\system32\nvapi64.dll
LoadedModule[98]=C:\WINDOWS\SYSTEM32\dxgi.dll
LoadedModule[99]=C:\WINDOWS\SYSTEM32\DEVOBJ.dll
LoadedModule[100]=C:\WINDOWS\System32\WINTRUST.dll
LoadedModule[101]=C:\WINDOWS\System32\MSASN1.dll
LoadedModule[102]=C:\WINDOWS\System32\CRYPT32.dll
State[0].Key=Transport.DoneStage1
State[0].Value=1
OsInfo[0].Key=vermaj
OsInfo[0].Value=10
OsInfo[1].Key=vermin
OsInfo[1].Value=0
OsInfo[2].Key=verbld
OsInfo[2].Value=18362
OsInfo[3].Key=ubr
OsInfo[3].Value=295
OsInfo[4].Key=versp
OsInfo[4].Value=0
OsInfo[5].Key=arch
OsInfo[5].Value=9
OsInfo[6].Key=lcid
OsInfo[6].Value=1033
OsInfo[7].Key=geoid
OsInfo[7].Value=244
OsInfo[8].Key=sku
OsInfo[8].Value=101
OsInfo[9].Key=domain
OsInfo[9].Value=0
OsInfo[10].Key=prodsuite
OsInfo[10].Value=768
OsInfo[11].Key=ntprodtype
OsInfo[11].Value=1
OsInfo[12].Key=platid
OsInfo[12].Value=10
OsInfo[13].Key=sr
OsInfo[13].Value=0
OsInfo[14].Key=tmsi
OsInfo[14].Value=84246
OsInfo[15].Key=osinsty
OsInfo[15].Value=3
OsInfo[16].Key=iever
OsInfo[16].Value=11.295.18362.0-11.0.140
OsInfo[17].Key=portos
OsInfo[17].Value=0
OsInfo[18].Key=ram
OsInfo[18].Value=32676
OsInfo[19].Key=svolsz
OsInfo[19].Value=117
OsInfo[20].Key=wimbt
OsInfo[20].Value=0
OsInfo[21].Key=blddt
OsInfo[21].Value=190318
OsInfo[22].Key=bldtm
OsInfo[22].Value=1202
OsInfo[23].Key=bldbrch
OsInfo[23].Value=19h1_release
OsInfo[24].Key=bldchk
OsInfo[24].Value=0
OsInfo[25].Key=wpvermaj
OsInfo[25].Value=0
OsInfo[26].Key=wpvermin
OsInfo[26].Value=0
OsInfo[27].Key=wpbuildmaj
OsInfo[27].Value=0
OsInfo[28].Key=wpbuildmin
OsInfo[28].Value=0
OsInfo[29].Key=osver
OsInfo[29].Value=10.0.18362.295.amd64fre.19h1_release.190318-1202
OsInfo[30].Key=buildflightid
OsInfo[30].Value=E61D6EF4-31F5-4EFA-8D73-A62AB2676F2F.1
OsInfo[31].Key=edition
OsInfo[31].Value=Core
OsInfo[32].Key=ring
OsInfo[32].Value=Retail
OsInfo[33].Key=expid
OsInfo[33].Value=RS:379B
OsInfo[34].Key=containerid
OsInfo[35].Key=containertype
OsInfo[36].Key=edu
OsInfo[36].Value=0
FriendlyEventName=Stopped working
ConsentKey=BEX64
AppName=aliceVision_depthMapEstimation - AliceVision 2.2.0
AppPath=C:\Portable\Meshroom-2019.2.0\aliceVision\bin\aliceVision_depthMapEstimation.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=1700A2128AB29719FDC496CB94DD0F98
MetadataHash=-173061553
The same thing happens in release 2019.1, which is to say that aliceVision_depthMapEstimation.exe appears to be crashing and in the Windows Event Viewer there is a crash that implicates ucrtbase.dll
I am searching Google for errors related to ucrtbase.dll and I have found this so far: https://superuser.com/questions/1397010/application-crash-due-to-ucrtbase-dll
@fabiencastan another update: I created a dump of the crash. I can send it to you if you would like. It is 38MB.
I ran the dump through the Debugging Tools for Windows (WinDbg). This is the analysis:
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(2994.2aa0): Security check failure or stack buffer overrun - code c0000409 (first/second chance not available)
For analysis of this file, run !analyze -v
ucrtbase!abort+0x4e:
00007ffc`8b35d33e cd29 int 29h
0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
*** WARNING: Unable to verify timestamp for aliceVision_depthMapEstimation.exe
*** WARNING: Unable to verify timestamp for openblas.dll
*** WARNING: Unable to verify timestamp for nvcuda.dll
*** WARNING: Unable to verify timestamp for IlmThread-2_3.dll
*** WARNING: Unable to verify timestamp for OpenImageIO.dll
KEY_VALUES_STRING: 1
Key : Timeline.Process.Start.DeltaSec
Value: 19
PROCESSES_ANALYSIS: 1
SERVICE_ANALYSIS: 1
STACKHASH_ANALYSIS: 1
TIMELINE_ANALYSIS: 1
Timeline: !analyze.Start
Name: <blank>
Time: 2019-08-16T18:21:26.155Z
Diff: 4034155 mSec
Timeline: Dump.Current
Name: <blank>
Time: 2019-08-16T17:14:12.0Z
Diff: 0 mSec
Timeline: Process.Start
Name: <blank>
Time: 2019-08-16T17:13:53.0Z
Diff: 19000 mSec
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
CONTEXT: (.ecxr)
rax=0000000000000001 rbx=0000000000000000 rcx=0000000000000007
rdx=000000000000000f rsi=00007ff6646f9bdc rdi=0000000000000000
rip=00007ffc8b35d33e rsp=000000bf162fcbc0 rbp=000000bf162ff9f0
r8=0000020d984a5690 r9=00007ffc8c061ec0 r10=0000000000000013
r11=0000020d984b5570 r12=000000bf162fcde0 r13=ffffffffffffffff
r14=0000000000000001 r15=00007ffc8c2ab790
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ucrtbase!abort+0x4e:
00007ffc`8b35d33e cd29 int 29h
Resetting default scope
FAULTING_IP:
ucrtbase!abort+4e
00007ffc`8b35d33e cd29 int 29h
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007ffc8b35d33e (ucrtbase!abort+0x000000000000004e)
ExceptionCode: c0000409 (Security check failure or stack buffer overrun)
ExceptionFlags: 00000001
NumberParameters: 1
Parameter[0]: 0000000000000007
Subcode: 0x7 FAST_FAIL_FATAL_APP_EXIT
PROCESS_NAME: aliceVision_depthMapEstimation.exe
ERROR_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.
EXCEPTION_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application.
EXCEPTION_CODE_STR: c0000409
EXCEPTION_PARAMETER1: 0000000000000007
WATSON_BKT_PROCSTAMP: 5d39f0fb
WATSON_BKT_PROCVER: 2019.0.0.0
WATSON_BKT_MODULE: ucrtbase.dll
WATSON_BKT_MODSTAMP: 80a13f7
WATSON_BKT_MODOFFSET: 6d33e
WATSON_BKT_MODVER: 10.0.18362.267
MODULE_VER_PRODUCT: Microsoft® Windows® Operating System
BUILD_VERSION_STRING: 18362.1.amd64fre.19h1_release.190318-1202
MODLIST_WITH_TSCHKSUM_HASH: cd7534d90e6c9596ba6ec4f89f143d047823dd36
MODLIST_SHA1_HASH: 4084fb95910959220d2986ffb6338539ce7e59b4
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
DUMP_FLAGS: 94
DUMP_TYPE: 1
ANALYSIS_SESSION_HOST: ANTEC
ANALYSIS_SESSION_TIME: 08-16-2019 14:21:26.0155
ANALYSIS_VERSION: 10.0.18362.1 x86fre
THREAD_ATTRIBUTES:
OS_LOCALE: ENU
BUGCHECK_STR: FAIL_FAST_FATAL_APP_EXIT
DEFAULT_BUCKET_ID: FAIL_FAST_FATAL_APP_EXIT
PRIMARY_PROBLEM_CLASS: FAIL_FAST
PROBLEM_CLASSES:
ID: [0n282]
Type: [FAIL_FAST]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
ID: [0n271]
Type: [FATAL_APP_EXIT]
Class: Addendum
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Add
Data: Omit
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
LAST_CONTROL_TRANSFER: from 00007ffc8b35ca7f to 00007ffc8b35d33e
STACK_TEXT:
000000bf`162fcbc0 00007ffc`8b35ca7f : 00000000`00000003 00000000`00000003 00000000`00000000 00007ff6`646f9000 : ucrtbase!abort+0x4e
000000bf`162fcbf0 00007ff6`646f9c13 : 00000000`00003000 00000020`00001000 00000000`01000000 00000000`00000000 : ucrtbase!terminate+0x1f
000000bf`162fcc20 00000000`00003000 : 00000020`00001000 00000000`01000000 00000000`00000000 00000000`00000000 : aliceVision_depthMapEstimation+0x19c13
000000bf`162fcc28 00000020`00001000 : 00000000`01000000 00000000`00000000 00000000`00000000 00007ffc`8c13fbca : 0x3000
000000bf`162fcc30 00000000`01000000 : 00000000`00000000 00000000`00000000 00007ffc`8c13fbca 00000000`00000000 : 0x00000020`00001000
000000bf`162fcc38 00000000`00000000 : 00000000`00000000 00007ffc`8c13fbca 00000000`00000000 00007ff6`646f9bdc : 0x1000000
THREAD_SHA1_HASH_MOD_FUNC: 041c5cbc22cc630155d0bd903e2c80ce982778c8
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 855a67a898f044d235c8525585417dfef9c5b1c3
THREAD_SHA1_HASH_MOD: 4ff6ac4a28a0b53422e8962b825743ee47a186e0
FOLLOWUP_IP:
ucrtbase!abort+4e
00007ffc`8b35d33e cd29 int 29h
FAULT_INSTR_CODE: 15ba29cd
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: ucrtbase!abort+4e
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: ucrtbase
IMAGE_NAME: ucrtbase.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 80a13f7
STACK_COMMAND: ~0s ; .ecxr ; kb
BUCKET_ID: FAIL_FAST_FATAL_APP_EXIT_ucrtbase!abort+4e
FAILURE_EXCEPTION_CODE: c0000409
FAILURE_IMAGE_NAME: ucrtbase.dll
BUCKET_ID_IMAGE_STR: ucrtbase.dll
FAILURE_MODULE_NAME: ucrtbase
BUCKET_ID_MODULE_STR: ucrtbase
FAILURE_FUNCTION_NAME: abort
BUCKET_ID_FUNCTION_STR: abort
BUCKET_ID_OFFSET: 4e
BUCKET_ID_MODTIMEDATESTAMP: 80a13f7
BUCKET_ID_MODCHECKSUM: ffdfb
BUCKET_ID_MODVER_STR: 10.0.18362.267
BUCKET_ID_PREFIX_STR: FAIL_FAST_FATAL_APP_EXIT_
FAILURE_PROBLEM_CLASS: FAIL_FAST
FAILURE_SYMBOL_NAME: ucrtbase.dll!abort
FAILURE_BUCKET_ID: FAIL_FAST_FATAL_APP_EXIT_c0000409_ucrtbase.dll!abort
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/aliceVision_depthMapEstimation.exe/2019.0.0.0/5d39f0fb/ucrtbase.dll/10.0.18362.267/ 80a13f7/c0000409/0006d33e.htm?Retriage=1
TARGET_TIME: 2019-08-16T17:14:12.000Z
OSBUILD: 18362
OSSERVICEPACK: 86
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
SUITE_MASK: 768
PRODUCT_TYPE: 1
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 WinNt SingleUserTS Personal
USER_LCID: 0
OSBUILD_TIMESTAMP: unknown_date
BUILDDATESTAMP_STR: 190318-1202
BUILDLAB_STR: 19h1_release
BUILDOSVER_STR: 10.0.18362.1.amd64fre.19h1_release.190318-1202
ANALYSIS_SESSION_ELAPSED_TIME: 4f9a
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:fail_fast_fatal_app_exit_c0000409_ucrtbase.dll!abort
FAILURE_ID_HASH: {e31753ac-c98a-8055-3663-47e707543d20}
Followup: MachineOwner
---------
What stands out to me is this:
EXCEPTION_CODE: (NTSTATUS) 0xc0000409 - The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of this application
Just a check: Did you make sure your Antivirus software does not interfere?
I have disabled the antivirus software (and also disabled its services in Services.msc) so I do not believe that it is interfering. The antivirus log files also do not contain any alerts for any Meshroom/Alicevision executables.
Just to add more data points to the discussion...
I also have a program termination "aliceVision_depthMapEstimation - AliceVision 2.1.0 has stopped working" for ver 2019.1 , and a program termination "aliceVision_depthMapEstimation - AliceVision 2.2.0 has stopped working" along with a kernel fault in the Nvidia display driver for ver 2019.2 . I have this error on two different machines, both with Nvidia GT730 graphics cards.
I am using the MonsterTree sample files. The 2018.1 version will finish the mesh correctly.
Meshroom works fine for me, but I also have this in my system log:
P1: aliceVision_incrementalSfM.exe P2: 2.1.0.0 P3: 5c61901e P4: ucrtbase.dll P5: 10.0.17134.677 P6: 9f346d3f P7: 000000000006e14e P8: c0000409 P9: 0000000000000007 P10:
same with
aliceVision_featureMatching.exe
aliceVision_depthMapFiltering.exe
This crash info is in my system log (event viewer), pointing to the Meshroom path of my 2019.1 installation. There are app crash reports for today even tough I did not start MR...
For aliceVision_depthMapEstimation.exe I found this entry from an actual MR 2019.2 crash two days ago:
Event name: RADAR_PRE_LEAK_64
with no other useful information.
I forgot to add.. I'm using Win 7 64 (all current updates), and my Nvidia driver is 431.60 (latest), DirectX 11 The first machine is FX-8350 with 8GB memory, the second is Athlon 7450 with 2GB memory.
I run into problems with meshroom … if i save the project in my (user) documents Folder ... but if i save the project on my desktop or on any other drive or folder it works fine.
@voto1979 Do you have special characters/non-ASCII characters in your file paths (#209)? Than this is the cause and not related to this specific issue.
@natowi … no I use a "mklink" for my user documents folder!
C:\Users\Name\Documents\
=
D:\Documents\
(C:\Users\Name\Documents) is linked to (D:\Documents). If I open the project over (C:\Users\Name\Documents\ it stops on the "StructureFromMotion" node and if I open it trough (D:\Documents) it works without any issues.
mklink /J "C:\Users\Name\Documents\" "D:\Documents\"
Sorry for confusing everybody!
This sounds specific to symbolic link junctions. Maybe reopen #599.
Update: The aliceVision_depthMapEstimation.exe crash occurs when downscale=1. I changed to downscale=2 and it completed the task but this is a severe hit to quality of the mesh that can be extracted in the subsequent meshing steps. By comparison, release 2018.1 does not crash with downscale=1.
Can confirm the windows event log error; for some projects downscale is > 1 works, for others we had to increase it to 4. Our last attempt fails at chunk 32 of 37 on Downscale = 2, while 4 works.
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2019-08-26T04:15:39.307564400Z" />
<EventRecordID>70692</EventRecordID>
<Channel>Application</Channel>
<Computer>DESKTOP-TR6MJER</Computer>
<Security />
</System>
<EventData>
<Data>aliceVision_depthMapEstimation.exe</Data>
<Data>2.0.0.0</Data>
<Data>5d4c28ec</Data>
<Data>ucrtbase.dll</Data>
<Data>10.0.17763.404</Data>
<Data>490b0aeb</Data>
<Data>c0000409</Data>
<Data>000000000006e91e</Data>
<Data>410</Data>
<Data>01d55bc4a031309d</Data>
<Data>D:\Mijn Downloads\Meshroom-2019.2.0-win64\aliceVision\bin\aliceVision_depthMapEstimation.exe</Data>
<Data>C:\WINDOWS\System32\ucrtbase.dll</Data>
<Data>63147727-4525-4377-a9ce-c4192c6823c1</Data>
<Data />
<Data />
</EventData>
</Event>
Can confirm the windows event log error; for some projects downscale is > 1 works, for others we had to increase it to 4.
This depends on the available memory. 8GB is the bare minimum for ~30-50 images (depending on resolution). 16gb are recommended for small scale reconstructions, 32gb+ for 100+ images. (all estimations)
If you don´t have enough memory, you need to increase the downscale factor accordingly.
I don't believe that system RAM is the bottleneck here. I have 32GB of RAM and aliceVision_depthMapEstimation.exe 2019.2 crashes on datasets of even only 40-50 images.
I've tested the Buddha and Monstree datasets and they both crash when downscale=1. Buddha crashes on the 19th chunk. Monstree crashes immediately on the 1st chunk.
2018.1 does not crash on any dataset that I throw at it including datasets that are 3x the size of Buddha and Monstree.
Same problem here, same photos in 2018.1 worked fine... updated to 2019.2 depthMap crashes. I tried all the suggested fixes but the only increasing downscale seems to do something. Doesn't seem a RAM problem since my ram was always at 50%/max 70% all the time with all downscales factors; including the fact that with 2018.1 no problems occurred.
I need to reproduce it to be able to fix it. But I never managed to reproduce a crash with the 2019.2 and I have tested lots of data sets. Could someone send me one or multiple data sets where you have the issue?
@fabiencastan You can download my data set (~912MB). However as I mentioned previously the Monstree data set and the Buddha data set also crash.
Curiously, though, the Buddha set does not crash until chunk 19 of the DepthMap stage and I'm struggling to understand why that would be the case. All of the images in the Buddha set have the exact same dimensions so I don't understand why it may crash on any individual image. If all of the images before chunk 19 work I would intuitively think that all the images should work all the way through. But I don't know how the underlying process works so all I can do is make wild non-programmer guesses (I'm just a nerdy artist) like if something is going out of bounds with the memory allocation or the garbage collector isn't deleting something.
I'm trying to uncover a pattern in the crashes but so far I don't see one. At first I thought it might be crashing if the images are larger than a certain number of megapixels but as we can see below the Buddha set crashes despite having a smaller number of megapixels than when my own data set has downscale=2 enabled, which does not crash.
My data set: 52 images | 6000x4000 | 24,000,000 pixels per image | Instant crash
Monstree set: 41 images | 3024x4032 | 12,192,768 pixels per image | Instant crash
My set downscale=2: 52 images | 3000x2000 | 6,000,000 pixels per image | No crash
Buddha set: 67 images | 2736x1540 | 4,213,440 pixels per image | Crash on chunk 19
Monstree downscale=2: 41 images | 1512x2016 | 3,048,192 pixels per image | No crash
I did some more googling about the specific errors involved (0xc0000409 STATUS_STACK_BUFFER_OVERRUN and 0x7 FAST_FAIL_FATAL_APP_EXIT) and came across these articles here and here. I don't know if that's helpful in any way.
From the above list, the interesting one for me is the FAST_FAIL_FATAL_APP_EXIT code. This code is used when the C runtime function abort() gets called. And abort() is called by std::terminate(). And std::terminate() is called automatically for things like throwing an exception out of a noexcept function, or if a thrown exception goes unhandled. It’s also used by some modules when an internal assertion failure is hit, or when an error occurs and the program simply wants to give up rather than try to recover. These are not necessarily security issues. It’s just the program saying, “Um, I’m in trouble. I think I’ll just stop right here.”
I also have the crashdump .dmp files available to share if those would be of any help.
Small update: I installed Linux Mint 19.2 on the same laptop where I was having the DepthMap crashes and did not experience a crash when using the linux version of Meshroom 2019.2.0. So it looks like the crash is likely exclusive to the Windows binaries.
@MightyBOBcnc Thanks for the feedback. BTW, for people who have the problem on Windows, it could make sense to try to update your drivers.
same issue for me , windows 10, crash at depthmap with low downscale 1 or 2 (needs to set at downscale 4 to avoid crash) with 2019 relases. 2018 version works ok with downscale1... Is the only solution is to install Inux?? thanks
same issue for me , windows 10 (32 Gb), K3100 (4Gb) crash at depthmap with low downscale 1 or 2 (needs to set at downscale 4 to avoid crash) with 2019 relases. Factory machine works well Xenon W-2123 (16 Gb), Driver 388.26, Quadro P620 (2 Gb)
Same issue for me aswell Very inconsiting DepthMap Crashes. At only one machine the other more less same spec don't has this Issue. Consistent Crashes at Downscale 1. Inconsitent on Downscale 2
Problem Machine (Laptop): Windows 10 64bit GTX 960M 4GB VRAM 16GB RAM Driver: 441.66 (newest)
Working Machine: Windows 10 64bit GTX 970 3,5/4GB VRAM 16GB RAM Driver: 441.08
No Hints for an out of memory case. RAM and VRAM don't go over 60%
Same Issue for me here Intel i7-5820k 64GB Ram Nvidia GeForce GTX 1060 3GB Nvidia GeForce GTX 750 Ti
Maybe 2018.1 used cpu to calculate depthmap so 64GB Ram was used? Now 2019.2 uses GPU fpr depthmap so it uses only 3GB from GTX 1060??
Can we set 2019.2 to use CPU instead of GPU aswell for testing purpose?
GPU has always been used to calculate depth maps, including in 2018.1. Something in the behavior changed between 2018.1 and 2019.1. Maybe related to AliceVision commits 671, 616, 642.
Some comments here sound very simmilar to a problem i had with the DepthMapNode (and the low performance of my Quadro K3000M). If you experience random crashes after around 2 seconds of freeze, it's very likely related to a Windows feature called Timeout Detection and Recovery (TDR). This feature completely resets the GPU after a 2 second GPU-Freeze causing Meshroom to crash with the message "... stopped working". I posted more details on issue #741 recently.
I highly recommend to raise the "TdrDelay" value to several seconds (maybe even 1 or 2 minutes) while trying to find the root cause of this bug to eliminate the possible interference with the TDR feature!
Oh gosh I had never considered that the Windows TDR may be interfering with Meshroom. I've heard of TDR before because Substance Painter explicitly calls it out as part of their recommended program setup and documentation. They have a good tutorial for changing it if anyone wants to give it a try: https://docs.substance3d.com/spdoc/gpu-drivers-crash-with-long-computations-128745489.html
I never heard of the TDR feature before i stumbled across bug report for Battlefield 5 lest week. I had problems with the DepthMap node since i started using MeshRoom in 2018. I couldn't go lower than downscale 4 and even that crashed sometimes. I blamed my poor K3000M and decided to wait using Meshroom for serious work until a buy a beefier machine. After raising the TdrDelay to 20sec a had no problems with downscale 2.
Maybe this should be mentioned in the MeshRoom FAQ?
Came here to figure out same issue. I did a side by side test of Monstree dataset in 2019.2.0 with all defaults I could not get past chunk 0 of DepthMap with Downscale=2, worked ok with Downscale=4. Under same conditions, 2018.1.0 would work with Downscale=2.
Interestingly I also have a GeForce GT 730 like another poster above and 16GB system memory Core i7-7700. Windows 10
Set registry values for TdrDelay and 2019.2 passes the first chunk now. Giving the registry settings here for others. Just copy paste into reg file and merge. Reboot required.
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers]
"TdrDelay"=dword:0000003c
"TdrDdiDelay"=dword:0000003c
Last few lines of log with is was 'crashing', it really just stops.
[19:20:06.455351][info] PlaneSweepingCuda:
- _nImgsInGPUAtTime: 2
- scales: 2
- subPixel: Yes
- varianceWSH: 4
CUDA device no 0 for 0
Device 0 memory - used: 361.649994, free: 1686.349976, total: 2048.000000
Device 0 memory - used: 479.774994, free: 1568.224976, total: 2048.000000
[19:20:17.295567][info] Estimate depth map, view id: 12251753
What is this ? I have a laptop with 8GB RAM. It's happen in the Depthmap.
and @davezap I don't understand where we have to do this.
@Theaudorant Your error log suggest that you do not have a nvidia GPU or your drivers are not up do date. The solution by davezap is for the case that depthmap fails during computation.
And if I don't a GPU, can I use Meshroom ?
@Theaudorant Read https://github.com/alicevision/meshroom/wiki/Draft-Meshing
ok thank you
but there is nothing
@Theaudorant The link works. Please navigate to https://github.com/alicevision/meshroom/wiki You can navigate to the page in the right hand side menu.
ok thank you I found
I've never worked with Meshroom before and just downloaded version 2019.2.0 from the official website. I was watching this tutorial while letting it run with 22 jpg files and when it got to about 15% of "DepthMap", it just stopped. Pressing "Start" again doesn't work, it just stops again. There's no error message in the log, it just stops:
CUDA device no 0 for 1 Device 1 memory - used: 202.800003, free: 821.200012, total: 1024.000000 Device 0 memory - used: 884.200012, free: 3211.800049, total: 4096.000000 [16:13:05.854836][info] Found only 7/10 nearest cameras for view id: 135287188 Device 1 memory - used: 355.799988, free: 668.200012, total: 1024.000000 [16:13:05.871827][info] Found only 8/10 nearest cameras for view id: 279736322 [16:13:33.674459][info] Estimate depth map, view id: 135287188 Device 0 memory - used: 987.700012, free: 3108.300049, total: 4096.000000 total size of volume map in GPU memory: 103.464844 nDepths 981, nDepthsToSearch 981 ps_computeSimilarityVolume elapsed time: 89.000000 ms [16:13:39.028682][info] Estimate depth map, view id: 279736322
My PC: Win 10, Xeon E3-1220V2, GTX 710/GTX 1050Ti (drivers from November 2019), 8gb of RAM. The ".mg" file and the images are in "C:\misc\3dstuff\scan". 10 of the images have a "Camera: Not constructed" symbol in the upper right corner but "StructureFromMotion" finished successfully, so not sure if I have to do aynthing about that.
Edit: I'm currently testing the same images in version 2018.1 and, even though now 12 images are marked as "not reconstructed", "DepthMap" is now already at 30% (working on chunk 2).
For me updating GPU drivers & increasing tdr delay solved the issue
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers] "TdrDelay"=dword:0000003c "TdrDdiDelay"=dword:0000003c
Same problem, DepthMap step fails. Win10 64bit 8GB RAM GeForce GT470M
Workaround #1 set downscale factor to 4 - confirmed to work. Workaround #2 modify registry - confirmed to work even if downscale set to 2 or 1 (also confirmed reboot is required)
Raising TDR values prevents the process from crashing, but the calculations never really finishes? Also, no calculations should make the GPU hang totally, should they?
Same issue with my NVIDIA GeForce GT 630m on Windows 10.
Works "fine" on Meshroom 2018.1 (but the mesh is totally a mess) but; it stops on Estimate depth map, view id
when running Meshroom 2019.2 GUI or CLI meshroom_photogrammetry.exe
(2019.2)
Maybe Meshroom 2019.2 is using CUDA toolkit > 8 which doesn't handle anymore CUDA 2.1 (GT 630 is CUDA 2.1)
log-error.txt
Not the same cuda dll are used :
Also when running aliceVision_depthMapEstimation
via command line; it goes further :
CUDA device no 0 for 0
Device 0 memory - used: 368.071869, free: 1679.928101, total: 2048.000000
Device 0 memory - used: 456.759369, free: 1591.240601, total: 2048.000000
[13:24:25.260260][info] Estimate depth map, view id: 105925454 (Remy: it normally stops here)
Device 0 memory - used: 495.009369, free: 1552.990601, total: 2048.000000
total size of volume map in GPU memory: 38.242188
nDepths 445, nDepthsToSearch 445
ps_computeSimilarityVolume elapsed time: 16.000000 ms
[EDIT] [SOLUTION] : Increase the TDR delay - it fixed the problem : works fine and texture mesh is good (tip found in Meshroom github ) - but seems slow as the DeshMap took 55min for that sample https://rcdata.capturingreality.com/testset-ga.zip
I confirm this solution for Nvidia GT 710 and Windows 7. Explicit setting of the TDR delay parameter helped to solve the problem (in my case, 8 seconds was enough). Everything is great now!
On my Lenovo W530 with a Nvidia Quadro K1000M card, everything stalls if I change the TDR setting to 60. The machine becomes unresponsive several seconds at the time, and then I can move the mouse for 2 seconds and it stalls again as long as the reconstruction runs - and after hours it has still not finished. If I run the same on a new Laptop with a Quadro RTX4000 it finishes in a few minutes. If the same reconstructions runs fine on the 2018 release, there must be something wrong going on in the new versions.
The registry edits worked for me, thanks, and my frustration with meshroom is replaced with enjoyment in getting some textured meshes! I think the developers should add a note in the FAQs about this - as it was far from obvious what was happening until I found this post.
Hey together. I had the same problem. So i testet it out. First Test: 4000x3000px Photos - The problem came up. Sec. Test: 1920x1080 px Photos. Result: No Problem.
Try it , maybe it works 4U 2 . Edit: I used only 12 Pics. With more i doesnt work. :-(
Many users, including myself, ran into an issue in 2019.1 where the DepthMap step would frequently fail, but would not have this issue in 2018.1. See #409
Unfortunately this continues to happen in 2019.2. I am running a set of 52 photos that works perfectly fine in 2018.1, but failed in 2019.1 and now fail 2019.2 as well. 52/52 photos have been successfully matched but the DepthMap step immediately fails with no error message.
I'm so sad that I'm stuck on 2018.1. :( Let me know if I can provide any additional information to help.