Open gingerbraden opened 1 year ago
It's not necessary, but it's recommended with PointerScope yes: http://bytedeco.org/news/2018/07/17/bytedeco-as-distribution/
AndroidFrameConverter.gray2rgba() should allocate memory through JavaCPP to avoid that error though... Contributions are welcome!
After wrapping the function like this:
PointerScope().use { scope -> return converterToBitmap.convert(converterToMat.convert(byteMap)) }
I am also getting these lines of errors:
E/cv::error(): OpenCV(4.6.0) Error: Requested object was not found (could not open directory: /data/app/~~mkI7ZXIjkcXrkLe2ozCALg==/com.example.testingsaliency5-xD2aRrfEAJXDI_AdGJeCoA==/base.apk!/lib/arm64-v8a) in glob_rec, file /__w/javacpp-presets/javacpp-presets/opencv/cppbuild/android-arm64/opencv-4.6.0/modules/core/src/glob.cpp, line 279
E/linker: normalize_path - invalid input: "lib/", the input path should be absolute
W/linker: Warning: unable to normalize "lib/" (ignoring)
E/linker: normalize_path - invalid input: "lib/", the input path should be absolute
W/linker: Warning: unable to normalize "lib/" (ignoring)
W/System.err: Warning: Version of org.bytedeco:openblas could not be found.
W/System.err: Warning: Version of org.bytedeco:opencv could not be found.
with many repeating copies of
E/BLASTBufferQueue: [VRI[MainActivity]#0](f:0,a:3) Faking releaseBufferCallback from transactionCompleteCallback
and finally the app crashes on the same error as above.
We have to use PointerScope where memory gets allocated with JavaCPP. I told you the problem here is that AndroidFrameConverter.gray2rgba() does not currently use JavaCPP to allocate memory, so PointerScope cannot have any effect there, that's normal.
Oh, sorry I understood
AndroidFrameConverter.gray2rgba() should allocate memory through JavaCPP to avoid that error though...
as that gray2rgba does allocate memory
Try to use PointerScope in other places where you use JavaCV. That should work around the issue at least.
I added PointerScope to all uses of JavaCV like so:
val converterToBitmap = AndroidFrameConverter()
val converterToMat = ToMat()
var frame = Frame();
PointerScope().use { scope -> frame = converterToBitmap.convert(bitmap) }
Log.d("saliency", "1")
val saliencyMap = Mat()
val byteMap = Mat()
val sal = StaticSaliencyFineGrained()
PointerScope().use { scope -> sal.computeSaliency(converterToMat.convert(frame), saliencyMap) }
Log.d("saliency", "2")
PointerScope().use { scope -> sal.computeBinaryMap(saliencyMap, byteMap) }
Log.d("saliency", "3")
PointerScope().use { scope -> return converterToBitmap.convert(converterToMat.convert(byteMap)) }
And it didn't help. I managed to get this funny error:
CPU usage from 0ms to 9543ms later (2023-01-23 10:35:02.363 to 2023-01-23 10:35:11.906) with 99% awake:
177% 12913/com.example.testingsaliency5: 176% user + 0.7% kernel / faults: 4922 minor 77 major
But this is the usueal log since start of the app to its crash. Interesting part is that can't find some objects in javacpp-presets, even though I have them in my dependencies.
2023-01-23 10:54:23.369 20734-20754 AdrenoUtils com.example.testingsaliency5 W <ReadGpuID_from_sysfs:197>: Failed to open /sys/class/kgsl/kgsl-3d0/gpu_model
2023-01-23 10:54:23.369 20734-20754 AdrenoUtils com.example.testingsaliency5 W <ReadGpuID:221>: Failed to read chip ID from gpu_model. Fallback to use the GSL path
2023-01-23 10:54:23.380 20734-20754 OpenGLRenderer com.example.testingsaliency5 E Unable to match the desired swap behavior.
2023-01-23 10:54:23.415 20734-20754 Parcel com.example.testingsaliency5 W Expecting binder but got null!
2023-01-23 10:54:29.842 20734-20746 System com.example.testingsaliency5 W A resource failed to call close.
2023-01-23 10:54:30.894 20734-20734 linker com.example.testingsaliency5 E normalize_path - invalid input: "lib/", the input path should be absolute
2023-01-23 10:54:30.894 20734-20734 linker com.example.testingsaliency5 W Warning: unable to normalize "lib/" (ignoring)
2023-01-23 10:54:30.894 20734-20734 linker com.example.testingsaliency5 E normalize_path - invalid input: "lib/", the input path should be absolute
2023-01-23 10:54:30.894 20734-20734 linker com.example.testingsaliency5 W Warning: unable to normalize "lib/" (ignoring)
2023-01-23 10:54:30.908 20734-20734 System.err com.example.testingsaliency5 W Warning: Version of org.bytedeco:openblas could not be found.
2023-01-23 10:54:30.911 20734-20734 System.err com.example.testingsaliency5 W Warning: Version of org.bytedeco:opencv could not be found.
2023-01-23 10:54:31.397 20734-20734 saliency com.example.testingsaliency5 D 1
2023-01-23 10:54:32.109 20734-20734 cv::error() com.example.testingsaliency5 E OpenCV(4.6.0) Error: Requested object was not found (could not open directory: /data/app/~~mkI7ZXIjkcXrkLe2ozCALg==/com.example.testingsaliency5-xD2aRrfEAJXDI_AdGJeCoA==/base.apk!/lib/arm64-v8a) in glob_rec, file /__w/javacpp-presets/javacpp-presets/opencv/cppbuild/android-arm64/opencv-4.6.0/modules/core/src/glob.cpp, line 279
2023-01-23 10:54:32.130 20734-20734 cv::error() com.example.testingsaliency5 E OpenCV(4.6.0) Error: Requested object was not found (could not open directory: /data/app/~~mkI7ZXIjkcXrkLe2ozCALg==/com.example.testingsaliency5-xD2aRrfEAJXDI_AdGJeCoA==/base.apk!/lib/arm64-v8a) in glob_rec, file /__w/javacpp-presets/javacpp-presets/opencv/cppbuild/android-arm64/opencv-4.6.0/modules/core/src/glob.cpp, line 279
2023-01-23 10:54:32.131 20734-20734 cv::error() com.example.testingsaliency5 E OpenCV(4.6.0) Error: Requested object was not found (could not open directory: /data/app/~~mkI7ZXIjkcXrkLe2ozCALg==/com.example.testingsaliency5-xD2aRrfEAJXDI_AdGJeCoA==/base.apk!/lib/arm64-v8a) in glob_rec, file /__w/javacpp-presets/javacpp-presets/opencv/cppbuild/android-arm64/opencv-4.6.0/modules/core/src/glob.cpp, line 279
2023-01-23 10:54:52.409 20734-20734 saliency com.example.testingsaliency5 D 2
2023-01-23 10:55:19.763 20734-20734 saliency com.example.testingsaliency5 D 3
2023-01-23 10:55:19.778 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.778 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.797 20734-20734 estingsaliency5 com.example.testingsaliency5 I Alloc young concurrent copying GC freed 192328(12MB) AllocSpace objects, 0(0B) LOS objects, 86% free, 3833KB/27MB, paused 28us,28us total 18.992ms
2023-01-23 10:55:19.797 20734-20734 estingsaliency5 com.example.testingsaliency5 I Forcing collection of SoftReferences for 254MB allocation
2023-01-23 10:55:19.797 20734-20744 estingsaliency5 com.example.testingsaliency5 I WaitForGcToComplete blocked NativeAlloc on Alloc for 19.206ms
2023-01-23 10:55:19.797 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.808 20734-20734 estingsaliency5 com.example.testingsaliency5 I Alloc concurrent copying GC freed 1859(61KB) AllocSpace objects, 0(0B) LOS objects, 86% free, 3772KB/27MB, paused 21us,894us total 10.880ms
2023-01-23 10:55:19.809 20734-20734 estingsaliency5 com.example.testingsaliency5 W Throwing OutOfMemoryError "Failed to allocate a 266501312 byte allocation with 25165824 free bytes and 252MB until OOM, target footprint 29028528, growth limit 268435456" (VmSize 17062520 kB)
2023-01-23 10:55:19.809 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.809 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.813 20734-20734 estingsaliency5 com.example.testingsaliency5 I Forcing collection of SoftReferences for 254MB allocation
2023-01-23 10:55:19.813 20734-20734 estingsaliency5 com.example.testingsaliency5 I Starting a blocking GC Alloc
2023-01-23 10:55:19.821 20734-20734 estingsaliency5 com.example.testingsaliency5 I Alloc concurrent copying GC freed 2037(79KB) AllocSpace objects, 0(0B) LOS objects, 86% free, 3708KB/27MB, paused 14us,15us total 7.827ms
2023-01-23 10:55:19.821 20734-20734 estingsaliency5 com.example.testingsaliency5 W Throwing OutOfMemoryError "Failed to allocate a 266501312 byte allocation with 25165824 free bytes and 252MB until OOM, target footprint 28963440, growth limit 268435456" (VmSize 17062520 kB)
2023-01-23 10:55:19.821 20734-20734 AndroidRuntime com.example.testingsaliency5 D Shutting down VM
2023-01-23 10:55:19.835 20734-20734 AndroidRuntime com.example.testingsaliency5 E FATAL EXCEPTION: main
Process: com.example.testingsaliency5, PID: 20734
java.lang.OutOfMemoryError: Failed to allocate a 266501312 byte allocation with 25165824 free bytes and 252MB until OOM, target footprint 28963440, growth limit 268435456
If that code for converterToBitmap is the only place where you're using JavaCV, try to use a single PointerScope around all that.
I'm building an app in Kotlin that heavily depends on image processing through saliency.
My code looks like this:
But when ran, after I press the button, I get this error on line
return converterToBitmap.convert(converterToMat.convert(byteMap))
:The quesiton is, do I need to manage memory in some way, when working with JavaCV on Android?