cmdbug / YOLOv5_NCNN

🍅 Deploy ncnn on mobile phones. Support Android and iOS. 移动端ncnn部署,支持Android与iOS。
GNU General Public License v3.0
1.48k stars 302 forks source link

关于yolact部署自己训练的模型 #92

Open walker-ai opened 2 years ago

walker-ai commented 2 years ago

我已经按照 nihui 的 知乎文章 成功将 onnx 转为 ncnnparambin,然后将其替换了 asset 下原始的 parambin,并且修改了 yolact.cppbox.javaYolactMask.java 里面的类别名,以及 nihui 文章中提到的自己的网络输出:

ex.extract("619", maskmaps);// 138x138 x 32
ex.extract("816", location);// 4 x 19248
ex.extract("818", mask);// maskdim 32 x 19248
ex.extract("820", confidence);// 81 x 19248

但这样导出的 apk 不知为何点进 yolact 就闪退,除了以上这些内容,还有哪些步骤是我忘记了吗?请作者大大指教!

walker-ai commented 2 years ago

按照 nihui 大神的说明成功将 resnet50 训练自己的数据集得到的模型部署至 ncnn 安卓端,运行正常(虽然就是fps有点低,只有可怜的0.2几QAQ),后又按照同样的步骤将 backbone 更换为 EfficientNet,不知为何就无法运行了,一点进去就闪退。感觉应该是模型的问题,但不知道是哪里的问题,期待大佬的指教!这是我的模型文件 parambin

android studio报错信息如下:

I/Hwaps: APS: EventAnalyzed: initAPS: version is 11.0.0.4
D/Hwaps: Fpsrequest create,type:EXACTLY_IDENTIFY
    Fpsrequest create,type:EXACTLY_IDENTIFY
    Fpsrequest create,type:OPENGL_SETTING
    FpsController create
D/Hwaps: APS: EventAnalyzed: reInitFpsPara :mBaseFps = 60; mMaxFps = 60
W/Settings: Setting device_provisioned has moved from android.provider.Settings.Secure to android.provider.Settings.Global.
V/HiTouch_HiTouchSensor: User setup is finished.
W/HwApsManager: HwApsManagerService, registerCallback, start !
D/Hwaps: APS: EventAnalyzed: registerCallbackInApsManagerService, mPkgName:com.wzt.yolov5; result = true
V/AudioManager: querySoundEffectsEnabled...
W/ActivityThread: handleWindowVisibility: no activity for token android.os.BinderProxy@6dc6b5d
V/ActivityThread: callActivityOnCreate
I/DecorView[]: pkgName:com.wzt.yolov5 old windowMode:0 new windoMode:1, isFixedSize:false
W/linker: Warning: "/data/app/com.wzt.yolov5-gC7uo3p7NhkEe0eu7DIzDA==/base.apk!/lib/arm64-v8a/libc++_shared.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)
    Warning: "/data/app/com.wzt.yolov5-gC7uo3p7NhkEe0eu7DIzDA==/base.apk!/lib/arm64-v8a/libc++_shared.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)
D/vulkan: searching for layers in '/data/app/com.wzt.yolov5-gC7uo3p7NhkEe0eu7DIzDA==/lib/arm64'
D/vulkan: searching for layers in '/data/app/com.wzt.yolov5-gC7uo3p7NhkEe0eu7DIzDA==/base.apk!/lib/arm64-v8a'
I/iGraphics: Platform: [kirin980] - [kirin980]
    [0030050c] PV 11
E/iGraphics: can not load local debug xml
D/vulkan: added global layer 'VK_LAYER_HUAWEI_iGraphics' from library '/system/lib64/libiGraphicsCore.huawei.so'
W/com.wzt.yolov5: type=1400 audit(0.0:4994754): avc: denied { map } for pid=19147 path="/dev/__properties__/u:object_r:huawei_perf_persist_public_read_prop:s0" dev="tmpfs" ino=16518 scontext=u:r:untrusted_app:s0:c150,c257,c512,c768 tcontext=u:object_r:huawei_perf_persist_public_read_prop:s0 tclass=file permissive=0
I/vulkan: Loaded layer VK_LAYER_HUAWEI_iGraphics
E/iGraphics: pfnCreateDebugReportCallback is nullptr
I/iGraphics: layers 0 VK_LAYER_HUAWEI_iGraphics
    extensions 0 VK_KHR_external_memory_capabilities
    extensions 1 VK_KHR_get_physical_device_properties2
    extensions 2 VK_KHR_get_surface_capabilities2
    extensions 3 VK_KHR_surface
W/ncnn: [0 Mali-G76]  queueC=0[2]  queueG=0[2]  queueT=0[2]
    [0 Mali-G76]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [0 Mali-G76]  fp16-p/s/a=1/1/1  int8-p/s/a=1/1/1
    [0 Mali-G76]  subgroup=8  basic=1  vote=0  ballot=0  shuffle=0
D/ActivityThread: add activity client record, r= ActivityRecord{61b3d1d token=android.os.BinderProxy@6dc6b5d {com.wzt.yolov5/com.wzt.yolov5.MainActivity}} token= android.os.BinderProxy@6dc6b5d
D/MouseWheelSynthesizer: mMoveStepInDp: 64, mMoveStepInPixel: 192, mUpTimeDelayed: 100
D/ViewRootImpl: ViewRootImpl mIsInProductivePCDisplay: false
D/InputEventReceiver: dispatchInputInterval 1000000
I/Choreographer: Skipped 35 frames!  The application may be doing too much work on its main thread.
D/HiTouch_PressGestureDetector: onAttached, package=com.wzt.yolov5, windowType=1, mIsHiTouchRestricted=false
D/mali_winsys: EGLint new_window_surface(egl_winsys_display *, void *, EGLSurface, EGLConfig, egl_winsys_surface **, EGLBoolean) returns 0x3000
I/HwViewRootImpl: removeInvalidNode all the node in jank list is out of time
D/DecorView: showOrHideHighlightView: hasFocus=false; winMode=1; isMrgNull=true
D/OpenGLRenderer: disableOutlineDraw is true
I/Camera2CameraInfo: Device Level: INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED
D/Camera: Use cases [ImageAnalysis:androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3, Preview:androidx.camera.core.Preview-b9f348d0-1150-4b69-a6a8-49c9464c9ecd] ONLINE for camera 0
D/UseCaseAttachState: Active and online use case: [] for camera: 0
D/Camera: Resetting Capture Session
D/Camera: Opening camera: 0
D/UseCaseAttachState: All use case: [androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3, androidx.camera.core.Preview-b9f348d0-1150-4b69-a6a8-49c9464c9ecd] for camera: 0
I/CameraManager: open camera: 0, package name: com.wzt.yolov5
D/DecorView: showOrHideHighlightView: hasFocus=true; winMode=1; isMrgNull=true
W/InputMethodManager: startInputReason = 1
W/HwRemoteInputMethodManager: isCasting false because IHwDistributedWindowManager is invalid.
D/Camera: Use case ImageAnalysis:androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3 ACTIVE for camera 0
D/UseCaseAttachState: Active and online use case: [androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3] for camera: 0
D/Camera: Use case Preview:androidx.camera.core.Preview-b9f348d0-1150-4b69-a6a8-49c9464c9ecd ACTIVE for camera 0
D/UseCaseAttachState: Active and online use case: [androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3, androidx.camera.core.Preview-b9f348d0-1150-4b69-a6a8-49c9464c9ecd] for camera: 0
D/Camera: CameraDevice.onOpened(): 0
D/UseCaseAttachState: All use case: [androidx.camera.core.ImageAnalysis-4131991d-3026-4a80-81e7-360e4f793ea3, androidx.camera.core.Preview-b9f348d0-1150-4b69-a6a8-49c9464c9ecd] for camera: 0
D/CaptureSession: Opening capture session.
D/CaptureSession: Attempting to send capture request onConfigured
    Issuing request for session.
D/CaptureSession: CameraCaptureSession.onConfigured() mState=OPENED
D/CaptureSession: CameraCaptureSession.onReady()
W/Gralloc3: allocator 3.x is not supported
W/libEGL: EGLNativeWindowType 0x7838a42950 disconnect failed
D/skia: onFlyCompress
D/HwGalleryCacheManagerImpl: mIsEffect:false
A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x7794200000 in tid 19342 (pool-2-thread-1), pid 19147 (com.wzt.yolov5)
D/ActivityThread: Won't deliver top position change in state=5

这是我模型输出的修改:

// EfficientNet
ex.input("x.1", in);
ex.extract("5322", maskmaps);
ex.extract("5519", location);
ex.extract("5521", mask);
ex.extract("5523", confidence);

@cmdbug 请大神帮忙看看,感激不尽!

cmdbug commented 2 years ago

直接调试一下啊,看是哪一句出问题,如果是调用ncnn的语句出问题可以自己用ncnn源码调试或去ncnn提issues。

walker-ai commented 2 years ago

直接调试一下啊,看是哪一句出问题,如果是调用ncnn的语句出问题可以自己用ncnn源码调试或去ncnn提issues。

已经在 ncnn 下提了 issue,感谢作者回复!

walker-ai commented 2 years ago

经调试,打开gpu模式,崩溃堆栈如下:

2022-05-15 11:07:49.244 10301-10429/com.wzt.yolov5 A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 10429 (pool-2-thread-1), pid 10301 (com.wzt.yolov5)
2022-05-15 11:07:49.342 10436-10436/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: Build fingerprint: 'google/sdk_gphone_x86/generic_x86_arm:11/RSR1.201013.001/6903271:user/release-keys'
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: Revision: '0'
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: ABI: 'x86'
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: Timestamp: 2022-05-15 03:07:49+0000
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: pid: 10301, tid: 10429, name: pool-2-thread-1  >>> com.wzt.yolov5 <<<
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: uid: 10154
2022-05-15 11:07:49.343 10436-10436/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
2022-05-15 11:07:49.351 10436-10436/? A/DEBUG: Cause: null pointer dereference
2022-05-15 11:07:49.351 10436-10436/? A/DEBUG:     eax 3bed7304  ebx e3a0a6f4  ecx 00000000  edx 00000002
2022-05-15 11:07:49.351 10436-10436/? A/DEBUG:     edi bf887dc0  esi 00000080
2022-05-15 11:07:49.351 10436-10436/? A/DEBUG:     ebp bf887b58  esp bf887b20  eip e389e90b
2022-05-15 11:07:49.412 10436-10436/? A/DEBUG: backtrace:
2022-05-15 11:07:49.412 10436-10436/? A/DEBUG:       #00 pc 000de90b  /system/lib/libndk_translation.so (ndk_translation::InsnInterpreter::FaultyStore(ndk_translation::DecoderReg, ndk_translation::DecoderReg)+395) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.412 10436-10436/? A/DEBUG:       #01 pc 000e2216  /system/lib/libndk_translation.so (ndk_translation::(anonymous namespace)::ArmInterpreter::GenFaultyStore(int, int)+54) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #02 pc 000f6ec8  /system/lib/libndk_translation.so (ndk_translation::SemanticsDecoder::STR_immediate(ndk_translation::STR_immediate_Args const&)+744) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #03 pc 0012d55b  /system/lib/libndk_translation.so (ndk_translation::ArmDecoder::TranslateThumbInsn(unsigned short const*)+5995) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #04 pc 000e1b50  /system/lib/libndk_translation.so (ndk_translation::(anonymous namespace)::DecodeInsn(unsigned int, ndk_translation::(anonymous namespace)::ArmInterpreter*, ndk_translation::ProcessState*, ndk_translation::ArmDecoder*)+128) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #05 pc 000e1d84  /system/lib/libndk_translation.so (ndk_translation::InterpretTrace(ndk_translation::ProcessState*)+340) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #06 pc 000d1e92  /system/lib/libndk_translation.so (ndk_translation_HandleNotTranslated+194) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #07 pc 000d021a  /system/lib/libndk_translation.so (ndk_translation::ExecuteGuest(ndk_translation::ProcessState*, unsigned int)+266) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #08 pc 000d04ad  /system/lib/libndk_translation.so (ndk_translation::(anonymous namespace)::RunGuestCall(unsigned int, int, int, unsigned int*)+285) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #09 pc 000d060a  /system/lib/libndk_translation.so (ndk_translation::RunGuestCall(unsigned int, ndk_translation::GuestArgumentBuffer*)+42) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #10 pc 000d33da  /system/lib/libndk_translation.so (ndk_translation::(anonymous namespace)::RunGuestJNIFunction(unsigned int, ndk_translation::GuestArgumentBuffer*)+74) (BuildId: 8b541b3f3d4c61fe9dcd94080c7032f4)
2022-05-15 11:07:49.413 10436-10436/? A/DEBUG:       #11 pc 003673ff  <anonymous:c359c000>

打开cpu模式后,崩溃堆栈如下:

2022-05-15 11:15:30.683 10553-10641/com.wzt.yolov5 A/libc: Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x9dccb000 in tid 10641 (pool-2-thread-1), pid 10553 (com.wzt.yolov5)
2022-05-15 11:15:30.852 10647-10647/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2022-05-15 11:15:30.852 10647-10647/? A/DEBUG: Build fingerprint: 'google/sdk_gphone_x86/generic_x86_arm:11/RSR1.201013.001/6903271:user/release-keys'
2022-05-15 11:15:30.853 10647-10647/? A/DEBUG: Revision: '0'
2022-05-15 11:15:30.853 10647-10647/? A/DEBUG: ABI: 'x86'
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG: Timestamp: 2022-05-15 03:15:30+0000
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG: pid: 10553, tid: 10641, name: pool-2-thread-1  >>> com.wzt.yolov5 <<<
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG: uid: 10154
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG: signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x9dccb000
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG:     eax bea93099  ebx 9c67c4b0  ecx 9dccaff8  edx ffffe0a8
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG:     edi 9dccb000  esi 1fff7657
2022-05-15 11:15:30.862 10647-10647/? A/DEBUG:     ebp e1a82030  esp b8df2000  eip c36b40db
2022-05-15 11:15:30.934 10647-10647/? A/DEBUG: backtrace:
2022-05-15 11:15:30.934 10647-10647/? A/DEBUG:       #00 pc 001180db  <anonymous:c359c000>

@cmdbug 能否帮忙看看是什么原因呢,折磨了好几天了 QAQ

cmdbug commented 2 years ago

调试打断点,看是哪一句出问题。