search

googlesamples / mlkit

A collection of sample apps to demonstrate how to use Google's ML Kit APIs on Android and iOS
Apache License 2.0
3.55k stars 2.92k forks source link

Digital Ink crashes when unzipping model on iOS device #231

Closed markyama closed 2 years ago

markyama commented 3 years ago

I tried the digital ink quickstart for iOS and it worked in both the simulator and on an iOS device

The crashing happens on my iOS device in my Unity app: I added the pod MLKitDigitalInkRecognition (1.1.0) to my unity project I added iOS bridge code to call the DigitalInk pod code.

Things work as expected for downloading a recognition model. When I try to recognize ink for the first time, the app crashes in SSZipArchive unzipFileAtPath:toDestination:overwrite:password:error: call when MLKit tries to unzip the downloaded files in com.google.mlkit.vision.digitalinkrecognition/Download/FileData.

I’ve spent a lot of time trying to figure this problem out. The crash always happens in the same place in their code. I think their stack is getting corrupted, and I think it happens in my Unity app because the app uses more more memory which probably causes the crash. I think the quickstart app is so basic and small so there is not enough memory pressure to cause the crash.

This is a good description of the crash and where it happens: https://forum.unity.com/threads/ios-ssziparchive-unzip-file-of-unity-crash.787928/

I’ve posted an issue to SSZipArchive: https://github.com/ZipArchive/ZipArchive/issues/605#issue-797124371

But I’m hoping you might look into it or contact SSZipArchive. The problem makes the MLKitDigitalInk unusable for me.

As a workaround test I added the UnzipKit cocoa pod to my Unity project. Then after having MLKit download a recognition model, I used UnzipKit’s unzipping code to unzip the DataFiles and then I moved them into the com.google.mlkit.vision.digitalinkrecognition/UnzippedModelCache folder before making my first recognition call. Then the recognizeInk call worked (because it didn’t have to unzip the model data files). So I’m pretty sure the problem is in SSZipArchive.

I can’t use this as a functional workaround because the MLKit code calls the SSZipArchive unzip when it needs to, but you unzip to a tmp location. Then you rename the file as you move it into the UnzippedModelCache folder. In my test workaround, I saw what filenames were created in the UnzippedModelCache via the working quickstart code for the specific language model I was trying. But I could not do this in my live app, as I don’t know what language model a user will download.

Any help would be appreciated.

BTW, the digital ink quickstart for iOS is great, thank you for making it.

zuoksrf commented 3 years ago

Hey @markyama, I am trying to develop a game using MLKit-DigitalInkRecognition in Unity (iOS), and there is some problems with writing wrapper-bridge methods Can you explain how did you do that, and Is there any problem with using MLKit in Unity that you know?

markyama commented 3 years ago

First I got the QuickStart I mentioned working in Xcode. Once I got that working, I got familiar with the iOS calls to make Digital Ink to work. Then I worked on bridging those calls from Unity. https://docs.unity3d.com/Manual/PluginsForIOS.html I never resolved the problem I talked about above where it would crash when MLKit used SSZipArchive to unzip files it downloads, and no one responded to this issue. I just looked and there is are more recent versions of SSZipArchive and the Digital Ink (1.3.0). It may be that the issue is resolved using those versions.

miworking3 commented 2 years ago

ML Kit doesn't support non-standard (via Unity, etc), so closing for now.