search

alphacep / vosk-android-demo

Offline speech recognition for Android with Vosk library.
Apache License 2.0
752 stars 203 forks source link

java.leng.UnsatisfiedLinkError: Shared library "data/app/.../libkaldi_jni.so" already opened/loaded by ClassLoader ... #74

Closed liuyibox closed 4 years ago

liuyibox commented 4 years ago

My issue comes a little complex which results from one of my repos. The demo works pretty well offline but, if I understood the code corrently, the model can only be initialized with a android activity class. This bring troubles when I need to use the model in a non-activity java class file, which is required in my repo. I am wondering if we can initialize the model inside a normal java class (non-activity)?

Actually, my repo contains a project that tries to load the voice assistant system apk non-activity components using DexClassLoader. The voice assistant system apk contains an activity and several non-activity java class files. This results in a conflict since the .so library cannot be loaded by more than one class loader. In the app's activity class, the app load the kaldi_jni once; in the project mstorm, it loads a non-activity component which loads the kaldi_jni again. The two cannot run at the same time, and will produce the error saying that libkaldi_jni.so already loaded (mstorm loaded the component first, the voice assistant system app was launched later)

I tried to solve this problem by either specifying a static kaldi_recognizer in the activity java class and call it in the non-activity component, or initialize the kaldi_recognizer by writing a constructor for the activity java class and produce the error in the attached screen shot here image

Any suggestions to work around this problem will also be appreciated.

nshmyrev commented 4 years ago

The demo works pretty well offline but, if I understood the code corrently, the model can only be initialized with a android activity class.

It is not true, you just should not load the jni twice

This bring troubles when I need to use the model in a non-activity java class file, which is required in my repo. I am wondering if we can initialize the model inside a normal java class (non-activity)?

You can load jni inside factory (or activity) and use everywhere else. There is no need to load jni in the same class where you use the classes.

liuyibox commented 4 years ago

It is not true, you just should not load the jni twice

Could the Model class be initialized in any class by directly providing the asset path? Since the kaldi Assets class constructor needs the activity context as the argument. I think I've tried to directly provide the Model class with the asset path in non-activity class, but it gives the error that this non-activity class cannot find the implementation of the Model. May

nshmyrev commented 4 years ago

Could the Model class be initialized in any class by directly providing the asset path?

Yes.

Since the kaldi Assets class constructor needs the activity context as the argument. I think I've tried to directly provide the Model class with the asset path in non-activity class, but it gives the error that this non-activity class cannot find the implementation of the Model.

Assets class only needs context to find the data folder and to find the resource. You can write your own class to unpack the model or even download it from the web.

Error should be easy to solve but you have to look on the code yourself.

liuyibox commented 4 years ago

Thanks. After exploring our system code for a while, I still got the following error:

2020-09-01 00:16:11.121 27937-28334/com.lenss.mstorm E/kaldilib: kaldi library not found 2020-09-01 00:16:11.130 27937-28334/com.lenss.mstorm E/om.lenss.mstor: No implementation found for long org.kaldi.VoskJNI.new_Model(java.lang.String) (tried Java_org_kaldi_VoskJNI_new_1Model and Java_org_kaldi_VoskJNI_new_1ModelLjava_lang_String_2) 2020-09-01 00:16:11.134 27937-28334/com.lenss.mstorm W/System.err: java.lang.UnsatisfiedLinkError: No implementation found for long org.kaldi.VoskJNI.new_Model(java.lang.String) (tried Java_org_kaldi_VoskJNI_new_1Model and Java_org_kaldi_VoskJNI_new_1ModelLjava_lang_String_2) 2020-09-01 00:16:11.135 27937-28334/com.lenss.mstorm W/System.err: at org.kaldi.VoskJNI.new_Model(Native Method) 2020-09-01 00:16:11.136 27937-28334/com.lenss.mstorm W/System.err: at org.kaldi.Model.(Model.java:39) 2020-09-01 00:16:11.139 27937-28334/com.lenss.mstorm W/System.err: at com.lenss.liuyi.MyVoiceConverter.prepare(MyVoiceConverter.java:55) 2020-09-01 00:16:11.141 27937-28334/com.lenss.mstorm W/System.err: at com.lenss.mstorm.executor.Executor.run(Executor.java:28) 2020-09-01 00:16:11.141 27937-28334/com.lenss.mstorm W/System.err: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458) 2020-09-01 00:16:11.143 27937-28334/com.lenss.mstorm W/System.err: at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2020-09-01 00:16:11.144 27937-28334/com.lenss.mstorm W/System.err: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 2020-09-01 00:16:11.144 27937-28334/com.lenss.mstorm W/System.err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 2020-09-01 00:16:11.144 27937-28334/com.lenss.mstorm W/System.err: at java.lang.Thread.run(Thread.java:764)

  1. This dlib face recognition repo has the exact .so file structure I can use. Actually I have tested this wrapper and it works in my case. From the error log, it seems that it tried to find the standard Java JNI Java_org_kaldi_VoskJNI_new_1Model but it failed. Maybe this is because the JNI call is exported by swigexport? Could we have similar dlib JNI call in libkaldi_jni.so(i.e., native call is available in the wrapper, native call can system.loadlibrary are in the same java wrapper)? Please refer to this wrapper call and its corresponding dlib jni call which works in my case.

  2. It seems if the build.gradle in a model is set to be apply plugin: 'com.android.library', the java files in the src/java directory cannot import org.kaldi.Model. This stops me from using the Model wrapper inside a library model, however this is needed in my case. I am wondering if this is correct and if there are any methods to hack this around without much effort?

  3. If I can import and call the org.kaldi.Model wrapper without an aar but with only so files in main/src/jniLibs, it should also resolve my problem. Is there such an alternative?

nshmyrev commented 4 years ago

(i.e., native call is available in the wrapper, native call can system.loadlibrary are in the same java wrapper)

I have just pushed the update into vosk-api and corresponding vosk-android-demo update which should load jni inside vosk-android jar. See here:

https://github.com/alphacep/vosk-api/commit/38dbaa15ead20ad055bf88c43584e512579b5738

and other recent commits too.

It seems if the build.gradle in a model is set to be apply plugin: 'com.android.library', the java files in the src/java directory cannot import org.kaldi.Model. This stops me from using the Model wrapper inside a library model, however this is needed in my case. I am wondering if this is correct and if there are any methods to hack this around without much effort?

It is hard to guess what is going on in your case, you need to explain better and provide more details. I have no idea what is "inside a library model".

If I can import and call the org.kaldi.Model wrapper without an aar but with only so files in main/src/jniLibs, it should also resolve my problem. Is there such an alternative?

You can do it too, I see no problem doing that.

liuyibox commented 4 years ago

Thanks @nshmyrev. This resolved my problem! Although I don't know maven at all, but the bintray is always working.

alphacep/vosk-api@38dbaa1

and other recent commits too.