Open Veango opened 1 year ago
Could be a mixup of files, but it better should not just crash. Cannot decode as reported, though.
Could you post the backtrace from a debug build? Also,
bt
command. If you look carefully, backtrace lines are all cut short. This info will be helpful.
what do you mean by "init kaldi", which code/program are you using? Is it something homebrewn? y.
On Tue, Sep 13, 2022 at 5:14 PM Veango @.***> wrote:
when I init the Kaldi , it will crash like this: [image: 1] https://user-images.githubusercontent.com/77492855/189939267-33ce957f-636d-4688-bd35-f389d74fcabf.png [image: 2] https://user-images.githubusercontent.com/77492855/189939305-dda30b50-97b2-42ad-aa59-1d1c6d551dfc.png
please tell me how can I fix it
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX6GDFQKWVVWMEACG73V6CK43ANCNFSM6AAAAAAQLRPLDY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
@kkm000
I used openblas,the code runs in iOS ,is Kaldi not support iOS?
another info is when I call make -C matrix test, it got some error like: numpy-array-test.cc:82:15: error: 'system' is unavailable: not available on iOS rc = std::system("python3 -c \"import numpy; " ...
'system' is unavailable: not available on iOS
Are you in fact building the whole of Kaldi on iOS, or is it a nonsensical message? You are likely much better off cross-compiling.
What is your platform? MacOS has its own linalg framework, I forgot its name (@jtrmal help me please, "accel" or something?), use it instead. It's configurable. Especially with their latest accelerator in M1/M2, it must use the accelerator. I've seen iOS used only on devices no beefier than the iPad. Does it have an accelerator for linear algebra? Does OpenBLAS support it? In any case, you will be much better off fully utilizing hardware stuff which is usually a-plenty on device SoCs, which speeds up a limited set of operation manifold, frees up CPU and reduces power consumption.
[does] Kaldi not support iOS?
A gray area. There are too many things that may be called iOS, and too many hardware variants it runs on. Since there's no one around with iOS experience, rather no, not in the full sense of "support." Do use the kaldi-help list, search its archives, post a question if nothing comes up. maybe you'll find a kindred soul.
For the crash, let's start off by removing threading from the picture: set threading to 1 globally, kaldi::g_num_threads = 1;
before using any Kaldi functions. i-vector extractor initialization is multithreaded by default, and you see it in the stack.
The default of 8 is (statically, at compile time) set in util/kaldi-thread.cc:25 and then used to initialize sequencer options a few times through src/ivector/ivector-extractor.cc.
If this has no effect, running the same init code on an x64 Linux or Windows, to make sure we don't have a case of broken or mismatched extractor files. There are a few.
Thank you for your reply,I cross-compiled openblas clapack openfst and kaldi,i ever used the same code in android , it hasn't any problem. But i used it in iPhone platform, it crash,so i don't know witch step i was wrong
Looks like an OpenBLAS issue. As I mentioned, the Apple's own solution, Accelerate, includes BLAS routines. And its docs mention that it's available in iOS. By all means, use it instead of OpenBLAS. It is supposed to use all hardware acceleration there is on the platform, and in the optimal way. For a device, it is especially important, as it not only does it faster but saves battery, too (assuming there is hardware available; I know little about iPhones and friends, but mid-line to high-end modern devices just have to have it).
Try to figure out why it is not selected by configure automatically. Maybe we just don't handle iOS cross correctly?
If you can find a simple way to fix configure
so that it picks up the correct library, we'd certainly welcome a patch. But if you find that you need to bend all the way backwards to make configure
work for you, I want to share how I productize Kaldi. For training and stuff on Linux, I of course use the standard workflow, configure + make. But for compiling the product, I select necessary files from Kaldi into a library (usually static, it allows for advanced optimizations such as PGO for the final build, but I don't know what's the best for your case), and simply build it. You need to define one preprocessor variable (otherwise defined in a generated base/version.h
file), namely KALDI_VERSION
(to any value, it's used only in diagnostic messages) in the command line to compiler. All compile flags and dependency libraries you can figure out from your platform SDK. You can also catch Kaldi output using the hook routine in base/kaldi-error.h
, so that they are logged your way instead of being output to stderr.
OpenFST also has a logging hook in log.h; that's separate from Kaldi, you need to call it too if you want to hook up their logging.
You mean i shouldn't use openblas and clapack,just only compile openfst and kaldi,don't set the MATHLIB,and it will use the system's accelerate if it's no problem?
No, this is not exactly what I said.
i shouldn't use openblas and clapack
Correct, use iOS native Accelerator framework.
don't set the MATHLIB, and it will use the system's accelerate
During configure? I don't know, possibly. Try it. It may or may not work. If it doesn't. try to figure out and easy fix to configure
, it's hand-coded, not generated. If it happens too tricky to get right, you may do it differently. You also may do it this way to get a greater control over build options, namely:
just only compile openfst and kaldi
As an alternative to configure
, you may compile the runtime-only files your own way. I do it both to OpenFST and Kaldi, normally. This is what I do for production builds. You'll have to write whatever build files your build system/IDE requires. But it is the most flexible option. Mind the KALDI_VERSION
. I can't recall any other tripping points.
Optionally, regardless of the way you build, hook logging (see kaldi-error.h). You'll probably need it for diagnostics at some point. This applies to all platforms, not only iOS. Kaldi also has SetVerbosityLevel()
, beyond INFO, WARNING and ERROR, from 0 to 5, 5 is insane amount of logging. When you don't do verbose logs, set to 0 (default), it's expensive to generate, so don't simply generate and discard it. INFO and above are always logged.
Thanks a lot,i will try it,if i find a simple way to fix it,i will report it here
Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while.
Accelerate framework. sorry for the delay No need to link with anything else y.
On Wed, Sep 14, 2022 at 7:36 PM kkm000 @.***> wrote:
'system' is unavailable: not available on iOS
Are you in fact building the whole of Kaldi on iOS, or is it a nonsensical message? You are likely much better off cross-compiling.
What is your platform? MacOS has its own linalg framework, I forgot its name @.*** https://github.com/jtrmal help me please, "accel" or something?), use it instead. It's configurable. Especially with their latest accelerator in M1/M2, it must use it. I've seen iOS used only on devices no beefier than the iPad. Does it have an accelerator for linear algebra? Does OpenBLAS support it? In any case, you will be much better off fully utilizing hardware stuff which is usually a-plenty on device SoCs, which speeds up a limited set of operation manifold, frees up CPU and reduces power consumption.
[does] Kaldi not support iOS?
A gray area. There are too many things that may be called iOS, and too many hardware variants it runs on. Since there's no one around with iOS experience, rather no, not in the full sense of "support." Do use the kaldi-help list, search its archives, post a question if nothing comes up. maybe you'll find a kindred soul.
For the crash, let's start off by removing threading from the picture: set threading to 1 globally, kaldi::g_num_threads = 1; before using any Kaldi functions. i-vector extractor initialization is multithreaded by default, and you see it in the stack.
The default of 8 is (statically) set in util/kaldi-thread.cc:25 and then used to initialize sequencer options a few times through src/ivector/ivector-extractor.cc.
If this has no effect, running the same init code on an x64 Linux or Windows, to make sure we don't have a case of broken or mismatched extractor files. There are a few.
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1247403766, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX7ZYKVHPTPAWF2ZYO3V6JOPJANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y.
On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.***> wrote:
Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while.
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1247519137, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.>
Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason?
kaldi itself should be fine. It's just the setup/compile pipeline and some scripts will/would have to be altered. y.
On Sat, Oct 1, 2022 at 5:58 AM Veango @.***> wrote:
IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … <#m-4983130712797081710> On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1247519137>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.>
Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason?
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1264310854, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
kaldi itself should be fine. It's just the setup/compile pipeline and some scripts will/would have to be altered. y. … On Sat, Oct 1, 2022 at 5:58 AM Veango @.> wrote: IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … <#m-4983130712797081710> On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 (comment) <#4791 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.> Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason? — Reply to this email directly, view it on GitHub <#4791 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.>
But as i know,clapack used System () method, may be i should rewrite it
Hi, clapack shouldn't need it, unless it's some special logging function or something like that. Or it might be called as a part of build process. I.e. should be possible to replace it. I cannot help, sorry. Not enough experience. y.
On Tue, Oct 4, 2022 at 6:27 AM Veango @.***> wrote:
kaldi itself should be fine. It's just the setup/compile pipeline and some scripts will/would have to be altered. y. … <#m-7141260418549791347> On Sat, Oct 1, 2022 at 5:58 AM Veango @.*> wrote: IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … <#m-4983130712797081710> On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 https://github.com/kaldi-asr/kaldi/issues/4791 (comment) <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1247519137>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.> Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason? — Reply to this email directly, view it on GitHub <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1264310854>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
But as i know,clapack used System () method, may be i should rewrite it
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1266735938, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZTV5W5KJEYINORXV3WBQBA7ANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
Hi, clapack shouldn't need it, unless it's some special logging function or something like that. Or it might be called as a part of build process. I.e. should be possible to replace it. I cannot help, sorry. Not enough experience. y. … On Tue, Oct 4, 2022 at 6:27 AM Veango @.> wrote: kaldi itself should be fine. It's just the setup/compile pipeline and some scripts will/would have to be altered. y. … <#m-7141260418549791347> On Sat, Oct 1, 2022 at 5:58 AM Veango @.> wrote: IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … <#m-4983130712797081710> On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 <#4791> (comment) <#4791 (comment) <#4791 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.> Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason? — Reply to this email directly, view it on GitHub <#4791 (comment) <#4791 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.**> But as i know,clapack used System () method, may be i should rewrite it — Reply to this email directly, view it on GitHub <#4791 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZTV5W5KJEYINORXV3WBQBA7ANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.>
https://github.com/alphacep/clapack/tree/master/F2CLIBS/libf2c)/system_.c this file used System () method,i can't cross-compile with ios sdk
I think it's safe to make this function body empty or something like that y.
On Fri, Oct 7, 2022 at 3:44 AM Veango @.***> wrote:
Hi, clapack shouldn't need it, unless it's some special logging function or something like that. Or it might be called as a part of build process. I.e. should be possible to replace it. I cannot help, sorry. Not enough experience. y. … <#m-3657227165052438647> On Tue, Oct 4, 2022 at 6:27 AM Veango @.> wrote: kaldi itself should be fine. It's just the setup/compile pipeline and some scripts will/would have to be altered. y. … <#m-7141260418549791347> On Sat, Oct 1, 2022 at 5:58 AM Veango @.> wrote: IOS has not been tried. Just MacOS... that might be the confusion. Also, it might make sense that for Ios (for whatever security reason), system() doesn't work y. … <#m-4983130712797081710> On Wed, Sep 14, 2022 at 10:58 PM kkm000 @.> wrote: Thanks, we really have no idea if we configure a cross-compile for iOS correctly (or at all)! All scripting for it, if exists at all, was contributed outside of the core team. I'll leave this issue open for a while. — Reply to this email directly, view it on GitHub <#4791 https://github.com/kaldi-asr/kaldi/issues/4791 <#4791 https://github.com/kaldi-asr/kaldi/issues/4791> (comment) <#4791 https://github.com/kaldi-asr/kaldi/issues/4791 (comment) <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1247519137>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYX3TIECARZZ3T2KB2JTV6KGEZANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.> Yes, ios didn't support System () method any more,so kaldi can't use in ios is that reason? — Reply to this email directly, view it on GitHub <#4791 https://github.com/kaldi-asr/kaldi/issues/4791 (comment) <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1264310854>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZQNMROXF2ZDEFTRMLWBADMHANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.> But as i know,clapack used System () method, may be i should rewrite it — Reply to this email directly, view it on GitHub <#4791 (comment) https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1266735938>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYXZTV5W5KJEYINORXV3WBQBA7ANCNFSM6AAAAAAQLRPLDY https://github.com/notifications/unsubscribe-auth/ACUKYXZTV5W5KJEYINORXV3WBQBA7ANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.>
https://github.com/alphacep/clapack/tree/master/F2CLIBS/libf2c)/system_.c this file used System () method,i can't cross-compile with ios sdk
— Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/4791#issuecomment-1271234110, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUKYX7GW3TPIC5I6GFPWOTWB7IEBANCNFSM6AAAAAAQLRPLDY . You are receiving this because you were mentioned.Message ID: @.***>
This issue has been automatically marked as stale by a bot solely because it has not had recent activity. Please add any comment (simply 'ping' is enough) to prevent the issue from being closed for 60 more days if you believe it should be kept open.
when I init the Kaldi ,
it will crash like this:
![2](https://user-images.githubusercontent.com/77492855/189939305-dda30b50-97b2-42ad-aa59-1d1c6d551dfc.png)
please tell me how can I fix it