Mac support

[realdoug]

I have a fork that builds & runs natively on a Mac without needing docker. Is that something you'd like to incorporate on master?

My use case is to load pre-trained weights and run inference mode on a Macbook. All tests pass, but I've mainly focused on testing inference rather than training (and I do not have an eGPU).

The changes are here. I can clean this up a bit, but wanted to check first before expending the time on that. Main update is that Flashlight does not support distributed mode on Mac, so Wav2letter must also support a non-distributed build to support Mac. Thanks.

[jacobkahn]

@realdoug — we can definitely take a look. We have several reasons for enforcing having built distributed mode, and we want to use the preprocessor as little as possible, but we're open to changes.

[lunixbochs]

I am extremely in support of a native Mac build. I'm the developer behind https://talonvoice.com and I'm working hard on incorporating wav2letter as a supported speech backend on Mac w/ CPU inferencing instead of Dragon.

In addition to the changes here, does MKL's cblas have some additional benefit / parallelization over something like Accelerate.framework? I simplified my Mac build a bit by doing the following and linking -framework Accelerate (to remove wav2letter's internal use of MKL just for two cblas functions):

diff --git a/src/feature/SpeechUtils.h b/src/feature/SpeechUtils.h
index 3b3a8df..8995951 100644
--- a/src/feature/SpeechUtils.h
+++ b/src/feature/SpeechUtils.h
@@ -10,7 +10,8 @@

 #include <vector>

-#include <mkl.h>
+#include <Accelerate/Accelerate.h>
+#define MKL_INT int

 #include "FeatureParams.h"

Then I was able to use the build instructions from https://github.com/intel/mkl-dnn.git in an automated build script instead of installing Intel's MKL SDK.

[jacobkahn]

@lunixbochs — we can definitely work towards a smoother build process for Mac.

From what I've seen, on MacOS, MKL and Accelerate have essentially the same performance. That said, we want to keep wav2letter as platform-agnostic as possible, and having MKL be the default BLAS framework is one of the ways in which we accomplish this. I empathize with it seeming like a lot of overhead for two functions, but it's better than making things too opinionated towards a particular OS given that we need a bit of BLAS.

It seems to me that this is a small change as is :) , but if you have other ideas that will be portable that will make building/using w2l on MacOS better, let us know or feel free to send a PR!

[lunixbochs]

It looks like the libmklml.dylib lib downloaded by the script in MKL-DNN also provides those cblas functions, so making it easier to detect and use that library would also solve the MKL SDK dependency, which I guess would be a FindMKL.cmake change.

I'm just unwilling to have a human register on Intel's website, download and install an SDK, and manually grab libraries out of it as part of my build process :)

I think this gives me all of the necessarily MKL libraries non-interactively (the prepare_mkl and external part grab precompiled libraries), and they have a batch file to grab the MKL ML library dependency on Windows as well:

git clone https://github.com/intel/mkl-dnn.git
cd mkl-dnn
./scripts/prepare_mkl.sh
cp -av external/mklml_*/{include,lib} "$prefix"
git checkout v0.18.1
if [[ ! -e Makefile ]]; then
    cmake . -DWITH_TEST=OFF -WITH_EXAMPLE=OFF -DCMAKE_INSTALL_PREFIX="$prefix"
fi
make -j4 && make install

(This is all in support of my pending script that can build a redistributable wav2letter from scratch on a Mac)

[lunixbochs]

Ok, my build script is attached. I removed the Accelerate dependency by manually setting -DMKL_LIBRARIES='-lmkldnn -lmklml -liomp5' to use mklml's cblas and I had to set LIBRARY_PATH because I was bypassing FindMKL.cmake.

export LIBRARY_PATH="$prefix/lib"
cmake .. -DW2L_BUILD_TESTS=NO -DCMAKE_BUILD_TYPE=Release -DW2L_CRITERION_BACKEND=CPU \
    -DCMAKE_INSTALL_PREFIX="$prefix" -DMKL_LIBRARIES='-lmkldnn -lmklml -liomp5' \
    -DCMAKE_EXE_LINKER_FLAGS="-L'$prefix'" -DW2L_BUILD_DISTRIBUTED=OFF

So I think there are three things needed:

• [ ] Fix FindMKL.cmake to support libmklml in wav2letter (it looks like ArrayFire maybe also fell back to Accelerate, I assume it uses the same FindMKL.cmake script)
• [ ] Fix distributed Flashlight (Gloo dependency) on Mac, or accept a temporary patch to allow disabling distributed wav2letter builds until Flashlight has distributed CPU support on Mac? I assume it also doesn't work on Windows atm because of Gloo?
• [ ] Merge or otherwise fix the couple of type signature issues in w2l.

build.sh.zip

[jacobkahn]

@lunixbochs — this looks great. I can try to make some of these changes in the next few days. I'm not sure if ArrayFire is aware of libmklml. As far as I know, mklml originated and is coupled with MKL-DNN. It's more likely that ArrayFire fell back to Accelerate; it certainly seems aware: https://git.io/fjqRt.

Have you tested that the cblas symbols in mklml run properly with the featurization components of w2l requiring those blas operations? If so, we can very easily tweak our MKL build process to also link to mklml if present, so it's good to know if those work or not. Also using accelerate may end up being an easier change depending on how easily it's found by CMake.

Responding to those todos you listed:

• Supporting finding mklml natively in fl is as simple as moving the requisite calls out of FindMKLDNN and into another module which we can then also call from FindMKL.
• I've been meaning to revisit removing the flashlight distributed requirement for w2l. I'll take another look, but it may require some modifications to flashlight to do right, so it's not quite a trivial change.
• Can you be more specific on those issues? Are these primarily #include <mkl.h>-related?

You should feel free to submit PRs for the quick changes — we'll happily review and accept them!

[lunixbochs]

How should I test that the two BLAS functions are working? It Test/Decode link and runs and featurize + inference is producing correct output with mfsc / asg for me. I’m on the fence about personally shipping/using their libmklml binary after I noticed it’s 86MB, supporting Accelerate would be nice in the long run for packaging alone.

I think the quick changes are just the couple of type casts and stuff from @realdoug’s branch to make functions like std::max work, which were part of this commit: https://github.com/realdoug/wav2letter/commit/3097d8defc0c746e4563fe6a1c8337599335d692 (though personally I’d maybe tweak some of the types used, if you don’t have a Mac to test I can take the time to clean up and PR that part)

[realdoug]

My 2c is that the underlying want is really to have a "production mode" build of w2l that essentially supports inference only, since I bet most people looking for Mac support are more interested in deployment than training (that's my use case at least). This is probably exploding this issue into a larger scope, but it would more directly address the core issue and make w2l useful for a much wider range of people and use cases.

I'd be really happy to see a feature like this and would be happy to help out in making it happen.

[TheMathMajor]

What are the exact steps and commands I need to run to build this version on Mac? How should l install and build the dependencies and in what order?

[jacobkahn]

@realdoug — circling back on this after a long time - we recently released a standalone inference pipeline that runs out of the box.

The simplest Mac support is still contingent on either removal of OpenMPI or relaxation of the requirement that wav2letter be linked to a flashlight built in distributed mode. We're recently considered revisiting some of these things., but it will likely be some time before they're implemented.