Running with Ivy Bridge and other non-AVX2 hardware

[nealmcb]

The release notes for Release Deep Speech 0.1.0 · mozilla/DeepSpeech say

• Supported Platforms....
  • Linux x86 64 bit with a modern CPU (Supports up to AVX2/FMA)

For those of us with slightly older hardware, e.g. Intel Ivy Bridge, it would be great to have a straightforward way to try Deep Speech out and get involved.

There are some fragmented notes and advice in #896 on how to build a Docker image that could be easily shared, and how to compile some things from source, but they haven't been pulled together and made to work for this situation and I think should actually be significantly easier with the 0.1.0 release.

Another alternative would be full support for this older hardware, which is after all supported by the default upstream Tensorflow builds (https://github.com/tensorflow/tensorflow/issues/6809), but I know the Mozilla team doesn't want to be pulled in too many different directions.

So does anyone have the Mozilla Tensorflow source built without AVX2, integrated with Deep Speech, either in Docker or just via a clear set of instructions?

[lissyx]

There is already documentation on how to rebuild in native_client/README.md. There's nothing fancy for the Tensorflow fork we use, no special build steps ; the readme refers to upstream for that part. You just need to apply those upstream docs with our fork.

[lissyx]

@nealmcb Have you been able to build it without AVX2 for your system ? Having to support multiple builds variants raises quite a lot of questions on our side, so I really cannot promise anything. We might discuss limit ourself to not require AVX2, but it also questions where should we put the limit and requires we benchmark to see if that is not going to be too bad performance-wise.

[nealmcb]

Thank you, @lissyx for your clear reference to the build instructions! I've been preoccupied with other stuff and haven't tried it out yet. And I appreciate the reasons you don't want to support multiple configurations!

Currently, I'm hoping that this thread inspires someone to put together a Docker configuration that both provides the sort of build I'm looking for, and also makes it easy for anyone to run this locally or in the cloud to scale this stuff up. But I probably won't have a lot of time to play with it in the next few weeks.

[pannous]

pasting for others who are searching why their pip package is 'broken'

/usr/local/bin/deepspeech -h
2017-11-29 21:52:49.519889: F tensorflow/core/platform/cpu_feature_guard.cc:35] The TensorFlow library was compiled to use AVX2 instructions, but these aren't available on your machine.
Aborted (core dumped)

solution: see above

[cmcfarla]

On older systems, you might also get the error "Illegal instruction (core dumped)".

The fact that something more than baseline x86_64 support is required in order to use the primary install instructions is rather surprising. If you must have AVX2, could you please add a note to the README?

[lissyx]

@cmcfarla We have that documented in the releases notes already, but PR are welcome :)

[cavac]

Of course, trying to build tensorflow also fails, after having to butcher my local Java install just to run "bazel", whatever that is...

ERROR: /home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/io_bazel_rules_closure/closure/private/defs.bzl:27:16: The set constructor for depsets is deprecated and will be removed. Please use the depset constructor instead. You can temporarily enable the deprecated set constructor by passing the flag --incompatible_disallow_set_constructor=false

Even trying to add the recommended parameter, the build still fails:

bazel build --config=opt --incompatible_disallow_set_constructor=false //tensorflow/tools/pip_package:build_pip_package ERROR: /home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/BUILD:460:1: Traceback (most recent call last): File "/home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/BUILD", line 460 cc_proto_library(name = "cc_test_protos", srcs = (L...), <4 more arguments>) File "/home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/protobuf.bzl", line 248, in cc_proto_library cc_libs += [default_runtime] trying to mutate a frozen object ERROR: /home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/BUILD:761:1: Traceback (most recent call last): File "/home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/BUILD", line 761 py_proto_library(name = "python_specific_test_pro...", <6 more arguments>) File "/home/cavac/.cache/bazel/_bazel_cavac/e114633076593be224d35a798cc2e8fd/external/protobuf/protobuf.bzl", line 374, in py_proto_library py_libs += [default_runtime] trying to mutate a frozen object ERROR: /home/cavac/src/deepspeech/tensorflow/tensorflow/tools/pip_package/BUILD:100:1: Target '@protobuf//:LICENSE' contains an error and its package is in error and referenced by '//tensorflow/tools/pip_package:licenses' ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted: Loading failed INFO: Elapsed time: 35.026s FAILED: Build did NOT complete successfully (66 packages loaded) currently loading: tensorflow/core ... (2 packages)

For now, i'm giving up on DeepSpeech. The project and installation procedures are clearly not yet ready to be used in serious projects.

[lissyx]

@cavac Are you having this with Bazel 0.5.4 ?

[cavac]

$ bazel version Build label: 0.8.0 Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar Build time: Mon Nov 27 13:19:39 2017 (1511788779) Build timestamp: 1511788779 Build timestamp as int: 1511788779

And it is the mozilla version of tensorflow

[lissyx]

@cavac Right, then you should use bazel 0.5.4, as this is the requirement from tensorflow 1.3~ish we are still based on. Work is currently ongoing to move to being based on r1.4 thus this kind of requirement of rebuilding would not be needed.

[cavac]

Hmm, ok...? I just installed it like it says on https://docs.bazel.build/versions/master/install-ubuntu.html

Using Ubuntu 16.04. How do i install an outdated version of bazel?

BTW, Processor is a i7-4930K CPU @ 3.40GHz. System is about 3 years old (i never buy the newest generation of hardware - too expensive and buggy).

[lissyx]

@cavac You can find it there: https://github.com/bazelbuild/bazel/releases/tag/0.5.4

[lissyx]

@cavac Have you been able to make any progress? FYI my desktop (4790K) is more than three years old and it does have AVX2.0: https://ark.intel.com/compare/77780,80807

[cavac]

No, haven't made progress yet. Currently quite busy, have to look back into it in a few weeks. I'll try it on a clean install of Ubuntu next.

Is the speech recognition part supposed to be able to run in VirtualBox, too?

[lissyx]

Yes, it should, as long as you have AVX2/FMA given the current binaries :). FYI I'm currently making variations of builds around flags to see, test and benchmark.

[lissyx]

Benchmarking a r1.4-based build set against two models (one overfit LDC93S1 with n_hidden=988 and one close to release one), and two audio files (WAV mono PCM 16kHz, one 2.924s and one 5.55s):

• AVX stands for --copt=-mtune=generic --copt=-march=x86-64 --copt=-msse --copt=-msse2 --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx
• AVX2+FMA stands for --copt=-mtune=generic --copt=-march=x86-64 --copt=-msse --copt=-msse2 --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma

[lissyx]

I just merged the tensorflow changes ; Once all builds on https://tools.taskcluster.net/groups/BPuULj2RTDedU8xtLtxb8Q are completed, then we can start building deepspeech against this tensorflow that bundles only upto AVX.

[nealmcb]

Thanks, @lissyx! It looks like it completed successfully, along with some other more recent builds. I'm presuming the results will show up when ready at https://github.com/mozilla/DeepSpeech/releases. Is there another place to track build / release progress?

[lissyx]

Not yet, but we'll do something soon, I guess, since we have several similar improvements. We're all in Austin this week, for the All Hands, this might help :)

[nealmcb]

deepspeech is working for me now, on my non-AVX2 hardware. Yay! I'm not sure if you've changed something, or if I have, by using the native client.

wget https://index.taskcluster.net/v1/task/project.deepspeech.deepspeech.native_client.master.cpu/artifacts/public/native_client.tar.xz

ls -l native_client.tar.xz
10186452 Dec 11 14:21 native_client.tar.xz

tar xf native_client.tar.xz

arecord -f S16 -r 16000 /tmp/test2.wav 

time ./deepspeech models/output_graph.pb /tmp/test2.wav models/alphabet.txt models/lm.binary models/trie
testing testing one two three testing

real    0m26.428s
user    0m16.496s
sys     0m4.796s

[lissyx]

@nealmcb Yes, we went with limiting the requirements to AVX now, so TaskCluster artifacts are working, thanks for confirming and testing!

[nealmcb]

Aha - thanks. But I'm still a bit confused, since deepspeech still fails after a pip install, expecting AVX2.

I guess that the a "native_client" install gets a more up-to-date version, built via a recent TaskCluster run.

But that doesn't provide the deepspeech python module, and I'm unclear on how to get that in a way that works with the new build, and with, e.g. https://github.com/MainRo/deepspeech-server.

[lissyx]

That's right, we have not yet pushed updated versions of the packages to the registries. But you can already download them from TaskCluster: https://tools.taskcluster.net/index/artifacts/project.deepspeech.deepspeech.native_client.master/cpu

Copy paste the proper URL of the wheel you need, then pip install <url> and it should be okay :)

[nealmcb]

Excellent! Note: using --upgrade may be important. E.g. for the Python 3.6 build:

pip install https://index.taskcluster.net/v1/task/project.deepspeech.deepspeech.native_client.master.cpu/artifacts/public/deepspeech-0.1.0-cp36-cp36m-manylinux1_x86_64.whl --upgrade

For clarity, and to fix that, I would think that the version numbers should be something like deepspeech-0.1.1-dev-<buildnumber>.

[ftry]

2017-12-15 22:17:31.710780: F tensorflow/core/platform/cpu_feature_guard.cc:35] The TensorFlow library was compiled to use AVX2 instructions, but these aren't available on your machine.

[ftry]

2017-12-15 22:17:31.710780: F tensorflow/core/platform/cpu_feature_guard.cc:35] The TensorFlow library was compiled to use AVX2 instructions, but these aren't available on your machine.

my computer is 64 i5 ubuntu14.04ls how i can use it????

[lissyx]

@ftry Please read the issue, we toggled off AVX2+FMA to lower the requirements for other people.

[luomingshuang]

i can run my code, but the results make me sad, it can't export the correct inference result....just can export 'i' and 'h'......

[lissyx]

@luomingshuang Please avoid spamming issues with unrelated bugs, and do not use screenshots for debugging.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.