search

tensorflow / tensorflow

An Open Source Machine Learning Framework for Everyone
https://tensorflow.org
Apache License 2.0
185.73k stars 74.2k forks source link

[Request] Pre-build support old CPU #18689

Closed tranhungnghiep closed 6 years ago

tranhungnghiep commented 6 years ago

System information

Describe the problem

Since version 1.6 prebuilt binaries use AVX instructions, thus TF cannot be used on "old" CPUs. However, many of them are not that old. In fact, I was surprise that my server cannot run TF. I believe many people run into the same problem because those CPUs are popular in many labs at universities.

This is bad because it blocks many potential users/researchers from trying TF and starting using TF. User should not build TF themselves, because building by each user is daunting and resource-wasteful.

So if it is not technically impossible, please provide alternative prebuilt binaries that support old CPUs.

PaulWang1905 commented 6 years ago

You can find unofficial builds from other repo.

tranhungnghiep commented 6 years ago

@PaulWang1905 I should have mentioned that I could not find unofficial prebuilt for neither 1.6 nor 1.7. Moreover, I think this is a legitimate need, thus it should be supported officially.

hadim commented 6 years ago

I am facing the same issue. My recent laptop can run TF without issue but on my desktop computer at the lab, I can't since the CPU is quite old.

Building it is a possibility but honestly, it's quite a complex task...

If you are aware of some repo that provides unofficial Linux builds for old CPU please tell us.

tranhungnghiep commented 6 years ago

Also please consider document this problem clearer, or make the error message more informative. Currently import tensorflow just exit the program, kills python and says Illegal instruction.

PaulWang1905 commented 6 years ago

which generation is your microarchitecture? I build tensorflow, but not for CPU. I can build unofficial builds if you guys really need

tranhungnghiep commented 6 years ago

@PaulWang1905 My CPU is Intel(R) Xeon(R) CPU E7- 8837. Detailed information is below. Thanks a lot!

vendor_id : GenuineIntel cpu family : 6 model : 47 model name : Intel(R) Xeon(R) CPU E7- 8837 @ 2.67GHz stepping : 2 microcode : 0x36 cpu MHz : 2659.832 cache size : 24576 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 8 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good noplxtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt aes lahf_lm kaiser tpr_shadow vnmi flexpriority ept vpid dthermida arat bugs : clflush_monitor bogomips : 5319.66 clflush size : 64 cache_alignment : 64 address sizes : 44 bits physical, 48 bits virtual

hadim commented 6 years ago

Here is mine:

vendor_id   : GenuineIntel
cpu family  : 6
model       : 26
model name  : Intel(R) Core(TM) i7 CPU         960  @ 3.20GHz
stepping    : 5
microcode   : 0x19
cpu MHz     : 1652.866
cache size  : 8192 KB
physical id : 0
siblings    : 8
core id     : 3
cpu cores   : 4
apicid      : 7
initial apicid  : 7
fpu     : yes
fpu_exception   : yes
cpuid level : 11
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm pti tpr_shadow vnmi flexpriority ept vpid dtherm ida
bugs        : cpu_meltdown spectre_v1 spectre_v2
bogomips    : 6414.75
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:
av8ramit commented 6 years ago

Unfortunately we are unable to provide pre-built binaries without AVX. Unofficial third-party builds may be your best option.

makeandbreak commented 6 years ago

Forcing usage of machines with Intel CPU's that contain the AVX instruction is a false move in my opinion. I have been testing other AI platforms and they have no such requirement, Also when one is primarily focusing on GPU acceleration for AI apps the usage of AVX instruction becomes a mute point. This move is making it problematic for many user groups not having access to the latest CPU's for example hobbyists and schools with older hardware. Please consider a AVX free build or me and others will be forced to ditch TensorFlow for another platform.

tfboyd commented 6 years ago

@makeandbreak

I have some questions to better understand the situation and please anyone response not just makeandbreak:

We are looking into options. AVX very much speeds up performance for CPU over SSE, this is also true for AMD processors. The move was done to improve out of the box performance for CPU users in general as well as those doing inference, which is often done on CPU and gets a large boost from AVX.

This section is not to win you over but you let you know the thought process at a high level. I looked at the Intel CPU line. Sandy bridge, which is where AVX started came out, was in 2011. AWS makes it fairly hard to get non-AVX systems for anything but random web serving and that is still limited. I suspected many users are using mac book pros and other laptops that are sandy bridge and newer. AVX provides a significant boost and I wanted to get that to end users.

Finally for this comment, the build matrix is very large because it needs to be supported, e.g. all unit tests need to pass and it should be performance tested. We want to make a pragmatic decision and your feed back is very helpful. We do not know what we do not know and I personally want to balance ensuring people can use TensorFlow, with TensorFlow being fast, and the testing workload.

The MKL build from Intel might work but I think they still default to AVX for ops other than those handled by MKL. We are looking at this option. It has been almost a year but I did test MKL on AMD Ryzen and it was fine and can be tested again. Painful to test as at the time and even still now there are not many or any cloud providers with AMD systems, I suspect some schools still have opterons laying around.

tfboyd commented 6 years ago

Assigning to myself in the short-term until we have a final answer. If someone has a public build for 1.8 feel free to share in the short-term. The rub is do you do python 2, 3 both and do you also do mac and windows along with linux. :-) The matrix explodes fast.

bhack commented 6 years ago

Why we cannot approach a solution like https://github.com/tensorflow/tensorflow/issues/12541? Update: I cannot find the original mention about intrinsics but there was already another very old issue when AVX was still not enabled.

makeandbreak commented 6 years ago

tfboyd

My main use case is learning the functionality provided in AI tool kits (TensorFlow being one of them) for specific problem solving scenarios.

As such the data and model sizes are always a subset of the real world cases.

When I have selected a specific toolkit and created the initial data and model the work will continue on a AVX enabled platforms with also magnificent GPU resources or we will use cloud computing solutions.

I have not found execution times to be unreasonable for the data and models I use.

Restricting the compiled (none AVX) binaries to Python 3 could be considered a reasonable limitation as 2 is in depreciation mode.

When is comes to platforms I would assume Windows 10 64bit and Ubuntu 18.04 LTS 64bit would cover most of the user base on none AVX platforms.

Just to note my 'old' platform is: Intel Core i7 875K, 16 GB DDR3 RAM, NVIDIA GeForce GTX 1050 Ti 4 GB DDR5

It's base performance has not dissapointed me so far so updating it just for AVX is as I see it a waste of money. I would need a new CPU, motherboard and possible memory (DDR4)

tranhungnghiep commented 6 years ago

@tfboyd Thanks for looking into this.

About TF working process, I code up the model on my laptop (AVX). Then test that everything is working well on the CPU server (No AVX, large HDD/RAM, many CPU). Finally run training on a GPU server (shared and scheduled for many experiments).

The alternative build TF on CPU does not need to be fast, just workable is enough.

Pardon me for not knowing much about the building process, but is it possible to change building flags and keep everything else the same?

rotemdan commented 6 years ago

I'm using a several years old CPU without AVX support and currently forced to stay with TF 1.5 for this reason, so I'm not able to benefit from newer updates and bug fixes.

I've got several options (except buying a whole new computer):

  1. Try to compile latest TF from source (I'm using windows) - I'm working on getting to it, but it seems quite challenging (installing updates would be a complicated procedure as well).
  2. Buy a compatible GPU and skip CPU-based training/prediction altogether (??!).

Question: will tensorflow-gpu work on a processor without AVX? (and if it currently will, will non-AVX systems be guaranteed to be supported for the next few years?)

rotemdan commented 6 years ago

Windows users: I've found an unofficial wheel repository that includes a comprehensive collection (seems to be actively maintained) of pre-compiled binaries for many system combinations, including TF 1.8.0/Py3.6/Amd64/SSE2 (No AVX) for both CPU and GPU (I've installed the CPU only version and it seems to work fine so far).

Several other repositories, including some MacOS and Linux builds: https://github.com/mind/wheels (not sure if maintained anymore) https://github.com/lakshayg/tensorflow-build https://github.com/yaroslavvb/tensorflow-community-wheels/issues (various community contributed builds but doesn't cover everything)

makeandbreak commented 6 years ago

https://github.com/fo40225/tensorflow-windows-wheel seems to work. I used 

pip install tensorflow_gpu-1.8.0-cp36-cp36m-win_amd64.whl

whl-file local copy downloaded from

tensorflow-windows-wheel/1.8.0/py36/GPU/cuda91cudnn71sse2/
tfboyd commented 6 years ago

This is really useful information, while words can seem easy at times I want to express that we are working through this for the right solution. What I am hearing is that it is really important to have a build that works anywhere as there are very likely "many" (hard to quantify but many seems fair) users on 6+ year old hardware and having it just work is very important. @tatianashp @martinwicke I helped make this choice and I am having second thoughts for sure and I do not like leaving you without a solution. I do not believe there are huge changes in versions, so I hope you will user a slightly older version while we sort everything out.

tfboyd commented 6 years ago

Adding @tatianashp as an owner for visibility.

hadim commented 6 years ago

Thank you for looking into this. Are you guys aware of similar builds for Linux? I know I could compile it myself but that would spare me one day of work...

tatianashp commented 6 years ago

@tfboyd I agree that we need a better solution. For now, the only choices for those with old CPUs are to use community supported build, build from source or use an older version of TensorFlow.

@hadim Building from source on Linux is not as difficult as it might seem and well documented. Please give a try if nobody from the community volunteers a pre-build wheel. Let us know if you have any questions.

hadim commented 6 years ago

I didn't say it was complicated, just that I would have to spend some time on it while a simple pip install would have been done in a minute :-)

tranhungnghiep commented 6 years ago

What if a user on linux cannot sudo? Can they compile tf locally?

hadim commented 6 years ago

Userspace compilation should be possible.

makeandbreak commented 6 years ago

I was looking at a few other AI toolkits to see if they require AVX support from the CPU. I could not find this to be the case. For example https://www.microsoft.com/en-us/cognitive-toolkit/ (CNTK) latest version works fine on my system.

bhack commented 6 years ago

I think that also pytorch has as SIMD dispatcher. In the updated TF Roadmap we have:

Dynamic loading of SIMD-optimized kernels

That was add on 22th Feb

tatianashp commented 6 years ago

We also have "MKL for Linux and Windows" in the roadmap. The build will be SSE-based and will use MKL dynamic SIMD dispatch to leverage AVX or AVX2 when it's present. This is likely to happen before dynamic loading of SIMD-optimized kernels.

sunils27 commented 6 years ago

I would like to ask the same question @rotemdan asked above i.e. "will tensorflow-gpu work on a processor without AVX?" I'm running into the issue where I have a new GPU (gtx 1060) but an old CPU (6+ years old). I'm stuck at TF 1.5. Is there a way to get 1.7 or 1.8 to work?

martinwicke commented 6 years ago

Yes, you should be able to install from source without trouble, assuming you have the dependencies installed (all called out in the installation guide).

makeandbreak commented 6 years ago

"will tensorflow-gpu work on a processor without AVX?"

I have tested TensorFlow 1.8 GPU on windows and it seems AVX code is in the base code. So in other words no it does not work.

hadim commented 6 years ago

FYI, here is a Docker image that can build TensorFlow https://github.com/hadim/docker-tensorflow-builder. It can help to compile TF on a wide range of configurations as long as you have Docker installed on it.

mosheliv commented 6 years ago

Just adding my 2c to the thread... The laptop i use for the initial check of the code before running it on the cloud has ssd and not enough space to build tf. It's uncomplicated build, yes... But huge.

maxhgerlach commented 6 years ago

I built Tensorflow 1.8 on a GPU-less Ubuntu box without AVX support. Maybe this wheel is useful for some of you: repo

mosheliv commented 6 years ago

Thank you! I actually did the same a few days ago, with external usb drive. Bazel was not impressed... But, after a short fight, i managed to make a wheel. It's on my github repos. Thank you again for your offer. I still think tf should provide such no strings attached wheel.

belarm commented 6 years ago

If you're not going to support non-AVX, at least improve your checks or errors. "Illegal instruction" in response to "import keras" is not helpful.

Lafaiet commented 6 years ago

I built Tensorflow 1.8 on a GPU-less Ubuntu box without AVX support. Maybe this wheel is useful for some of you: repo

This worked great on my old xeon E5507. Thanks a bunch!!!!!

bzamecnik commented 5 years ago

Confirming that TF 1.8 CPU build without AVX from @maxhgerlach works on Intel Pentium G4400 (Skylake) without AVX.

Anakeyn commented 4 years ago

I built a Tensorflow 2.0 version running with old CPUs not compatible with AVX instructions nor SSE. See here on my Github : https://github.com/Anakeyn/Tensorflow2.0ForOldComputers. Rem : this is a Linux version !!! I tested it under Windows 10 WSL Ubuntu 18.04 LTS on a Lenovo PR-G700 with Intel Celeron CPU 1000M 1,80 Ghz (2013 CPU) and directly under Linux Ubuntu 18.04.3 LTS on an HP Pavillon DV6 with Intel Core 2 Duo CPU T6600 2.2 GHz x 2 (2009 CPU) and it works.

rtrevinnoc commented 4 years ago

I built a Tensorflow 2.0 version running with old CPUs not compatible with AVX instructions nor SSE. See here on my Github : https://github.com/Anakeyn/Tensorflow2.0ForOldComputers. Rem : this is a Linux version !!! I tested it under Windows 10 WSL Ubuntu 18.04 LTS on a Lenovo PR-G700 with Intel Celeron CPU 1000M 1,80 Ghz (2013 CPU) and directly under Linux Ubuntu 18.04.3 LTS on an HP Pavillon DV6 with Intel Core 2 Duo CPU T6600 2.2 GHz x 2 (2009 CPU) and it works.

Thank you man! I was trying to update tensorflow with a source build, but accidentally broke my server while trying. So I was trying to find precompiled binaries and I am glad I found yours!

Anakeyn commented 4 years ago

I built a Tensorflow 2.0 version running with old CPUs not compatible with AVX instructions nor SSE. See here on my Github : https://github.com/Anakeyn/Tensorflow2.0ForOldComputers. Rem : this is a Linux version !!! I tested it under Windows 10 WSL Ubuntu 18.04 LTS on a Lenovo PR-G700 with Intel Celeron CPU 1000M 1,80 Ghz (2013 CPU) and directly under Linux Ubuntu 18.04.3 LTS on an HP Pavillon DV6 with Intel Core 2 Duo CPU T6600 2.2 GHz x 2 (2009 CPU) and it works.

Thank you man! I was trying to update tensorflow with a source build, but accidentally broke my server while trying. So I was trying to find precompiled binaries and I am glad I found yours!

You're welcome! Thank you for your feedback!

anavc97 commented 3 years ago

I built a Tensorflow 2.0 version running with old CPUs not compatible with AVX instructions nor SSE. See here on my Github : https://github.com/Anakeyn/Tensorflow2.0ForOldComputers. Rem : this is a Linux version !!! I tested it under Windows 10 WSL Ubuntu 18.04 LTS on a Lenovo PR-G700 with Intel Celeron CPU 1000M 1,80 Ghz (2013 CPU) and directly under Linux Ubuntu 18.04.3 LTS on an HP Pavillon DV6 with Intel Core 2 Duo CPU T6600 2.2 GHz x 2 (2009 CPU) and it works.

@Anakeyn, I tried to install tensorflow following your instructions and it seems to be installed ("import tensorflow" gives no errors) but it seems like for some reason the tensorflow has no attributes installed at all. I ran multiple attributes (tf.test, tf.constant, tf.session) and all of them returned the error: " module 'tensorfow' has no attribute ..".

Did you have to manually install anything else after that? Am I missing something? Thanks!

Anakeyn commented 3 years ago

@anavc97 It seems that you are using tensorflow 1.x syntax. Some modules were removed in version 2. x See last documentation for tensorflow 2 : https://www.tensorflow.org/api_docs/python/tf/version Check also how to migrate from tensorflow 1 to 2 https://www.tensorflow.org/guide/migrate You can also running tensorflow 1.x code with the following instructions :

import tensorflow.compat.v1 as tf tf.disable_v2_behavior()

anavc97 commented 3 years ago

@anavc97 It seems that you are using tensorflow 1.x syntax. Some modules were removed in version 2. x See last documentation for tensorflow 2 : https://www.tensorflow.org/api_docs/python/tf/version Check also how to migrate from tensorflow 1 to 2 https://www.tensorflow.org/guide/migrate You can also running tensorflow 1.x code with the following instructions :

import tensorflow.compat.v1 as tf tf.disable_v2_behavior()

@Anakeyn Thank you for your reply. It didn't recognize compat either, actually. I fixed it by moving the folders in the zip to the site-packages directory of python. Unfortunately I'm having trouble using the TF2.0 instalation with GPU. It's not recognizing my device, even though I installed the compatible CUDA and Cudnn versions (CUDA: 10.0, Cudnn: 7.4). I'm currently debugging this, but there's not much information except it returns "False" when I run

import tensorflow as tf print(tf.test.is_gpu_available()) print(tf.test.is_built_with_cuda())

I don't suppose you have, by any chance, information that might help me?

Anakeyn commented 3 years ago

@anavc97 This version is not compatible with GPU ! I didn't build it with GPU options ! This is a version for VERY old computers :-) Please take a look at this page to see how to build a version with GPU options : https://www.tensorflow.org/install/source

anavc97 commented 3 years ago

@anavc97 This version is not compatible with GPU ! I didn't build it with GPU options ! This is a version for VERY old computers :-) Please take a look at this page to see how to build a version with GPU options : https://www.tensorflow.org/install/source

@Anakeyn thanks for the input! I thought any version 2 of TF was compatible with GPU, my mistake. I'll try to build TF from source :)

IFFranciscoME commented 3 years ago

@tatianashp @av8ramit @tfboyd it seems that, as members of TensorFlow, you are trying to justify the AVX generalization and that there is a lack of evidence for rejecting the idea "everybody that cares about good performance has AVXs ready CPUs" when you do not have enough evidence to prove it either. Have you ever considered ALL THE OTHER users/practitioners that stumble upon this AVX exclusivity and did not leave a message here? (let alone try to build it from source).

I will add my part then. I work at a university in Mexico and we have a 64 threads CPU IBM cluster, we use it for research and teaching in undergraduate and graduate courses. Only a few universities in Mexico have the resources to buy a physical server or to grant a budget for cloud computing, and so many countries in the world are in the same general position or worst.

I do appreciate the effort and maintenance of the library, but your rhetoric about "generalization" is the one I strongly disagree with. (and just to be clear, I'm having problems since the server I'm using does not support AVX)

av8ramit commented 3 years ago

Hello @IFFranciscoME. I would first like to apologize to you for the difficulties you have been having. We made this decision in 2018 with a lot of different factors in mind. I definitely did not make this decision singlehandedly, and I was just one of the first to respond to the issue. Obviously that decision was not perfect, and I'm sure there were users without AVX that had to turn to building from source.

That being said if you'd like to continue this discussion I'd like to invite you to the Build Special Interest Group where we discuss different TensorFlow build topics, configurations, and solutions to problems similar to this one.