Closed antogerva closed 8 years ago
The PROTOBUF_DEPRECATED_ATTR
macro only exists in earlier versions of Protobuf; it has recently been changed to GOOGLE_PROTOBUF_DEPRECATED_ATTR
. However, TensorFlow requires this earlier version and makes references to PROTOBUF_DEPRECATED_ATTR
. The protobuf.sh
script scans the tensorflow repository for references to Protobuf, trying to find the commit hash to reset to. The external project builds and runs correctly in my environment (Ubuntu 16.04 LTS), but I will attempt to reproduce using your system.
The <Source_DIR>
issue is unrelated and stems from a small error in Protobuf.cmake
, which I will soon correct.
I believe I have resolved this problem.
There was an error in the CMake modules for the external project; the external/include
directory was not being included. This caused the project to search for the required files in either /usr/local/include
or /usr/include
. If you had a different version of profobuf installed on the machine, those header files were used instead, and this resulted in the PROTOBUF_DEPRECATED_ATTR
macro not being found.
I made some minor changes to the CMake logic to include the required directories. I tested the fix on my Ubuntu 16.04 LTS machine as well as a Docker container using Ubuntu 14.04.4 LTS; they both worked as expected.
@antogerva, please pull the latest version of this repository and test this yourself.
Ok, I pulled the latest version and now it works. I have been able to run the example project correctly. So, I'm closing this issue.
That being said, in the CLion IDE, if I try to execute the "external-project" target, I get the expected result if I copy the graph.pb in the bin folder(and with additional casting on few call over the Tensorflow API). Thought, if I try to debug the "external-project" target, it won't break on any line with breakpoint. Maybe I'm missing a debug flag somewhere? It seem the breakpoints are ignored if I run the project directly with gdb. I tried using cmake -DCMAKE_BUILD_TYPE=Debug ..
but look like this isn't enough.
Also, I believe there's no "fast build" target directly available from cmake. So, every time I try to build the "external-project" target, Eigen and Protobuf get rebuild as well.
In any case, I'll probably do more tests on the matter later this week and if I'm stuck, I'll create a new issue about these problems.
The C++ file in the example projects is quite naive; it tries to open graph.pb
in the current directory. The default behavior of CLion is to execute the program from the bin
directory, which forced you to copy the graph file. Simply changing the working directory in the build configuration works as well:
I debugged the external project with CLions's built in gdb (7.11.1). The program did stop at each breakpoint, although it was slow at times. I am currently using CLion 2016.2 (build #CL-162-1236-16). If you find any more information as to the source of your debugger problems, let me know.
This is the problem with the External Project model as opposed to installing to a local directory. However, once CMake has built Eigen and Protobuf the first time, the process should be much faster in future builds. For Eigen, CMake sees that it has already downloaded and built Eigen, and since the hashes are the same, no action is taken. Protobuf is slightly different; CMake first realizes that it has already been downloaded and the commit hash is the one requested. CMake still, however, performs the configure step, which takes about 10 - 30 seconds to complete. Once the configure step is finished, Protobuf is then rebuilt, although this takes almost no time at all, since it has already been compiled.
The first build of the external project can take anywhere from 15 minutes to an hour, depending on the speed of your computer. However, subsequent builds should take less than a minute, as Protobuf and Eigen have already been compiled (but the configure step for Protobuf is still executed, which does take time).
Is this the behavior you are experiencing?
Thanks for the detailed answer. Now about the graph.pb
, indeed the workspace was the issue, so there's no need to copy the graph.pb
file. My bad about that.
Regarding the debugger, I have exactly the same version/build-version CLion and gdb. Yet when I go to Run > Debug 'external-project'
, it simply won't break on any of my breakpoints.
Furthermore, I get the same result from /usr/bin/gdb
(still version 7.7.1, on my ubuntu system). Here's an attempt to break on line 19:
$ gdb ./bin/external-project -q
Reading symbols from ./bin/external-project...done.
(gdb) break 19
Breakpoint 1 at 0x409fc5: file /root/git/tensorflow-cmake/examples/external-project/main.cc, line 19.
(gdb) info break
Num Type Disp Enb Address What
1 breakpoint keep y 0x0000000000409fc5 in main(int, char**) at /root/git/tensorflow-cmake/examples/external-project/main.cc:19
(gdb) run
Starting program: /root/git/tensorflow-cmake/examples/external-project/bin/external-project
warning: Error disabling address space randomization: Operation not permitted
Success: 42!
During startup program exited normally.
(gdb)
So as you can see, during this session, I get the result output directly without breaking on any breakpoints. I've also tried with other line numbers to break without success. (note: the warning about address space randomization can be fix with set disable-randomization off
within gdb so this isn't relevant).
My guess is the makefile generated by cmake fail to produce the correct symbols, yet I'm not sure how to deal with this issue. One possible idea might be to add a dbg
flag to bazel, such as bazel build -c dbg tensorflow:libtensorflow_all.so
, but if you didn't need to do this, then I'll need to figure out why your setup actually works.
Finally, about the fast builds, you're right. Once the first build is done, the next builds simply do a small reconfigure task and the amount of time taken is reasonable.
If your example of command line gdb is on your Docker container, have you tried running the container with the --privileged
flag? I encountered your exact error at first:
# 'tfcf' is an Ubuntu 14.04.2 LTS image with TensorFlow and tensorflow-cmake
$ sudo docker run -t -i cjweeks/tfcf
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bin/external-project...done.
(gdb) break 19
Breakpoint 1 at 0x409fc5: file /root/git/tensorflow-cmake/examples/external-project/main.cc, line 19.
(gdb) info break
Num Type Disp Enb Address What
1 breakpoint keep y 0x0000000000409fc5 in main(int, char**)
at /root/git/tensorflow-cmake/examples/external-project/main.cc:19
(gdb) run
Starting program: /root/git/tensorflow-cmake/examples/external-project/bin/external-project
warning: Error disabling address space randomization: Operation not permitted
Success: 42!
During startup program exited normally.
(gdb) quit
But after inserting the flag, I got the expected result:
$ sudo docker run --privileged -t -i cjweeks/tfcf
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from bin/external-project...done.
(gdb) break 19
Breakpoint 1 at 0x409fc5: file /root/git/tensorflow-cmake/examples/external-project/main.cc, line 19.
(gdb) info break
Num Type Disp Enb Address What
1 breakpoint keep y 0x0000000000409fc5 in main(int, char**)
at /root/git/tensorflow-cmake/examples/external-project/main.cc:19
(gdb) run
Starting program: /root/git/tensorflow-cmake/examples/external-project/bin/external-project
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, main (argc=1, argv=0x7fffffffed08)
at /root/git/tensorflow-cmake/examples/external-project/main.cc:20
20 tf::Status status = tf::NewSession(tf::SessionOptions(), &session);
(gdb) continue
Continuing.
[New Thread 0x7fffef586700 (LWP 12)]
[New Thread 0x7fffeed85700 (LWP 13)]
[New Thread 0x7fffee584700 (LWP 14)]
[New Thread 0x7fffedd83700 (LWP 15)]
[New Thread 0x7fffed582700 (LWP 16)]
[New Thread 0x7fffecd81700 (LWP 17)]
[New Thread 0x7fffec580700 (LWP 18)]
[New Thread 0x7fffebd7f700 (LWP 19)]
[New Thread 0x7fffeb57e700 (LWP 20)]
Success: 42!
[Thread 0x7fffebd7f700 (LWP 19) exited]
[Thread 0x7fffec580700 (LWP 18) exited]
[Thread 0x7fffecd81700 (LWP 17) exited]
[Thread 0x7fffed582700 (LWP 16) exited]
[Thread 0x7fffedd83700 (LWP 15) exited]
[Thread 0x7fffee584700 (LWP 14) exited]
[Thread 0x7fffeed85700 (LWP 13) exited]
[Thread 0x7fffef586700 (LWP 12) exited]
[Thread 0x7ffff7feb780 (LWP 8) exited]
[Inferior 1 (process 8) exited normally]
(gdb) quit
Let me know if this fixes your problem. Regarding CLion's debugger, I really have no idea, especially since mine works perfectly well with the same versions.
Wow, I had no idea about the --privileged
flag. Indeed this the fixed my issue with debugging! Thanks!
No problem!
First, I found this project thanks to this post on stackoverflow.
Environment info
OS: I'm running on Docker with an Ubuntu 14.04.4 LTS x86_64 (gcc/g++ version is 4.8.4) No CUDA or cuDNN installed
Please note, I already managed to compile Tensorflow with Bazel before using this tutorial.
I tried to run the examples with both branch master and r0.10 from tensorflow and I got the same issue. Here's what I did:
Then it fail, here's the last output of the
make
command.Look there's an error with the
set_version
function. You can see the full output over there.Furthermore, there's an empty
<SOURCE_DIR>
folder inside the build folder:This look a bit wrong... shouldn't the
<SOURCE_DIR>
point to a project source directory? Maybe there's a configuration missing with those.cmake
files. I did this a quick search but I'm unsure about what to do exactly in order to fix the issue:Thought, maybe this isn't directly relevant to the
set_version
problem.That being said, for the record my main goal is to debug
tensorflow-cmake/examples/external-project/main.cc
with the CLion debugger... I'm not there yet, but if I can get the cmake setup to works, I could probably do some progress about that.